After spending some time playing around sketching with the HTML5 canvas element earlier this week, I figured “why not add some ‘enterprise’ concepts to this example?”… Next thing you know we’ve got a multi-device shared sketching/collaboration experience.
To keep things straightforward, I chose to demonstrate the near-realtime collaboration using a short-interval HTTP poll. HTTP polling is probably the simplest form of near-realtime data in web applications, however you may experience lag when compared to a socket connection of equivalent functionality. I’ll discuss the various realtime data options you have in Flex/Flash and HTML/JS and their pros & cons further in this post.
What you’ll see in the video below is the sketching example with realtime collaboration added using short-interval data polling of a ColdFusion application server. The realtime collaboration is shown between an iPad 2, a Kindle Fire, and a Macbook Pro.
Before we get into the code for this example, let’s first review some realtime data basics…
First, why/when would you need realtime data in your applications? Here are just a few:
- Time sensitive information, where any delay could have major repercussions
- Realtime financial information
- Emergency services (medical, fire, police)
- Military/Intelligence scenarios
- Business critical efficiency/performance metrics
- Collaboration
- Realtime audio/video collaboration
- Shared experience (presentations/screen sharing)
- Entertainment
- Streaming media (audio/video)
- Gaming
Regardless of whether you are building applications for mobile, the web, or desktop, using any technology (Flex/Flash, HTML/JS, Java, .NET, Objective C, or C/C++ (among others)), there are basically 3 methods for streaming/realtime data:
- Socket Connection
- HTTP Polling
- HTTP Push
Socket Connections
Socket connectionss are basically end-to-end communications channels between two computer processes. Your computer (a client) connects to a server socket and establishes a persistent connection that is used to pass data between the client and server in near-realtime. Persistent socket connections are generally based upon TCP or UDP and enable asynchronus bidirectional communication. Binary or Text-based messages can be sent in either direction at any point in time, in any sequence, as data is available. In HTML/JS applications you can use web sockets, which I recently discussed, or use a plugin that handles realtime socket communication. Did you also know that the next version of ColdFusion will even have web socket support built in? In Flash/Flex/AIR, this can be achieved using the RTMP protocol (LCDS, Flash Media Server, etc…) or raw sockets (TCP or UDP).
In general, direct socket based communication is the most efficient means of data transfer for realtime application scenarios. There is less back and forth handshaking and less packet encapsulation required by various protocols (HTTP, etc…), and you are restricted by fewer network protocol rules. However, socket based communications often run on non-standard or restricted ports, so they are more likely to be blocked by IT departments or stopped by network firewalls. If you are using socket based communication within your applications, which are running on non-standard ports, and you don’t govern the network, you may want a fallback to another realtime data implementation for failover cases.
HTTP Polling
HTTP Polling is the process of using standard HTTP requests to periodically check for data updates on the server. The client application requests information from the server. Generally, the client will send a timestamp indicating the last data update time. If there is information available on the server that is newer than the timestamp, that data will be immediately sent back to the client (and the client’s timestamp will be updated). After a period of time, another request will be made, and so forth until the polling is stopped within the application. Using this approach, the application is more-or-less “phoning home” periodically to the server to see if there are any updates. You can achieve near-realtime performance by setting a very short polling interval (less than one second).
HTTP polling uses standard web protocols and ports, and generally will not be blocked by firewalls. You can poll on top of standard HTTP (port 80) or HTTPS (port 443) without any issue. This can be achieved by polling JSON services, XML Services, AMF, or any other data format on top of a HTTP request. HTTP polling will generally be slower than a direct socket method, and will also utilize more network bandwidth b/c of request/response encapsulation and the periodic requests to the server. It is also important to keep in mind that the HTTP spec only allows for 2 concurrent connections to a server at any point in time. Polling requests can consume HTTP connections, thus slowing load time for other portions of your application. HTTP polling can be employed in HTML/JS, Flex/Flash/AIR, desktop, server, or basically any other type of application using common libraries & APIs.
HTTP Push
HTTP Push technologies fall into 2 general categories depending upon the server-side technology/implementation. This can refer to HTTP Streaming, where a connection is opened between the client and server and kept open using keep-alives. As data is ready to send to the client, it will be pushed across the existing open HTTP connection. HTTP Push can also refer to HTTP Long Polling, where the client will periodically make a HTTP request to the server, and the server will “hold” the connection open until data is available to send to the client (or a timeout occurs). Once that request has a complete response, another request is made to open another connection to wait for more data. Once Again, with HTTP Long Poll there should be a very short polling interval to maintain near-realtime performance, however you can expect some lag.
HTTP Streaming & HTTP Long polling can be employed in HTML/JS applications using the Comet approach (supported by numerous backend server technologies) and can be employed in Flex/Flash/AIR using BlazeDS or LCDS.
Collaborative Applications
Now back to the collaborative sketching application shown in the video above… the application builds off of the sketching example from previous blog posts. I added logic to monitor the input sketches and built a HTTP poll-based monitoring service to share content between sessions that share a common ID.
In the JavaScript code, I created an ApplicationController class that acts as an observer to the input from the Sketcher class. The ApplicationController encapsulates all logic handling data polling and information sharing between sessions. When the application loads, it sets up the polling sequence.
The polling sequence is setup so that a new request will be made to the server 250MS after receiving a response from the previous request. Note: this is very different from using a 250MS interval using setInterval. This approach guarantees 250MS from response to the next request. If you use a 250MS interval using setInterval, then you are only waiting 250MS between each request, without waiting for a response. If your request takes more than 250 MS, you will can end up have stacked, or “concurrent” requests, which can cause serious performance issues.
When observing the sketch input, the start and end positions and color for each line segment get pushed into a queue of captured transactions that will be pushed to the server. (The code supports multiple colors, even though there is no method to support changing colors in the UI.)
[js]ApplicationController.prototype.observe = function(start, end, color) {
this.capturedTransactions.push( {"sx":start.x, "sy":start.y, "ex":end.x, "ey":end.y, "c":color} );
}[/js]
When a poll happens, the captured transactions are sent to the server (a ColdFusion CFC exposed in JSON format) as a HTTP post.
[js]ApplicationController.prototype.poll = function () {
this.pendingTransactions = this.capturedTransactions;
this.capturedTransactions = [];
var data = { "method":"synchronize",
"id":this.id,
"timestamp":this.lastTimeStamp,
"transactions": JSON.stringify(this.pendingTransactions),
"returnformat":"json" };
var url = "services/DataPollGateway.cfc";
$.ajax({
type: ‘POST’,
url: url,
data:data,
success: this.getRequestSuccessFunction(),
error: this.getRequestErrorFunction()
});
}[/js]
The server then stores the pending transactions in memory (I am not persisting these, they are in-ram on the server only). The server checks the transactions that are already in memory against the last timestamp from the client, and it will return all transactions that have taken place since that timestamp.
[cf]<cffunction name="synchronize" access="public" returntype="struct">
<cfargument name="id" type="string" required="yes">
<cfargument name="timestamp" type="string" required="yes">
<cfargument name="transactions" type="string" required="yes">
<cfscript>
var newTransactions = deserializeJSON(transactions);
if( ! structkeyexists(this, "id#id#") ){
this[ "id#id#" ] = ArrayNew(1);
}
var existingTransactions = this[ "id#id#" ];
var serializeTransactions = ArrayNew(1);
var numberTimestamp = LSParseNumber( timestamp );
//check existing tranactions to return to client
for (i = 1; i lte ArrayLen(existingTransactions); i++) {
var item = existingTransactions[i];
if ( item.timestamp GT numberTimestamp ) {
ArrayAppend( serializeTransactions, item.content );
}
}
var newTimestamp = GetTickCount();
//add new transactions to server
for (i = 1; i lte ArrayLen(newTransactions); i++) {
var item = {};
if ( structkeyexists( newTransactions[i], "clear" )) {
serializeTransactions = ArrayNew(1);
existingTransactions = ArrayNew(1);
}
item.timestamp = newTimestamp;
item.content = newTransactions[i];
ArrayAppend( existingTransactions, item );
}
var result = {};
result.transactions = serializeTransactions;
result.timestamp = newTimestamp;
this[ "id#id#" ] = existingTransactions;;
</cfscript>
<cfreturn result>
</cffunction>[/cf]
When a poll request completes, any new transactions are processed and a new poll is requested.
[js]ApplicationController.prototype.getRequestSuccessFunction = function() {
<pre> var self = this;
return function( data, textStatus, jqXHR ) {
var result = eval( "["+data+"]" );
if ( result.length > 0 )
{
var transactions = result[0].TRANSACTIONS;
self.lastTimeStamp = parseInt( result[0].TIMESTAMP );
self.processTransactions( transactions );
}
self.pendingTransactions = [];
self.requestPoll();
}
}[/js]
You can access the full client and server application source on Github at:
I used ColdFusion in this example, however the server side could be written in any server-side language… Java, PHP, .NET, etc…
If you were building this application using web sockets, you could simply push the data across the socket connection without the need for queueing.