Earlier this week I had the privilege of speaking at ApacheCon in Austin, TX on the topic of data management for apps that work as well offline as they do online. This is an important topic for mobile apps, since, as we all painfully know already, there is never a case when you are always online on your mobile devices. There always ends up being a time when you need your device/app, but you can’t get online to get the information you need. Well, this doesn’t always have to be the case. There are strategies you can employ to build apps that work just as well offline as they do online, and the strategy I’d like to highlight today is based upon data management using the IBM Cloudant NoSQL database as a service, which is based upon Apache CouchDB.
Here’s a link to the presentation slides (built using reveal.js) – just use the space bar to advance the presentation slides:
The “couch” in CouchDB is actually an acronym for Cluster of Unreliable Commodity Hardware. At the core of this cluster is the concept of replication, which in the most basic of terms means that data is shared between multiple sources. Replication is used to share information between nodes of the cluster, which provides for cluster reliability and fault tolerance.
If you’d like to learn more about replication in Cloudant and CouchDB, you can read more using the links below:
Cloudant is a clustered NoSQL database services that provides an extremely powerful and searchable data store. It is designed to power the web and mobile apps, and all information is exposed via REST services. Since the IBM Cloudant service is based on CouchDB (and not so coincidentally, IBM is a major contributor to the CouchDB project), replication is also core the the Cloudant service.
With replication, you only have to write your data/changes to a single node in the cluster, and replication takes care of propagating these changes across the cluster.
If you are building apps for the web or mobile, there are options to extend the data replication locally either on the device or in the browser. This means that you can have a local data store that automatically pushes and/or pulls data from the remote store using replication, and it can be done either via native languages, or using JavaScript.
If you want to have local replication in either a web or hybrid (Cordova/PhoneGap) app, you can use PouchDB. PouchDB is a local JavaScript database modeled after CouchDB and implements that CouchDB replication API. So, you can store your data in the browser’s local storage, and those changes will automatically be replicated to the remote Cloudant store. This works in the browser, in a hybrid (web view) app, or even inside of a Node.js instance. Granted, if you’re in-browser you’ll need to leverage the HTML5 cache to have your app cached locally.
If you are building a native app, don’t worry, you can take advantage of the Cloudant Sync API to leverage the local data store with replication. This is available for iOS and Android, and implements the CouchDB replication API.
The sample app that I showed in the presentation is a native iOS application based on the GeoPix MobileFirst sample app that I detailed in a previous post. The difference is that in this case I showed it using the Cloudant Sync API, instead of the MobileFirst data wrapper classes, even though it was pointing at the exact same Cloudant database instance. You can see a video of the app in action below.
All that you have to do is create a local data store instance, and then use replication to synchronize data between the local store and a remote store.
Replication be either one-way (push or pull), or two-way. So, any changes between the local and remote stores are replicated across the cluster. Essentially, the local data store just becomes a node in the cluster. This provides complete access to the local data, even if there is no network available. Just save your data to the local store, and replication takes care of the rest.
In the native Objective-C code, you just need to setup the CDTDatastore manager, and initialize your datastore instance.
[objc]self.manager = [[CDTDatastoreManager alloc] initWithDirectory:path error:nil];
self.datastore = [self.manager datastoreNamed:@"geopix" error:nil];[/objc]
Once your datastore is created, you can read/write/modify any data in the local store. In this case I am creating a generic data object (basically like a JSON object), and creating a document containing this data. A document is a record within the data store.
You can add attachments to the document or modify the document as your app needs. In the code below, I add a JPG atttachment to the document.
[objc]//create a document revision
CDTMutableDocumentRevision *rev = [CDTMutableDocumentRevision revision];
rev.body = @{
@"sort": [NSNumber numberWithDouble:[now timeIntervalSince1970]],
@"clientDate": dateString,
@"latitude": [NSNumber numberWithFloat:location.coordinate.latitude],
@"longitude": [NSNumber numberWithFloat:location.coordinate.longitude],
@"altitude": [NSNumber numberWithFloat:location.altitude],
@"course": [NSNumber numberWithFloat:location.course],
@"type": @"com.geopix.entry"
};
//add the jpg attachment
NSData *imageData = UIImageJPEGRepresentation(image, 0.1);
[imageData writeToFile:imagePath atomically:YES];
CDTUnsavedFileAttachment *att1 = [[CDTUnsavedFileAttachment alloc]
initWithPath:imagePath
name:imageName
type:@"image/jpeg"];
rev.attachments = @{ imageName: att1 };
//create a new document from the revision
NSError *error = nil;
CDTDocumentRevision *doc = [self.datastore createDocumentFromRevision:rev error:&error];
if (doc == nil) {
[logger logErrorWithMessages:@"Error creating document: %@", error.localizedDescription];
}
[logger logDebugWithMessages:@"Document created ID: %@", doc.docId];[/objc]
Replication is a fire-and-forget process. You simply need to initialize the replication process, and any changes to the local data store will be replicated to the remote store automatically when the device is online.
[objc]//initialize the replicator factory with the local data store manager
CDTReplicatorFactory *replicatorFactory =
[[CDTReplicatorFactory alloc] initWithDatastoreManager:self.manager];
NSURL *remoteDatabaseURL = [NSURL URLWithString:REMOTE_DATABASE_URL];
//setup push replication for local->remote changes
NSError *error = nil;
CDTPushReplication *pushReplication =
[CDTPushReplication replicationWithSource:self.datastore target:remoteDatabaseURL];
//create the replicator instance
self.replicator = [replicatorFactory oneWay:pushReplication error:&error];
if (!self.replicator) {
[logger logErrorWithMessages:@"An error occurred: %@", error.localizedDescription];
}
//assign the replicator delegate
self.replicator.delegate = self;
//auto start replication
error = nil;
if (![self.replicator startWithError:&error]) {
[logger logErrorWithMessages:@"An error occurred: %@", error.localizedDescription];
}[/objc]
By assigning a replicator delegate class (as shown above), your app can monitor and respond to changes in replication state. For example, you can update status if replication is in progress, complete, or if an error condition was encountered.
[objc]- (void)replicatorDidChangeState:(CDTReplicator *)replicator {
[logger logDebugWithMessages:@"Replicator changed State: %@", [CDTReplicator stringForReplicatorState:replicator.state]];
}
– (void)replicatorDidChangeProgress:(CDTReplicator *)replicator {
[logger logDebugWithMessages:@"Replicator progress: %d/%d", replicator.changesProcessed, replicator.changesTotal];
NSDictionary *userInfo = @{ @"status":[NSString stringWithFormat:@"%d/%d", replicator.changesProcessed, replicator.changesTotal] };
[[NSNotificationCenter defaultCenter]
postNotificationName:@"ReplicationStatus"
object:self
userInfo:userInfo];
}
– (void)replicatorDidError:(CDTReplicator *)replicator info:(NSError *)info {
[logger logErrorWithMessages:@"An error occurred: %@", info.localizedDescription];
self.replicator = nil;
[[NSNotificationCenter defaultCenter]
postNotificationName:@"ReplicationError"
object:self];
}
– (void)replicatorDidComplete:(CDTReplicator *)replicator {
[logger logDebugWithMessages:@"Replication completed"];
self.replicator = nil;
[[NSNotificationCenter defaultCenter]
postNotificationName:@"ReplicationComplete"
object:self];
}[/objc]
If you want to access data from the local store, it is always available within the app, regardless of whether or not the device has an active internet connection. For example, this method will return all documents within the local data store.
[objc]-(NSArray*) getLocalData {
NSArray *docs = [self.datastore getAllDocuments];
return docs;
}[/objc]
Be sure to review the documentation and/or Cloudant Synch API source code for complete details.