iOS Framework: Introducing MKNetworkKit

iOS Framework: Introducing MKNetworkKit:

How awesome would it be if a networking framework automatically takes care of caching responses for you?

How awesome would it be if a networking framework automatically remembers your operations when your client is offline?

You favorite a tweet or mark a feed as read when you are offline and the Networking Framework performs all these operations when the device comes back online, all with no extra coding effort from you. Introducing MKNetworkKit.

What is MKNetworkKit?

MKNetworkKit is a networking framework written in Objective-C that is seamless, block based, ARC ready and easy to use.

MKNetworkKit is inspired by the other two popular networking frameworks, ASIHTTPRequest and AFNetworking. Marrying the feature set from both, MKNetworkKit throws in a bunch of new features. In addition to this, MKNetworkKit mandates you to write slightly more code than the other frameworks at the expense of code clarity. With MKNetworkKit, it’s hard to write ugly networking code.

Features

Super light-weight

The complete kit is just 2 major classes and some category methods. This means, adopting MKNetworkKit should be super easy.

Single Shared Queue for your entire application.

Apps that depend heavily on Internet connectivity should optimize the number of concurrent network operations. Unfortunately, there is no networking framework that does this correctly. Let me give you an example of what can go wrong if you don’t optimize/control the number of concurrent network operations in your app.

Let’s assume that you are uploading a bunch of photos (think Color or Batch) to your server. Most mobile networks (3G) don’t allow more than two concurrent HTTP connections from a given IP address. That is, from your device, you cannot open more than two concurrent HTTP connections on a 3G network. Edge is even worse. You can’t, in most cases, open more than one connection. This limit is considerably high (six) on a traditional home broadband (Wifi). However, since your iDevice is not always connected to Wifi, you should be prepared for throttled/restricted network connectivity. On any normal case, the iDevice is mostly connected to a 3G network, which means, you are restricted to upload only two photos in parallel. Now, it is not the slow upload speed that hurts. The real problem arises when you open a view that loads thumbnails of photos (say on a different view) while this uploading operations are running in the background. When you don’t properly control the queue size across the app, your thumbnail loading operations will just timeout which is not really the right way to do it. The right way to do this is to prioritize your thumbnail loading operation or wait till the upload is complete and the load thumbnails. This requires you to have a single queue across the entire app. MKNetworkKit ensures this automatically by using a single shared queue for every instance of it. While MKNetworkKit is not a singleton by itself, the shared queue is.

Showing the Network Activity Indicator correctly

While there are many third party classes that uses “incrementing” and “decrementing” the number of network calls and using that to show the network activity indicator, MKNetworkKit backs on the single shared queue principle and shows the activity indicator automatically when there is an operation running in the shared queue by observing (KVO) the operationCount property. As a developer, you normally don’t have to worry about setting the network activity indicator manually, ever again.

if (object == _sharedNetworkQueue && [keyPath isEqualToString:@"operationCount"]) {

[UIApplication sharedApplication].networkActivityIndicatorVisible =
([_sharedNetworkQueue.operations count] > 0);       
}

Auto queue sizing

Continuing the previous discussion, I told that most mobile networks don’t allow more than two concurrent connections. So your queue size should be set to two, when the current network connectivity is 3G. MKNetworkKit automatically handles this for you. When the network drops to 3G/EDGE/GPRS, it changes the number of concurrent operations that can be performed to 2. This is automatically changed back to 6 when the device connects back to a Wifi network. With this technique in place, you will see a huge performance benefit when you are loading thumbnails (or multiple similar small requests) for a photo library from a remote server over 3G.

Auto caching

MKNetworkKit can automatically cache all your “GET” requests. When you make the same request again, MKNetworkKit calls your completion handler with the cached version of the response (if it’s available) almost immediately. It also makes a call to the remote server again. After the server data is fetched, your completion handler is called again with the new response data. This means, you don’t have to handle caching manually on your side. All you need to do is call one method,

[[MKNetworkEngine sharedEngine] useCache];

Optionally, you can override methods in your MKNetworkEngine subclass to customize your cache directory and in-memory cache cost.

Operation freezing

With MKNetworkKit, you have the ability to freeze your network operations. When you freeze an operation, in case of network connectivity losses, they will be serialized automatically and performed once the device comes back online. Think of “drafts” in your twitter client.

When you post a tweet, mark that network call as freezable and MKNetworkKit automatically takes care of freezing and restoring these requests for you! So the tweets get sent later without you writing a single line of additional code. You can use this for other operations like favoring a tweet or sharing a post from your Google reader client, adding a link to Instapaper and similar operations.

Performs exactly one operation for similar requests

When you load thumbnails (for a twitter stream), you might end up creating a new request for every avatar image. But in reality, you only need as many requests as there are unique URLs. With MKNetworkKit, every GET request you queue gets executed exactly once. MKNetworkKit is intelligent enough not to cache “POST” http requests.

Image Caching

MKNetworkKit can be seamlessly used for caching thumbnail images. By overriding a few methods, you can set how many images should be held in the in-memory cache and where in the Caches directory it should be saved. Overriding these methods are completely optional.

Performance

One word. SPEED. MKNetworkKit caching is seamless. It works like NSCache, except that, when there is a memory warning, the in-memory cache is written to the Caches directory.

Full support for Objective-C ARC

You normally choose a new networking framework for new projects. MKNetworkKit is not meant for replacing your existing framework (though you can, it’s quite a tedious job). On new projects, you will almost and always want to enable ARC and as on date of writing this, MKNetworkKit is probably the only networking framework that is fully ARC ready. ARC based memory management is usually an order of magnitude faster than non-ARC based memory management code.

How to use

Ok, Enough self-praises. Let us now see how to use the framework.

Adding the MKNetworkKit

1. Drag the MKNetworkKit directory to your project.
   
2. Add the CFNetwork.Framework and SystemConfiguration.framework.
   
3. Include MKNetworkKit.h to your PCH file
   
4. Under your “Compile Sources” section, add -fno-objc-arc to the Reachability.m file.
   

You are done. Just 5 core files and there you go. A powerful networking kit.

Classes in MKNetworkKit

1. MKNetworkOperation
   
2. MKNetworkEngine
   
3. Miscellaneous helper classes (Apple’s Reachability) and categories
   

I believe in simplicity. Apple has done the heavy lifting of writing the actual networking code. What a third-party networking framework should provide is an elegant queue based networking with optional caching. I believe that, any third party framework should have under 10 classes (whether it’s networking or UIKit replacement or whatever). More than that is a bloat. Three 20 library is an example of bloat and so is ShareKit. May be it’s good. But it still huge and bloated. ASIHttpRequest or AFNetworking are lean and lightweight, unlike RESTKit. JSONKit is lightweight unlike TouchJSON (or any of the TouchCode libraries). May be it’s just me, but I just can’t take it when more than a third of source code lines in my app comes from a third party library.

The problem with a huge framework is the difficulty in understanding the internal working and the ability to customize it to suit your needs (in case you need to). My frameworks (MKStoreKit for adding In App Purchases to your app) have always been super easy to use and I believe MKNetworkKit would also be the same. For using MKNetworkKit, all you need to know are the methods exposed by the two classes MKNetworkOperation and MKNetworkEngine. MKNetworkOperation is similar to the ASIHttpRequest class. It is a subclass of NSOperation and it wraps your request and response classes. You create a MKNetworkOperation for every network operation you need in your application.

MKNetworkEngine is a pseudo-singleton class that manages the network queue in your application. It’s a pseudo-singleton, in the sense, for simple requests, you should use MKNetworkEngine methods directly. For more powerful customization, you should subclass it. Every MKNetworkEngine subclass has its own Reachability object that notifies it of server reachability notifications. You should consider creating a subclass of MKNetworkEngine for every unique REST server you use. It’s pseudo-singleton in the sense, every single request in any of it’s subclass goes through one and only one single queue.

You can retain instances of your MKNetworkEngine in your application delegate just like CoreData managedObjectContext class. When you use MKNetworkKit, you create an MKNetworkEngine sub class to logically group your network calls. That is, all Yahoo related methods go under one single class and all Facebook related methods into another class. We will now look at three different examples of using this framework.

Example 1:

Let’s now create a “YahooEngine” that pulls currency exchange rates from Yahoo finance.

Step 1: Create a YahooEngine class as a subclass of MKNetworkEngine. MKNetworkEngine init method takes hostname and custom headers (if any). The custom headers is optional and can be nil. If you are writing your own REST server (unlike this case), you might consider adding client app version and other misc data like client identifier.

NSMutableDictionary *headerFields = [NSMutableDictionary dictionary];
[headerFields setValue:@"iOS" forKey:@"x-client-identifier"];

self.engine = [[YahooEngine alloc] initWithHostName:@"download.finance.yahoo.com"
customHeaderFields:headerFields];

Note that, while yahoo doesn’t mandate you to send x-client-identifier in the header, the sample code shown above sends this just to illustrate this feature.

Since the complete code is ARC, it’s up to you as a developer to own (strong reference) the Engine instance.

When you create a MKNetworkEngine subclass, Reachability implementation is done automatically for you. That’s when your server goes down or due to some unforeseen circumstances, the hostname is not reachable, your requests will automatically be queued/frozen. For more information about freezing your operations, read the section Freezing Operations later in the page.

Step 2: Designing the Engine class (Separation of concerns)

Let’s now start write the methods in Yahoo Engine to fetch exchange rates. The engine methods will be called from your view controller. A good design practice is to ensure that your engine class doesn’t expose URL/HTTPHeaders to the calling class. Your view should not “know” about URL endpoints or the parameters needed. This means, parameters to methods in your Yahoo Engine should be the currencies and the number of currency units. The return value of this method could be a double value that is the exchange rate factor and may be the timestamp of the time it was fetched. Since operations are not performed synchronously, you should return these values on blocks. An example of this would be,

-(MKNetworkOperation*) currencyRateFor:(NSString*) sourceCurrency
inCurrency:(NSString*) targetCurrency
onCompletion:(CurrencyResponseBlock) completion
onError:(ErrorBlock) error;

MKNetworkEngine, the parent class defines three types of block methods as below.

typedef void (^ProgressBlock)(double progress);
typedef void (^ResponseBlock)(MKNetworkOperation* operation);
typedef void (^ErrorBlock)(NSError* error);

In our YahooEngine, we are using a new kind of block, CurrencyResponseBlock that returns the exchange rate. The definition looks like this.

typedef void (^CurrencyResponseBlock)(double rate);

In any normal application, you should be defining your own block methods similar to this CurrencyResponseBlock for sending data back to the view controllers.

Step 3: Processing the data

Data processing, that is converting the data you fetch from your server, whether it’s JSON or XML or binary plists, should be done in your Engine. Again, relieve your controllers of doing this task. Your engine should send back data only in proper model objects or arrays of model objects (in case of lists). Convert your JSON/XML to models in the engine. Again, to ensure proper separation of concerns, your view controller should not “know” about the “keys” for accessing individual elements in your JSON.

That concludes the design of your Engine. Most networking framework doesn’t force you to follow this separation of concerns. We do, because we care for you

Step 4: Method implementation

We will now discuss the implementation details of the method that calculates your currency exchange.

Getting currency information from Yahoo, is as simple as making a GET request.

I wrote a macro to format this URL for a given currency pair.

#define YAHOO_URL(__C1__, __C2__) [NSString stringWithFormat:@"d/quotes.csv?e=.csv&f=sl1d1t1&s=%@%@=X", __C1__, __C2__]

Methods you write in your engine class should do the following in order.

1. Prepare your URL from the parameters.
   
2. Create a MKNetworkOperation object for the request.
   
3. Set your method parameters.
   
4. Add completion and error handlers to the operation (The completion handler is the place to process your responses and convert them to Models.)
   
5. Optionally, add progress handlers to the operation. (Or do this on the view controller)
   
6. If your operation is file download, set a download stream (normally a file) to it. This is again optional.
   
7. When the operation completes, process the result and invoke the block method to return this data to the calling method.
   

This is illustrated in the following code

MKNetworkOperation *op = [self operationWithPath:YAHOO_URL(sourceCurrency, targetCurrency)
params:nil
httpMethod:@"GET"];

[op onCompletion:^(MKNetworkOperation *completedOperation)
{
DLog(@"%@", [completedOperation responseString]);

// do your processing here
completionBlock(5.0f);

}onError:^(NSError* error) {

errorBlock(error);
}];

[self enqueueOperation:op];

return op;

The above code formats the URL and creates a MKNetworkOperation. After setting the completion and error handlers it queues the operation by calling the super class’s enqueueOperation method and returns a reference to it. Your view controller should own this operation and cancel it when the view is popped out of the view controller hierarchy. So if you call the engine method in, say viewDidAppear, cancel the operation in viewWillDisappear. Canceling the operation will free up the queue for performing other operations in the subsequent view (Remember, only two operations can be performed in parallel on a mobile network. Canceling your operations when they are no longer needed goes a long way in ensuring performance and speed of your app).

You view controller can also (optionally) add progress handlers and update the user interface. This is illustrated below.

[self.uploadOperation onUploadProgressChanged:^(double progress) {

DLog(@"%.2f", progress*100.0);
self.uploadProgessBar.progress = progress;
}];

MKNetworkEngine also has convenience methods to create a operation with just a URL. So the first line of code can also be written as

MKNetworkOperation *op = [self operationWithPath:YAHOO_URL(sourceCurrency, targetCurrency)];

Do note here that request URLs are automatically prefixed with the hostname you provided while initializing your engine class.

Creating a POST, DELETE or PUT method is as easy as changing the httpMethod parameter. MKNetworkEngine has more convenience methods like this. Read the header file for more.

Example 2:

Uploading an image to a server (TwitPic for instance).

Now let us go through an example of how to upload an image to a server. Uploading an image obviously requires the operation to be encoded as a multi-part form data. MKNetworkKit follows a pattern similar to ASIHttpRequest.

You call a method addFile:forKey: in MKNetworkOperation to “attach” a file as a multi-part form data to your request. It’s that easy.

MKNetworkOperation also has a convenience method to add a image from a NSData pointer. That’s you can call addData:forKey: method to upload a image to your server directly from NSData pointer. (Think of uploading a picture from camera directly).

Example 3:

Downloading files to a local directory (Caching)

Downloading a file from a remote server and saving it to a location on users’ iPhone is super easy with MKNetworkKit.

Just set the outputStream of MKNetworkOperation and you are set.

[operation setDownloadStream:[NSOutputStream
outputStreamToFileAtPath:@"/Users/mugunth/Desktop/DownloadedFile.pdf"
append:YES]];

You can set multiple output streams to a single operation to save the same file to multiple locations (Say one to your cache directory and one to your working directory)

Example 4:

Image Thumbnail caching

For downloading images, you might probably need to provide an absolute URL rather than a path. MKNetworkEngine has a convenience method for this. Just call operationWithURLString:params:httpMethod: to create a network operation with an absolute URL.

MKNetworkEngine is intelligent. It coalesces multiple GET calls to the same URL into one and notifies all the blocks when that one operation completes. This drastically improves the speed of fetching your image URLs for populating thumbnails.

Subclass MKNetworkEngine and override image cache directory and cache cost. If you don’t want to customize these two, you can directly call MKNetworkEngine methods to download images for you. I would actually recommend you to do that.

Caching operations

MKNetworkKit caches all requests by default. All you need to do is to turn on caching for your Engine. When a GET request is performed, if the response was previously cached, your completion handler is called with the cached response almost immediately. To know whether the response is cached, use the isCachedResponse method. This is illustrated below.

[op onCompletion:^(MKNetworkOperation *completedOperation)
{
if([completedOperation isCachedResponse]) {
DLog(@"Data from cache");
}
else {
DLog(@"Data from server");
}

DLog(@"%@", [completedOperation responseString]);
}onError:^(NSError* error) {

errorBlock(error);
}];

Freezing operations

Arguably, the most interesting feature of MKNetworkKit is built in ability to freeze operations. All you need to do is set your operation as freezable. Almost zero effort!

[op setFreezable:YES];

Freezable operations are automatically serialized when the network goes down and executed when connectivity is restored. Think of having the ability to favorite a tweet while you are offline and the operation is performed when you are online later.

Frozen operations are also persisted to disk when the app enters background. They will be automatically performed when the app is resumes later.

Convenience methods in MKNetworkOperation

MKNetworkOperation exposes convenience methods like the following to get the format your response data.

1. responseData
   
2. responseString
   
3. responseJSON (Only on iOS 5)
   
4. responseImage
   
5. responseXML
   
6. error
   

They come handy when accessing the response after your network operation completes. When the format is wrong, these methods return nil. For example, trying to access responseImage when the actual response is a HTML response will return nil. The only method that is guaranteed to return the correct, expected response is responseData. Use the other methods if you are sure of the response type.

Convenience macros

The macros, DLog and ALog were stolen unabashedly from Stackoverflow and I couldn’t again find the source. If you wrote that, let me know.

A note on GCD

I purposefully didn’t use GCD because, network operations need to be stopped and prioritized at will. GCD, while more efficient that NSOperationQueue cannot do this. I would recommend not to use GCD based queues for your network operations.

Documentation

The header files are commented and I’m trying out headerdoc from Apple. Meanwhile, you can use/play around (read: Fork) with the code.

Source Code

The source code for MKNetworkKit along with a demo application is available on Github.

MKNetworkKit on Github

Feature requests

Please don’t email me feature requests. The best way is to create an issue on Github.

Licensing

MKNetworkKit is licensed under MIT License

All of my source code can be used free of charge in your app, provided you add the copyright notices to your app. A little mention on one of your most obscure “about” page will do.

Attribution free licensing available upon request. Contact me at mknetworkkit@mk.sg

–

Mugunth