Testing Environment

When you set up a Core Data stack using a file based store (say, using the sqlite store), that file becomes tied to the data model. That is for good reason. Core Data needs to first verify several things before it can use that data store. It must verify the entities have all the attributes the data model expects, as well as to ensure all entities are present in the store. The data and the description of that data must match exactly. However, when you are developing an app using Core Data, this schema changes fast. Adding a new attribute, or even simply changing it’s type can alter the version hashes enough so that the file version and the model version don’t match.

All this is to say that using a file based datastore for unit testing is problematic. Even if you were to set up your file based Core Data model to be auto migrating, it’s still highly likely you’ll perform a refactoring that is simply not compatible with automigrations. Another likely scenario when testing is that you’ll want to create simple test data for a single test, or suite of tests, and then delete it.

While testing with a file based Core Data store is possible, it’s fairly tricky and error prone. Generally, when you setup your test cases, you’ll want to load your sample set of data. When you’re finished, you’ll then want to delete all that data. Unit tests should fail on an error with our application code, not our test infrastructure, setup or teardown code. And besides, I know you’ve already jumped to the same conclusion I have by now, and that’s to use an In-Memory persistent store. Not only are there no file deletions to deal with, you will never have to perform a migration. Since during unit testing, you want a temporary store, the in-memory store is a perfect setup.

Note: Using an in-memory store to perform tests against a Core Data store is not an entirely new concept. Graham Lee has also previously described his process and setup for testing Core Data with an in-memory store.

Quickly Setting up Tests

Now that we’ve gone over a little background, let’s get to how we can use MagicalRecord to easily setup our Core Data tests.

As we’ve discussed previously, MagicalRecord comes with a set of helpers to build a simple Core Data stack for you. We’ll use one that’s fairly straight forward:

[MagicalRecord setupCoreDataStackWithInMemoryStore];

We’ll also want to clean up the Core Data stack after some tests, and, yes, MagicalRecord provides a handy cleanup method for you:

[MagicalRecordHelpers cleanUp];

Let’s explore when and where to incorporate these method calls into your tests.

Testing Frameworks

Up to this point, the discussion hasn’t been geared toward any particular testing framework, although the syntax is common to the SenTestingKit testing framework that is now built in (and a first class citizen in Xcode4). However, there are now several testing frameworks now that provide many structural similarities, while improving on the syntax. Let’s see how things differ across a few of the more popular testing frameworks available for Objective C.

SenTest, OCUnit, GHUnit

SenTestingKit and GHUnit offer a very similar experience from a testing perspective. Primarily, you create a new subclass of SenTestCase, or GHTestCase, and author all your tests in methods starting with the word “test“. You can also run a method before all tests in the class are performed, or before each method.

Depending on how you set up your tests, you have two options on where to put the simple MagicalRecord setup method. If you want a pristine data store for every test (useful for isolating tests), then you’ll want to put this method in the setUp method like so:

- (void)setUp;
{
	[MagicalRecord setupCoreDataStackWithInMemoryStore];
}

However, there are certain cases where you’ll need to preload a larger test data set to test against. I would recommend doing this before each and every test as it will only slow your test runs down. And making tests slower can eventually lead to you not running them at all. I recommend setting this up once per test suite:

- (void)setUpClass;
{
	[MagicalRecord setupCoreDataStackWithInMemoryStore];
}

And to clean up your stack, simply add a call to the clean up method using the appropriate post-test method, tearDown, or tearDownClass.

Kiwi, Cedar

Kiwi and Cedar are the Behavior Driven Development (BDD) style frameworks. Rather than methods, you format your tests using blocks, however there are similar mechanics when runnings tests. That is, you also have ways to setup and teardown your test Core Data stack even if the methods aren’t called setup and teardown. However, with it’s block type syntax, you can have nested setup calls to beforeAll and beforeEach, afterAll and afterEach (the BDD versions of the setup and teardown methods). You will want to place your calls to set up your test core data stack at an appropriate level within your block contexts.

describe(@"MyEntity", ^{
 
  beforeEach(^{
	[MagicalRecord setupCoreDataStackWithInMemoryStore];
  });
 
  afterEach(^{
	[MagicalRecord cleanUp];
  });
 
  it(@"should do something awesome", ^{ /* Put your test here! */ });
 
});

As you can see, setting up and cleaning up after your tests is fairly easy with MagicalRecord.

Supporting Xcode’s Unit Test Support

When using a testing framework like GHUnit, the afore mentioned steps are all that’s required to get started. However, when it comes to using Xcode’s built in unit test bundle, there is one extra step required in order to get the stack setup.

When you first need to load up the Core Data stack, you typically use the method:

[NSManagedObjectModel mergedModelsFromBundle:nil];

This method, give the parameter of nil, will look in the main bundle, which is generally the application bundle, and look for all data model files, and merge those files into a single Core Data model instance for use when instantiating an NSPersistentStoreCoordinator and it’s related NSPersistentStore. However, with the Unit Test bundle, this call does not return the correct answer, as documented by Graham Lee. The solution to this problem is simple: we need to specify the correct bundle so that the Core Data stack initialization process can continue on it’s way. To do this with MagicalRecord, we use the following setUp method for our tests:

- (void)setUp;
{
	[MagicalRecord setDefaultModelFromClass:[self class]];
	[MagicalRecord setupCoreDataStackWithInMemoryStore];
}

The setDefaultModelFromClass: method will perform the solution provided by Graham Lee, which is to load the model from the specified class, which, in turn, is the class of the currently executing test case. And now that we have something more generic, we have the ability to create a very simple boilerplate Core Data unit test case:

//CoreDataTestCase.h
 
@interface CoreDataTestCase : SenTestCase
@end
 
//CoreDataTestCase.m
#import "CoreDataTestCase.h"
 
@implementation CoreDataTestCase
 
- (void)setUp;
{
	[MagicalRecord setDefaultModelWithClass:[self class]];
	[MagicalRecord setupCoreDataStackWithInMemoryStore];
}
 
- (void)tearDown;
{
	[MagicalRecord cleanUp];
}
 
@end

This means that when you begin authoring unit tests requiring Core Data, you can simply inherit from this base test case, and you’re ready to start writing tests.

A Test Case

First of all, it goes without saying that when you’re using Core Data, you create a set of custom entity classes, which are subclasses of NSManagedObject.  And, to make your life easier, you should always use mogenerator to create your custom entities. This is where your custom logic will live. Let’s author a sample test:

@implementation SampleTestCase

- (void)setUp;
{
	[MagicalRecord setDefaultModelWithClass:[self class]];
	[MagicalRecord setupCoreDataStackWithInMemoryStore];
}
 
- (void)tearDown;
{
	[MagicalRecord cleanUp];
}
 
- (void)testSomeCalculationOnMyEntity;
{
	MyEntity *testEntity = [MyEntity MR_createEntity];
	float expectedValue = …;
	STAssert([testEntity customCalculation] == expectedValue, @"expected a good calculation");
}
 
@end

The MR_createEntity method will use the default context that was set up when the setupCoreDataStackWithInMemoryStore method was called earlier on in the test run. This makes for clean tests based on Core Data, but doesn’t let the Core Data syntax get in the way of expressing the intent of the test. And, when the test has completed, this sample entity will be deleted by virtue of the store being released from memory, so that the next test can have a clean slate with which to work.

Conclusion

And with that, you have an extremely simple and fast way to get a test environment setup for unit testing with Core Data. What I enjoy most about this setup is that when authoring unit tests which require Core Data, I am not focused on the details of setting up and tearing down Core Data. Rather, I’m focused on a single entity, and the things it requires in order to work properly. A sample set of unit tests which, in turn, test MagicalRecord itself are included in the MagicalRecord source on github.

Until recently the obvious thing would have been to tell NSData to create a memory-mapped instance. Given NSString *path pointing to a file, you could create an NSData with almost no memory hit regardless of file size by creating it as:

NSData *data = [NSData dataWithContentsOfMappedFile:path];

Starting with iOS 5 though, this method has been deprecated. Instead, what you’re supposed to do is:

NSError *error = nil;
NSData *data = [NSData dataWithContentsOfFile:path options:NSDataReadingMappedAlways error:&error];

So, fine, whatever, it’s a different call, so what? Well, it wasn’t working. Instruments was showing that the app was taking the full memory hit when the NSData was created. Mapping wasn’t working despite using NSDataReadingMappedAlways. So what could he do? The wheels of my mind started turning.

Memory mapped files

But first, a brief aside about memory mapping.

Memory mapping is a cool Unix trick that lets you load a file into memory without, as it were, actually loading it into memory. It’s a way of using virtual memory to your advantage when you have a really big file and you don’t want to spend the RAM on it.

Contrary to common misconception, iOS does have virtual memory. It just doesn’t create swap files. But the full power of virtual memory is at your disposal. When you create a memory mapped file, the operating system gives you a memory pointer that you can use to access the file’s data. It’s as if the file was already loaded into memory but had since been swapped back out. When you access bytes in the memory map, data blocks are selectively read from the file as needed, and disposed of when they aren’t.

In short, it’s exactly what you need when your data file is too big to load, and if NSData won’t do it, I’ll just have to force it.

By the power of Greyskull Unix!

To create memory mapped files NSData is making use of iOS’s excellent Unix core. NSData isn’t actually mapping files itself, instead it’s using the Unix mmap(2) call. I can use that too. Given an NSString *path pointing to a file, you can create an memory mapped file like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

// Get an fd
int fd = open([path fileSystemRepresentation], O_RDONLY);
if (fd < 0) {
    return nil;
}
 
// Get file size
NSDictionary *fileAttributes = [[NSFileManager defaultManager] attributesOfItemAtPath:path error:nil];
if (fileAttributes == nil) {
    close(fd);
    return nil;
}
NSNumber *fileSize = [fileAttributes objectForKey:NSFileSize];
 
// mmap
void *mappedFile;
mappedFile = mmap(0, [fileSize intValue], PROT_READ, MAP_FILE|MAP_PRIVATE, fd, 0);
close(fd);
 
if (mappedFile == MAP_FAILED) {
    NSLog(@"Map failed, errno=%d, %s", errno, strerror(errno));
    return nil;
}

To call mmap(2) this code first gets a file descriptor for the file via open(2). That, combined with the file’s size, is enough to create the memory mapping. Once the mapping exists, the code disposes of the file descriptor via close(2). At this point, mappedFile points to a bunch of bytes which is more or less indistinguishable from what you’d get if you had actually read the file into memory. And NSData knows how to use byte blobs.

// Create the NSData
NSData *mappedData = [NSData dataWithBytesNoCopy:mappedFile length:[fileSize intValue] freeWhenDone:NO];

I can convert the pointer into an NSData using one of NSData’s longer convenience initializers. Why create it this way? Because (a) I don’t want to copy the bytes, since that would negate the advantage of memory mapping, and (b) I don’t want NSData to try and clean up those bytes when it gets deallocated.

The Complication

But I do need to clean up those bytes. I just don’t want NSData to do it, because it doesn’t know the bytes came from a mapping and won’t clean them up properly. I need to remove the memory map. What needs to happen is a call to munmap(2) when the NSData deallocates.

So, what does the code need to look like in order to make this call? Ideally the cleanup should happen automatically. I could just call munmap(2) directly. But that would mean keeping the map pointer around and then making a separate call. All to clean up what is, conceptually at least, an internal data structure. It would work but it’s ugly.

Anyway, with ARC there’s the chance that I’d make the call before the NSData deallocated, with disastrous results. Really what I’d like to do is wrap the code above into a convenient API where you can create the mapped NSData, use it, and dispose of it normally. Any extra cleanup should just happen. After all this is Cocoa and it’s supposed to work well.

My first thought was to subclass NSData and have the subclass store the map pointer. Then the subclass could call munmap(2) from its -dealloc method. But NSData is a class cluster, and subclassing class clusters is kind of a pain in the ass. Class clusters are a bunch of classes that masquerade as a specific public class. Examples include NSString, NSArray, and others. And of course NSData.

You may recall that -init is not required to return an instance of the class you thought you were creating. That is, if you call

Foo *myFoo = [[Foo alloc] init];

…the resulting object is not required to actually be an instance of Foo. Class clusters are a case where this happens. When you create an NSData, what you’re probably getting is an instance of something like NSConcreteData. That class isn’t documented but it acts like an NSData and you generally can’t tell the difference.

Subclassing class clusters can be challenging. You need to override all of the superclass’s primitive methods. Those are the methods that access the object’s data directly. NSData’s primitive methods aren’t documented, either. If you don’t get them all you’ll get crashes with fairly incomprehensible exceptions like:

Catchpoint 6 (exception thrown).2012-02-07 21:04:10.620 MapTest[9719:f803] *
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*
initialization method -initWithBytes:length:copy:freeWhenDone:bytesAreVM: cannot be
sent to an abstract object of class NSMappedData: Create a concrete instance!'

I could probably figure out what the primitive methods are but doing that without documentation has a strong whiff of reverse engineering. I’d really rather avoid that. But how else can I make sure the call happens? And can I make it happen automatically?

Adding dealloc code in a category

The method I had in mind for creating mapped NSData instances would look something like:

+ (NSData *)dataWithContentsOfReallyMappedFile:(NSString *)path;

I could put that in a category except that to make it easy to use I’d need to have some code to -dealloc that could call munmap(2). That’s more or less what what I’m going to do.

Wait, what? You can’t override -dealloc in a category. But thanks to associated objects, you don’t have to. If have some object A, and you associate a secondary object B with it, then when A gets deallocated, B will too. If you have something you really need to happen when A gets deallocated, you can call that code from B’s dealloc method. Bingo, deallocation code for A that runs in a separate class. As long as you don’t need to access A’s private internal data, anyway.

This could be pretty useful so I decided to write it as a generic system that I could use with memory mapped NSData instances but that isn’t tied to them. I created a generic class called DeallocationHandler:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

@interface DeallocHandler : NSObject
@property (readwrite, copy) void (^theBlock)(void);
@end
 
@implementation DeallocHandler
@synthesize theBlock;
 
- (void)dealloc
{
    if (theBlock != nil) {
        theBlock();
    }
}
@end

This doesn’t do much on its own. Give it a block and it will run the block when it gets deallocated. Where it gets interesting is when you use it in a category on NSObject:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

static char *deallocArrayKey = "deallocArrayKey";
 
@implementation NSObject (deallocBlock)
 
- (void)addDeallocBlock:(void (^)(void))theBlock;
{
    NSMutableArray *deallocBlocks = objc_getAssociatedObject(self, &deallocArrayKey);
    if (deallocBlocks == nil) {
        deallocBlocks = [NSMutableArray array];
        objc_setAssociatedObject(self, &deallocArrayKey, deallocBlocks, OBJC_ASSOCIATION_RETAIN_NONATOMIC);
    }
    DeallocHandler *handler = [[DeallocHandler alloc] init];
    [handler setTheBlock:theBlock];
    [deallocBlocks addObject:handler];
}
@end

The -addDeallocBlock: method associates an array of DeallocationHandler instances with the target object. Each DeallocationHandler in turn has a block provided by the caller. The upshot is that, when the target object deallocates, all of those blocks will be run. This lets me attach dealloc-time code to any object. In fact I could add as many blocks as I needed and they’d run in order.

One minor caveat is that you really need to watch the memory references in these blocks. If the block references the object that it’s attached to, then the circular references mean that the object won’t ever get deallocated. This is just standard safe block usage, though.

Getting back to NSData

Using the deallocation block scheme I can add a block calling munmap(2) to my NSData object by adding this code right after I create it:

1
2
3

[mappedData addDeallocBlock:^{
    munmap(mappedFile, [fileSize intValue]);
}];

This leaves me with two categories instead of just one, but in the end I can just create mapped NSData instances and not bother with special cleanup code in my app.

Conclusion

Never forget that Apple’s APIs are not a menu of possibilities. They’re a set of tools, but you can and should build your own tools when you need them. Unix is available on iOS, so don’t be afraid to use it. In this example we’ve seen how to:

• Add deallocation code to any object without subclassing.
• Create memory mapped NSData instances even though the official API has been deprecated and the new one doesn’t currently work.

The code described in this post can be found at Github.

Update: In response to queries I’ve had via Twitter, please be aware that the code described here assumes that you’re using ARC. If you’re not using ARC you’ll need to add a line that releases the new DeallocHandler before -addDeallocBlock: returns.

Now, my sandbox technical challenges are not DOA. Some apps and classes of app are in effect outlawed by the sandbox rules. I feel for the developers of those apps, but fortunately for me, I’m not facing that.

My challenges fall more the “a few features may need rethinking and rewriting” category. At first I expected to have a long hiatus between updates while I worked on these features. But instead, I’m easing this transition by implementing sandboxing in 2 phases.

1. Add sandboxing to the MAS build of my app. Use temporary entitlements where necessary to maintain functionality. Make minor adjustments to run inside the sandbox. Submit an update to test the approval process.
2. Begin the rethinking and rewriting of sandbox-incompliant features.

Advantages I can see to this approach:

• I can continue to fix bugs, and issue updates beyond March 1st without a long hiatus.
• Development on my app does not appear quiescent.
• By submitting early, I’m not testing approval when trying to push out a needed bug fix.
• By using temporary entitlements, I signal to Apple that these are needed and useful.
• I buy more time to rethink and rewrite.

Sandboxing is a great leap in application security but every app has a different story when it comes to the implementation details. I encourage bug reporting. It’s important to explain to the Apple engineers what we developers need. And I hope that some of the thoughts here help you navigate the challenges.

1. Provide a safe, generic alternative to Cocoa’s -setValuesForKeysWithDictionary: for use with NSManagedObject and its subclasses
2. Handle cases where JSON data didn’t match up with what the managed objects expected. Getting a string where you expect a numeric value, or vice versa, for example, or getting a string representation of a date when you want a real NSDate object.

The first item was the most important. It’s tempting to use -setValuesForKeysWithDictionary: to transfer JSON to a model object in one step. The method runs through the dictionary and calls -setValue:forKey: on the target object for every entry. It has a fatal flaw though, in that it doesn’t check to see if the target object actually has a key before trying to set it. Using this method when you don’t have absolute control over the dictionary contents is an invitation to unknown key exceptions and nasty app crashes.

Fixing this for managed objects was relatively easy because Core Data provides convenient Objective-C introspection methods. The general approach was:

• Get a list of the target object’s attributes
• For each attribute, see if the incoming dictionary has an entry. If so,
• Compare the incoming type to the expected type, and convert if necessary.
• Call -setValue:forKey: with that key and its value.

And then just last week I had the thought, wouldn’t it be nice if this worked for any object, not just for managed objects?

Objective-C introspection

Since Objective-C is dynamic, pretty much everything you’d want to know about a class is available at run time. I’m not just talking about methods like -respondsToSelector: and -isKindOfClass:, though those are extremely useful. You can go much, much deeper than that, inspecting (and even changing) every aspect of a class’s implementation. Much of this happens via C function calls rather than Objective-C method calls. The Objective-C runtime is not actually written in Objective-C, and it’s the runtime that has the information.

To update the code for use with objects that don’t inherit from NSManagedObject the new code looks through the properties declared on a class and uses those to run through the incoming JSON. The general approach is the same but the implementation uses NSObject properties instead of NSEntityDescription attributes.

It’s also possible to look through the instance variables instead of the properties. Often this would amount to the same thing. Where they differ is when the backing ivar has a different name, i.e. when you’re using something like:

@synthesize foo = __myReallyBizarrePrivateName____;

In that case iterating over properties would find foo while iterating over instance variables would find __myReallyBizarrePrivateName____. Either is valid but I’m going with the properties because (at least for me) they’re more likely to match up with the JSON keys.

First pass: iterating over properties

A simple version that meets requirement #1 looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

- (void)setValuesForKeysWithJSONDictionary:(NSDictionary *)keyedValues dateFormatter:(NSDateFormatter *)dateFormatter
{
    unsigned int propertyCount;
    objc_property_t *properties = class_copyPropertyList([self class], &propertyCount);
 
    for (int i=0; i<propertyCount; i++) {
        objc_property_t property = properties[i];
        const char *propertyName = property_getName(property);
        NSString *keyName = [NSString stringWithUTF8String:propertyName];
 
        id value = [keyedValues objectForKey:keyName];
        if (value != nil) {
            [self setValue:value forKey:keyName];
        }
    }
    free(properties);
}

This starts with a call to class_copyPropertyList(), which gets a C-style array of property declarations for the requested class. The propertyCount argument indicates how many properties are in the array. The array contains zero or more objc_property_t entries, which is an opaque structure.

The code iterates through this array. For each one it uses property_getName to get the property name as a C-style string. Then it converts this to an NSString and uses that to look up entries in the incoming dictionary. And, voila, we’re using the class’s own properties to look up values in the dictionary instead of the other way around.

A final detail– unusual in Objective-C code– is the call to free(). Since class_copyPropertyList() has copy in its name, the calling code is responsible for disposing of the returned data. And since it’s a C call, this needs to be done C style. This call would need to be there even if the project were using ARC.

Back to Core Data, briefly

The great thing about this solution is that it’s not the non-managed-object alternative to the previous version, it’s a direct replacement. This approach works just as well on managed objects as on other objects– provided, that is, that you create custom subclasses of NSManagedObject for your entities that declare properties for managed object attributes. So long as the properties exist, the code works. If you’re using Xcode or mogenerator to generate your managed object subclasses, you’re covered. If you aren’t creating custom subclasses, first of all, why not? But in that case this approach won’t work since NSManagedObject doesn’t have the necessary property declarations.

Does it have to be like this?

Some of you may have noticed that it’s possible to do the same thing without any mucking about with the runtime by doing something like this:

1
2
3
4
5
6
7
8

    for (NSString *key in keyedValues) {
        @try {
            [self setValue:[keyedValues objectForKey:key] forKey:key];
        }
        @catch (NSException *exception) {
            // Do nothing
        }
    }

In this case the dictionary keys still drive the action, but exception handling means the code doesn’t crash on unknown keys. So why bother then? Because of requirement #2 above. Coercing JSON into appropriate data types is going to require introspection. With this simplified approach you don’t crash, but you also don’t get type conversions. If not crashing is all you’re interested in, this works just as well and is probably faster. It’s certainly simpler anyway. It’s not going to do what I need though.

JSON Fixes

To meet requirement #2 the code needs to go deeper. As with the Core Data implementation, it needs to look up the expected value for the property and compare that to the type of the incoming data. To do this I’ll expand the if block beginning on line 12 above to look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

if (value != nil) {
    char *typeEncoding = NULL;
    typeEncoding = property_copyAttributeValue(property, "T");
 
    if (typeEncoding == NULL) {
        continue;
    }
    switch (typeEncoding[0]) {
        case '@':
        {
            // Object
            Class class = nil;
            if (strlen(typeEncoding) >= 3) {
                char *className = strndup(typeEncoding+2, strlen(typeEncoding)-3);
                class = NSClassFromString([NSString stringWithUTF8String:className]);
            }
            // Check for type mismatch, attempt to compensate
            if ([class isSubclassOfClass:[NSString class]] && [value isKindOfClass:[NSNumber class]]) {
                value = [value stringValue];
            } else if ([class isSubclassOfClass:[NSNumber class]] && [value isKindOfClass:[NSString class]]) {
                // If the ivar is an NSNumber we really can't tell if it's intended as an integer, float, etc.
                NSNumberFormatter *numberFormatter = [[NSNumberFormatter alloc] init];
                [numberFormatter setNumberStyle:NSNumberFormatterDecimalStyle];
                value = [numberFormatter numberFromString:value];
                [numberFormatter release];
            } else if ([class isSubclassOfClass:[NSDate class]] && [value isKindOfClass:[NSString class]] && (dateFormatter != nil)) {
                value = [dateFormatter dateFromString:value];
            }
 
            break;
        }
 
        default:
        {
            break;
        }
    }
    [self setValue:value forKey:keyName];
    free(typeEncoding);
}

The runtime provides information about properties via the call to property_copyAttributeValue() on line 3. The first argument is the property of interest. The second one can have a bunch of different values depending on what you want to look up. A “T” requests a C-style string that describes the property type. For full details on what you can do with the second argument, see the (somewhat out of date as of this writing) “Declared Properties” section of Apple’s Objective-C Runtime Programming Guide.

If the property type is a pointer to a class, the return value will be something like T@"NSNumber", T@"NSString", T@"MyClass", etc. The next thing the code does then is to check for a leading @ and, if found, set about getting the Class that the type string names. This happens in lines 12-16. The code strips off the @ and the quotes, converts to NSString, and uses NSClassFromString to get the Class.

The rest of this code is remarkably similar to the Core Data version, looking for type mismatches and converting the incoming value where needed. The chief difference is that it uses -isSubclassOfClass: to check on the expected type of the property instead of looking at the Core Data-specific NSAttributeType.

Except for one important difference. For numeric properties, the Core Data attribute type would indicate whether a floating point or integer value was expected. With NSNumber we have no way of knowing. So the code uses NSNumberFormatter to parse incoming strings and leaves it at that. If this kind of mismatch occurs then there are bigger problems anyway. You could change the code to round a float to an int, but is that actually going to be a valid result? Maybe, maybe not.

The type comparisons in this code could go on forever but in this case the code is specifically looking for problems that sometimes crop up with JSON.

Since property_copyAttributeValue() has copy in its name, this block adds another call to free() to clean up after itself.

Handling primitives

But what if the expected value is not an object at all? What if it’s a primitive int? Fortunately property_copyAttributeValue() covers that case as well. In this case the type encoding string is shorter, with values like Ti for int, Tf for float, TQ for unsigned long long, etc (again, see Apple’s docs for a full list).

With this information, the switch statement above can be expanded with a few more cases.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

        case 'i': // int
        case 's': // short
        case 'l': // long
        case 'q': // long long
        case 'I': // unsigned int
        case 'S': // unsigned short
        case 'L': // unsigned long
        case 'Q': // unsigned long long
        case 'f': // float
        case 'd': // double
        case 'B': // BOOL
        {
            if ([value isKindOfClass:[NSString class]]) {
                NSNumberFormatter *numberFormatter = [[NSNumberFormatter alloc] init];
                [numberFormatter setNumberStyle:NSNumberFormatterDecimalStyle];
                value = [numberFormatter numberFromString:value];
                [numberFormatter release];
            }
            break;
        }
 
        case 'c': // char
        case 'C': // unsigned char
        {
            if ([value isKindOfClass:[NSString class]]) {
                char firstCharacter = [value characterAtIndex:0];
                value = [NSNumber numberWithChar:firstCharacter];
            }
            break;
        }

The first half of this, up to line 20, handles all the numeric types. As with the previous code block there’s no attempt to work out floating point/integer conflicts since it’s impossible to know how to handle this in the general case. It would be possible to break this up into more specific checks, say by using -[NSString intValue] when an integer result is expected and hope for valid data. But the code above handles the case where you actually have an integer value, and if you don’t have one then again, you have bigger problems.

The rest of this block handles a primitive char property by taking the first character in the incoming string. Longer strings can’t be stored in a char anyway, so this is the best approach.

In both cases the conversion results in an NSNumber instead of a primitive type. That’s OK though– -setValue:forKey: has our back here and will unbox the object for us.

Conclusion

Using this approach makes it a lot easier to deal with web services. You can’t always rely on the results matching what you expect (or what’s documented). Inspecting classes at run time takes a more defensive approach to dealing with data you can’t control.

A category on NSObject that implements this code can be found at github.

There are several different ways to solve this problem. The easiest is to have the child ping the parent whenever it changes and then the parent can update any values it needs to as a result of that ping. Another solution that I considered was to have the parent observe the property on each child. Of course this requires additional code to handle when a child is removed from the parent and when a new child is added to the parent. Lots of code to manage and the potential for fragility is rather high.

The most interesting solution that I offered was to have the parent watch for change notifications and react to them. It is this solution that I will discuss in a little more depth.

Change Notifications

Core Data produces several different notifications during its lifecycle. The most commonly used notification is NSManagedObjectContextDidSaveNotification. Whenever a NSManagedObjectContext saves it will fire two notifications, one before the save and one after the save. The one before the save does not contain any direct information about what is going to be saved although you can discover it yourself.

The second one, NSManagedObjectContextDidSaveNotification contains a reference to each entity that it saved; whether they are newly inserted entities, updated entities or deleted entities.

There is another notification that is not used nearly as often; NSManagedObjectContextObjectsDidChangeNotification. This notification is significantly more chatty than the other two. It potentially can fire every time an entity changes. More specifically it will fire at the end of each run loop where an entity has changed. While this notification is more chatty, it is more useful for this purpose.

Why use NSManagedObjectContextObjectsDidChangeNotification?

Of the three, the NSManagedObjectContextObjectsDidChangeNotification is the most useful for a parent/child monitoring. Both NSManagedObjectContextDidSaveNotification and NSManagedObjectContextWillSaveNotification only fire when a NSManagedObjectContext has been asked to save. A save can be a very rare event; possibly only once during the life of the application.

However, NSManagedObjectContextObjectsDidChangeNotification fires frequently enough that we can use it without having to worry about when a save is going to occur. We can use it during the life of the application and keep a parent updated to the status of its children.

An Example

A common example that I like to use while I am testing these theories is my very simple RSS reader. Those who have watched my Core Data videos are familiar with Zeader.

In Zeader, there are only two entities:

In this example we want the Server entity to be updated whenever the read attribute on a child has changed. To do that we are going to make a few enhancements to the Server entity subclass.

-awakeFromInsert and -awakeFromFetch

The first change we need to make is with the awake methods. Whenever a Server entity is created or loaded we need to start listening for NSManagedObjectContextObjectsDidChangeNotification postings.

- (void)awakeFromInsert
{
  [super awakeFromInsert];

  NSString *name = NSManagedObjectContextObjectsDidChangeNotification;
  NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
  [center addObserver:self 
             selector:@selector(changes:) 
                 name:name 
               object:nil];
}

- (void)awakeFromFetch
{
  [super awakeFromFetch];

  NSString *name = NSManagedObjectContextObjectsDidChangeNotification;
  NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
  [center addObserver:self 
             selector:@selector(changes:) 
                 name:name 
               object:nil];
}

-willTurnInfoFault

To be clean we need to stop listening whenever the Server is about to be removed from memory. Since it is inappropriate to override the -dealloc method of a NSManagedObject we need to do so in the -willTurnIntoFault method.

- (void)willTurnIntoFault
{
  [super willTurnIntoFault];
  NSString *name = NSManagedObjectContextObjectsDidChangeNotification;
  NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
  [center removeObserver:self 
                    name:name 
                  object:nil];
}

This method of course, just like the two awake methods previously can be rolled up into just two lines. They are broken apart here to make it easier to consume on the web.

-changes:

The last piece to the puzzle is the need to react to those changes as they are posted.

- (void)changes:(NSNotification*)notification
{
  NSSet *objects = nil;
  NSMutableSet *combinedSet = nil;
  NSPredicate *predicate = nil;
  NSSet *unreadItems = nil;

  NSDictionary *userInfo = [notification userInfo];

  objects = [userInfo valueForKey:NSInsertedObjectsKey];
  combinedSet = [NSMutableSet setWithSet:objects];

  objects = [[notification userInfo] valueForKey:NSUpdatedObjectsKey];
  [combinedSet unionSet:objects];

  objects = [[notification userInfo] valueForKey:NSDeletedObjectsKey];
  [combinedSet unionSet:objects];

  predicate = [NSPredicate predicateWithFormat:@"entity.name == %@ && server == %@", 
              @"FeedItem", self];
  [combinedSet filterUsingPredicate:predicate];

  if ([combinedSet count] == 0) {
    return;
  }

  predicate = [NSPredicate predicateWithFormat:@"read == NO"];
  unreadItems = [[self feedItems] filteredSetUsingPredicate:predicate];
  [self setUnreadCount:[unreadItems count]];
  DLog(@"server status changed %i", [unreadItems count]);
}

First, we do not really care whether the entities are inserted, deleted or updated. We just need an answer to a simpler question: Are any of the entities that I care about in this change set? Since our question is easier, we first can lump all of the changes into a single set.

Once we have all of the changes in a single set we then filter that set on the important question. The predicate needs to be in a specific order; first we filter on the type of entity (FeedItem) we care about and then we filter on its relationship (is its Server me?).

This predicate reduces the NSMutableSet to only those FeedItem entities that are children of this instance of Server. If that resulting NSMutableSet is zero then there are no relevant changes and we return; done.

If there are changes we want to update our unreadCount. To do that we grab all of our feedItems and filter them with a simpler predicate of @"read == NO". The result of that filter can then be plugged into our unreadCount.

Conclusion

There are a few potential performance hot spots in this example. However, the monitoring of children by the parent is always going to be a performance hotspot. Therefore, great care should be given whenever to adding a watcher like this.

One of the nice things about developing software for OS X is all the “freebies” we get out of Cocoa. For example, when we are building a UI with text input we get undo support for free! How cool is that?

Likewise, when we are working with Core Data, it also has undo support built right in. Every NSManagedObjectContext has a NSUndoManager that we can use. However there are some situations where the default undo support is insufficient.

In this article we are going to walk through one such situation.

Edit Window Undo

There are situations where we want to group actions together so that from the user’s point of view they are an atomic operation. For example, if we have a window with an edit sheet; once the edit sheet is closed we want to avoid walking through the individual edits that occurred while the sheet was present.

However, while the sheet is present, we do want to give the user the ability to undo each individual edit. To perform this, we will be creating a commit group on the undo manager contained within the NSManagedObjectContext and we will use a separate Undo Manager that is responsible for the edit sheet. We could use the undo manager within Core Data for both but I have found it to be cleaner to keep them separated in this situation.

Building Our Application

In this application the main window will present the user with a single table view which has two columns, a date and a name. The table itself is non-editable but double clicking on a row will present a sheet for the user to edit the row. In addition, clicking on the add button will present the same sheet but with a new entry. The edit sheet contains three fields for data entry as shown.

The data model for our project contains a single NSManagedObject named TestEntity with three properties: name, date and desc. The model is shown in.

Implementing the App Delegate

Most of the AppDelegate is straight out of the Core Data Application template provided by Xcode. However we are going to add a double click action to the table view as well as handling for the add and remove buttons. To do this we need to add the table view as an outlet with the AppDelegate. We also need to add the NSArrayController as an outlet.

@interface AppDelegate : NSObject
{
  IBOutlet NSWindow *window;
  IBOutlet NSArrayController *arrayController;
  IBOutlet NSTableView *tableView;

  NSPersistentStoreCoordinator *persistentStoreCoordinator;
  NSManagedObjectModel *managedObjectModel;
  NSManagedObjectContext *managedObjectContext;
}

- (NSManagedObjectContext *)managedObjectContext;

- (IBAction)saveAction:(id)sender;
- (IBAction)addEntity:(id)sender;
- (IBAction)removeEntity:(id)sender;

@end

Once we have those bound in Interface Builder we can finish the configuration within the -awakeFromNib: method.

- (void)awakefromNib
{
  [tableView setTarget:self];
  [tableView setDoubleAction:@selector(editEntry:)];
}

In the -awakeFromNib: we set ourselves as the target for the tableView and add the -editEntry: method as its double click action. Now when a user double-clicks on a row our method will be called.

- (void)editEntry:(id)sender
{
  id object = [[arrayController selectedObjects] lastObject];
  if (!object) return;
  SEL endSelector = @selector(editSheetDidEnd:returnCode:object:);
  [[[self managedObjectContext] undoManager] beginUndoGrouping];
  [[[self managedObjectContext] undoManager] setActionName:@"Undo Object Edit"];
  [EditWindowController editSheetForWindow:window
                                  delegate:self
                               endSelector:endSelector
                                    entity:object];
}

In the -editEntry: method we first get the object for the row that was clicked on. We do a quick nil check and then hand the object off to our edit window controller. As you can see we have a convenience method in our EditWindowController to make it easy to create one, pass off the object and display the sheet. The method also accepts a delegate and a callback @selector.

You should pay particular attention to the line of code in the middle; the call to the NSManagedObjectContext’s NSUndoManager. In that line we are starting a grouping before we present the sheet. This guarantees that any changes made while the sheet is displayed will be considered “atomic” to the NSUndoManager contained within the NSManagedObjectContext.

Implementing the EditWindowController

The EditWindowController is a subclass of NSWindowController with a fairly simple header.

#import <Cocoa/Cocoa.h>

@interface EditWindowController : NSWindowController 
{
  NSUndoManager *undoManager;
  NSManagedObject *testEntity;
  IBOutlet NSObjectController *entityController;
}

@property (assign) NSManagedObject *testEntity;

+ (void)editSheetForWindow:(id)window 
                  delegate:(id)delegate 
               endSelector:(SEL)selector 
                    entity:(NSManagedObject*)object;

- (IBAction)saveAction:(id)sender;
- (IBAction)cancelAction:(id)sender;

@end

We retain a reference to the NSManagedObject that gets passed in, an NSObjectController which is initialized within Interface Builder and a NSUndoManager which we will discuss below. In addition to the iVars, we also have the accessor method that we used in the AppDelegate and two IBActions for the buttons on the edit sheet.

The interface builder xib only has a couple of objects:

The idea behind this design is that the window controller will receive an object which we will set into the NSObjectController. The panel/sheet will then feed from this NSObjectController to display all of the editable fields. Once the edit is complete the user will click either save or cancel which will close the sheet and cause the callback @selector to fire.

+ (void)editSheetForWindow:(id)window 
                  delegate:(id)delegate 
               endSelector:(SEL)selector 
                    entity:(NSManagedObject*)object;
{
  EditWindowController *controller;
  controller = [[EditWindowController alloc] initWithWindowNibName:kNibName];

  [controller setTestEntity:object];

  [NSApp beginSheet:[controller window] 
     modalForWindow:window 
      modalDelegate:delegate 
     didEndSelector:selector 
        contextInfo:object];
}

Within our helper method we take advantage of the existing underlying API. We first initialize an instance of the EditWindowController using the parent’s -initWithWindowNibName: which will load in the nib and attach all of the bindings for us. Once that is complete we use the -setTestEntity: method to pass in the NSManagedObject which will populate all of the fields. We then use the NSApplication call +beginSheet: modalForWindow: modalDelegate: didEndSelector: contextInfo: to present our associated window (a NSPanel really) as a sheet on top of the passed in window; which in this case happens to be the main window for our application. The beauty of using this method to present the sheet is that we do not need to retain the delegate or the callback method, the existing structure handles it for us.

- (void)saveAction:(id)sender;
{
  [[self window] orderOut:sender];
  [NSApp endSheet:[self window] returnCode:NSOKButton];
}

- (void)cancelAction:(id)sender;
{
  [[self window] orderOut:sender];
  [NSApp endSheet:[self window] returnCode:NSCancelButton];
}

We can further take advantage of the underlying structure in the save and cancel methods. In each of these methods the only thing we need to do is call -orderOut: on our window and then call -endSheet: returnCode: on the NSApplication itself. This will in turn call the delegate’s callback method with the appropriate return code. Keeps our sheet very simple and clean.

The interesting part of this sheet is the undo manager. In Interface Builder, I assigned the EditWindowController as the delegate to the window. This means that the window looks to the EditWindowController to get a reference to the NSUndoManager that it should be using. It is possible to use the NSUndoManager that is part of Core Data but we want to keep the edits made in this window local. Therefore we are going to create a separate NSUndoManager.

- (NSUndoManager*)windowWillReturnUndoManager:(NSWindow*)window
{
  if (!undoManager) {
    undoManager = [[NSUndoManager alloc] init];
  }
  return undoManager;
}

We create this NSUndoManager lazily. We wait until it is requested the first time and then initialize it.

Handling the Sheet Closing

When the user is done with the sheet they will either hit save or cancel. Both of those actions will cause the delegate’s callback method to be invoked.

- (void)editSheetDidEnd:(id)sheet returnCode:(int)returnCode object:(id)object
{
  [[[self managedObjectContext] undoManager] endUndoGrouping];
  if (returnCode == NSOKButton) return;
  [[[self managedObjectContext] undoManager] undo];
}

In the callback method within the AppDelegate we first end the undo grouping that we started just before presenting the sheet. We then check to see if the user pressed save or cancel. If they pressed save we return and are done. If they pressed cancel, however, we then request an undo on the NSManagedObjectContext‘s NSUndoManager. This will roll the NSManagedObject back to the state it was before we presented the sheet to the user. Because we named this action the user will actually see it in the Edit menu.

Handling Add Item

We can reuse this edit sheet for when we are dealing with a new entity as well as editing an existing entity. The only difference is in the AppDelegate‘s -addEntity: method.

- (void)addEntity:(id)sender;
{
  NSManagedObject *newObject;
  NSManagedObjectContext *moc = [self managedObjectContext];
  SEL endSelector = @selector(editSheetDidEnd:returnCode:object:);

  [[[self managedObjectContext] undoManager] beginUndoGrouping];
  [[[self managedObjectContext] undoManager] setActionName:@"Undo Object Add"];
  newObject = [NSEntityDescription insertNewObjectForEntityForName:@"TestEntity"
                                            inManagedObjectContext:moc];
  [EditWindowController editSheetForWindow:window
                                  delegate:self
                               endSelector:endSelector
                                    entity:newObject];
}

Instead of grabbing the selected object from the NSArrayController we construct a new one. However we set the undoGrouping to begin before we create the object. This allows us to include the creation of the NSManagedObject as part of the atomic undo. If the user hits cancel on the sheet, the NSManagedObject will automatically be removed from the NSManagedObjectContext for us.

Handling the Remove Item

Like the Add menu item above, we can use the NSUndoManager to make the removing of objects easy for us as well.

- (void)removeEntity:(id)sender;
{
  id object = [[arrayController selectedObjects] lastObject];
  if (!object) return;

  [[[self managedObjectContext] undoManager] beginUndoGrouping];
  [[[self managedObjectContext] undoManager] setActionName:@"Undo Object Remove"];
  [[self managedObjectContext] deleteObject:object];
  [[[self managedObjectContext] undoManager] endUndoGrouping];
}

Here we are grabbing the selected object from the table and if it exists we remove it from the NSManagedObjectContext. However, we make sure to give the action a name. Since it is a single action there is no need for a grouping but it is helpful, and improves the user experience to name the action so that the user understands exactly what they are undoing.

Wrap-Up

In this short article, we walked through how the NSUndoManager plays very nicely with Core Data and how we can use more than one NSUndoManager to keep the edits separate from each other instead of rolling them all into one undo stack and thereby avoiding all of the complication that entails.

It is possible to combine all of this work within one NSUndoManager but that causes the undo stack to be larger and more complicated unnecessarily. There is no reason to retain the edits that were made within the edit sheet in the same NSUndoManager that handles the changes to the context. By pulling these into their own NSUndoManager we can keep each separate function of the UI separate and easier to manage.

Download Sample Code

I tend to frequent the StackOverflow website and watch for questions about Core Data. Generally if it is a question I can answer I will and if it is a question I can’t answer then I really dig in for a fun session of exploration and attempt to find the answer. One question that I have been seeing with a tremendous amount of regularity is importing and displaying data on CocoaTouch devices. Generally the question comes from one of a few problems:

• Trying to import a large amount of data causes the UI to hang.
• Trying to save a large block of data causes the UI to hang.
• Saving on exit causes the OS to kill the app.
• Importing on launch causes the OS to kill the app.

All of these “problems” are symptoms of the same two issues.

Issues with large imports

Importing a large amount of data, either from disk or from a network takes time and memory. If this is done on the main/UI thread then it becomes obvious to the user because the user interface is slow or unresponsive. This leads to a poor experience for the user.

On the other side of this, once you get the data imported into Core Data then you need to save it. If you are using a SQLite database then it can take a considerable amount of time. If you are using a binary store then it will take even longer. Again a poor user experience.

The answer to this problem involves multi-threading and breaking the job down into smaller pieces.

Multi-threading

There is a persistent rumor that you cannot use Core Data in a multi-threaded environment. This is patently false. Core Data works very well in a multi-threaded environment but you need to play by the rules:

• One NSManagedObjectContext per thread.
• NSManagedObject instances cannot pass between threads.

As long as we follow those two rules, we can use Core Data in a multi-threaded environment.

Breaking It Apart

The biggest issue with imports is that they are generally huge. That is going to kill performance even if it is done on a background thread. This is especially apparent on Cocoa Touch right now because of the single core devices that it runs on. Therefore, the first thing we must do is break that import apart.

I recommend splitting the incoming data (XML/JSON/whatever) into multiple files that you save on disk. Number these files sequentially. When the data is fully transferred from the network then start processing them. In addition, keep track of which file you are currently working on. When that file is complete and saved, move on to the next file and update the marker. You can store the marker either in the NSUserDefaults or inside of the metadata of the NSPersistentStore itself. In either case the idea is that if the user quits in the middle of the import you do not need to start over at zero. You pick up right where you left off.

Code Example

In the attached sample project I have created a very trivial example of an import occurring on a background thread. If you were actually pulling in data from the net as I described above, you would actually have numerous NSOperation objects that are queued up to run. However in this example we have a single NSOperation that sleeps periodically.

The heart of this example is the periodic saves. We want to save frequently enough that each save is very quick. In addition, when the save occurs, we want to update the main NSManagedObjectContext so that it can refresh which will in turn update the NSFetchedResultsController which will in turn update the UI. Crazy enough? Let’s dive in.

RootViewController -startImport:

If you are familiar with Core Data on Cocoa Touch then most of this class will be very familiar. Other than code clean up it is straight from the current Xcode template. Having said that, there are a couple of differences.

The -startImport: method is fired from tapping the start button in the NSNavigationItem shown at the top of the screen. This method will create the NSOperationQueue if it has not already been created and then it will create our NSOperation which will do the simulated work in the background.

- (void)startImport:(id)sender
{
  [[[self navigationItem] rightBarButtonItem] setEnabled:NO];
  [[[self navigationItem] leftBarButtonItem] setEnabled:NO];

  if (![self operationQueue]) {
    operationQueue = [[NSOperationQueue alloc] init];
  }

  ZSImportOperation *op = [[ZSImportOperation alloc] init];
  [op setPersistentStoreCoordinator:[[self managedObjectContext] persistentStoreCoordinator]];
  [op setRunSpeed:0.25];
  [op setEntriesToCreate:1000];
  [op setSaveFrequency:10];

  [operationQueue addOperation:op];
  [op release], op = nil;
}

Note that we have several parameters that we are setting on the ZSImportOperation instance. This dependency injection allows us to pass along the NSPersistentStoreCoordinator to the operation so that it can construct its own NSManagedObjectContext once it starts. We cannot create the NSManagedObjectContext ourselves and pass it along because that will break the thread barrier and the results are undetermined (i.e. very bad).

We also tell it how many objects to create, how long to pause between each creation and how often to save (i.e. switch files). Once everything is configured we pass it along to the NSOperationQueue and it starts.

RootViewController -contextChanged:

This is where the magic happens as they say. This method is called whenever any NSManagedObjectContext in the entire application saves. We configured this in the -viewDidLoad method by calling:

[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(contextChanged:) name:NSManagedObjectContextDidSaveNotification object:nil];

When this method is fired we know that the data has changed somehow, somewhere. So we need to narrow it down and confirm it is a data change we care about.

- (void)contextChanged:(NSNotification*)notification
{
  if ([notification object] == [self managedObjectContext]) return;

  if (![NSThread isMainThread]) {
    [self performSelectorOnMainThread:@selector(contextChanged:) withObject:notification waitUntilDone:YES];
    return;
  }

  [[self managedObjectContext] mergeChangesFromContextDidSaveNotification:notification];
}

Fortunately we are not in a document based application so we really just need to check that the object pointer in the notification is not pointing to our NSManagedObjectContext. If it is, we bail.

We also need to make sure that all work done is on the main thread. Now I could just perform the last line of code in the -performSelectorOnMainThread:withObject:waitUntilDone: method and in this case I probably should. However this is another option in case the code you need to perform on the main thread is complex and you don’t want to write 12 -performSelectorOnMainThread:withObject:waitUntilDone: calls or write yet another method to be called. In this example, if we are not on the main thread we recursively call ourselves from the main thread and wait. This will guarantee that what happens next will always be on the main thread.

The final line is what triggers everything. This line of code is telling the main NSManagedObjectContext that the underlying data has changed and that it needs to update itself. This will in turn cause the NSFetchedResultsController to get woken up which finally updates the UI.

ZSImportOperation -main

This NSOperation subclass is the most trivial example I could come up with.

- (void)main
{
  ZAssert([self persistentStoreCoordinator], @"PSC is nil");
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

  [[NSRunLoop currentRunLoop] addPort:[NSPort port] forMode:NSRunLoopCommonModes];

  NSManagedObjectContext *moc = [[NSManagedObjectContext alloc] init];
  [moc setPersistentStoreCoordinator:[self persistentStoreCoordinator]];

  NSError *error = nil;

  for (NSInteger index = 0; index < [self entriesToCreate]; ++index) {
    id object = [NSEntityDescription insertNewObjectForEntityForName:@"Event" inManagedObjectContext:moc];
    [object setValue:[NSString stringWithFormat:@"User %i", index] forKey:@"name"];
    [object setValue:[NSNumber numberWithInteger:(arc4random() % 99)] forKey:@"age"];

    [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:[self runSpeed]]];

    if (index % [self saveFrequency] != 0) continue;

    ZAssert([moc save:&error], @"Error saving context on operation: %@\n%@", [error localizedDescription], [error userInfo]);

    DLog(@"saving background context");
    [moc reset];
    [pool release];
    pool = [[NSAutoreleasePool alloc] init];
  }

  ZAssert([moc save:&error], @"Error saving context on operation: %@\n%@", [error localizedDescription], [error userInfo]);

  [moc release], moc = nil;

  [[NSNotificationCenter defaultCenter] postNotificationName:kImportRoutineComplete object:self];

  [pool release], pool = nil;
}

It starts with creating a separate NSManagedObjectContext as per the rules. We then add an empty port so that we can lazily sleep this thread. From there we start looping based on the set entriesToCreate value.

Inside of the loop we create a new NSManagedObject and give it some random values. We then sleep based on the runSpeed value. From there we check to see if it is time to save and if not we move on to the next iteration of the loop.

If it is time to save we then perform a -save: on the NSManagedObjectContext instance wrapped inside of a ZAssert. The ZAssert allows us to do a conditional check on the results and then spit out either a NSAssert or NSLog depending on whether we are compiling in debug (which would throw an assertion) or production (which will spit out an NSLog).

Once the save is complete we reset the NSManagedObjectContext instance to release the memory it is holding onto and then drain the NSAutoreleasePool.

Once all of the requested objects have been created we perform one final save of the NSManagedObjectContext and drain the NSAutoreleasePool one final time. We also send out a NSNotification just so that we can update the UI and re-enable the buttons.

Wrap Up

If you run the sample application you will notice a few things:

• The app runs smoothly even while processing data.
• The app does not crash on launch due to the OS killing it.
• THe app does not crash on exit because the save takes too long.

Naturally this is more work for us as the developers but the end result is an application that can handle any amount of data without fear of freezing the UI or having the OS kill our application.

You can download the sample project here.

Schedule and Additional Information

The speaker line up for MacTech this year is amazing. I am quite honored to be sharing the stage with such a group of people.

What is MacTech Conference?

MacTech Conference is a 3-day, immersive, technical conference specifically designed for Apple developers, IT Pros, and Enterprise. “The whole idea of MacTech Conference is to allow members of the Apple community to meet and exchange ideas,” said Edward Marczak, Conference Chair and Executive Editor of MacTech Magazine. “This will be spurred on by presentations from some of the best and well-known experts in the community.”

MacTech Conference will have two separate tracks: one focused on programming / development, and the other on IT/Enterprise. Sessions will focus on both desktop and mobile, with appropriate levels of attention paid to the Mac, iPhone, iPad and iPod.

One scenario I’ve been asked about on more than one occasion recently is that of temporary, or transient entities. It seems that more and more developers have a need for temporary, or transient instances of entities in your Mac or iOS apps. Having temporary object instances can be useful and necessary in certain situations. Unfortunately, Transient Entities do not technically exist within the Core Data framework, however there are simple solutions to add temporary, un-persisted, data within the context of Core Data. Let’s go over some methods to effectively use the concept of Transient, or more appropriately Temporary Entities in Core Data without direct built-in support.

Why even put them in Core Data in the first place?

This should be the first qustion you ask when you’re talking about going down the path of doing something with Core Data that it doesn’t already do for you. So, why not just use Plain Old Objective-C Objects … POOCOs anyone? There are a couple scenarios in which you would want even your temporary objects to be NSManagedObjects. The main situation is where you actually do persist the instance to the persistent store. Another case may be that your data objects are easily defined, modeled and imported through the Core Data toolset. That is, you have an entity defined and related to other entities in your object graph in your Model, and the import code is shared or related. In this case, it may be more maintainable to just let that all coexist. The most common reason is that you want to have a temporary object with the option to cancel and destroy it at any time. This is common in a “New Document” situtation where the user wants to enter new data, but decides to cancel half way in, and discard what’s already been entered.

Whatever the reason, it should be through a rigorous process that you have ultimately decided that your objects are better off as NSManagedObject instances and not just plain old Objective C objects with references to NSManagedObjects. In most cases you will be better off not even dealing with the complexity of temporary NSManagedObjects in the first place.

That being said, since you’ve come this far, it seems likely that maybe you really do have a good reason to have a temporary object instance as a Core Data object. Great, let’s go over three common, and easy to implement, solutions with code to show how to handle this as simply as possible. Also, we’ll go over the pros and cons of each approach as there are certainly tradeoffs which you must evaluate in order to choose the correct solution for your problem.

The code samples shown here will be using the helpers found in Magical Record, available on github.

Scratch Managed Object Context

Perhaps the simplest scenario in creating temporary NSManagedObjects is to create them in a temporary NSManagedObjectContext. Apple’s documentation commonly refers to the NSManagedObjectContext as a ‘scratch pad’ or ‘working area’ for changes to NSManagedObjects. When you create any NSManagedObject in a Managed Object Context, it is not perisisted to the store util you call save: on that context. By not calling save: on a second scratch context, any changes made will dissappear when the context, and the managed objects it contains, is dealloc’ed. Let’s take a look at implementing this example in code:

@interface MyViewController ()
@property (nonatomic, retain) NSManagedObjectContext *scratchContext;
@end
 
@implementation MyViewController
@synthensize scratchContext = scratchContext_;
- (void)viewDidLoad
{
  self.scratchContext = [NSManagedObjectContext context];
}
...
- (void)importData
{
  MyEntity *instance = [MyEntity createInContext:self.scratchContext];
  // data initialization code
  // don't call save:
  [[NSManagedObjectContext defaultContext] save]; // won't save anything in the scratch context
}
@end

In this first example, we have two NSManagedObjectContext objects, the first is created for you when you use one of MagicalRecord’s setup methods. As described in a previous post, this first context is set as the default context for all Core Data operations. However, in this scenario, we want to create a second context that uses the same persistent store. Using MagicalRecord, this is as simple as using the context method. This will create a new NSManagedObjectContext instance, and initialize it with the default NSPersistentStoreCoordinator. This is all that’s necessary to create a usable scratch context. Now, instead of creating entities in the default context, perform them in this scratch context. This is what’s happening in the importData method. By creating a new entity in the scratch context, we’re making a temporary object that potentially can be thrown away when we’re done with the view controller.

In Memory SQLite Persistent Store

One of the useful features of SQLite is that it can also persist simply in memory, or system RAM, only. This feature translates to Core Data persistent stores quite nicely as it’s a simple configutation change to the addPersistentStore: call when setting up your NSPersistentStoreCoordinator instance.  To simplify things, while adding this option, we’re going to go over adding an in-memory SQLite store to the same default NSPersistentStoreCoordinator.  This route may not be the common route in iOS apps, but Core Data is set up to handle the mapping and complexity of multiple stores for you, so it makes sense to use that functionality for this solution to try to keep the code as simple as possible.  Let’s take a look at some code:

@interface SampleAppDelegate <UIApplicationDelegate>
@property (nonatomic, retain, readonly) NSPersistentStore *inMemoryStore;
@property (nonatomic, retain, readonly) NSPersistentStore *fileBasedStore;
@end
 
@implementation SampleAppDelegate
@synthesize inMemoryStore = inMemoryStore_;
@synthesize fileBasedStore = fileBasedStore_;
- (void)awakeFromNib
{
    [MagicalRecordHelpers setupDefaultCoreDataStack];
    self.fileBasedStore = [[[NSPersistenStoreCoordinator defaultCoordinator] persistentStores] objectAtIndex:0];
    self.inMemoryStore = [[NSPersistentStoreCoordinator defaultCoordinator] addInMemoryStore];  //configure in memory store<br />
}
...
@implementation MyViewController
//assign new object to in memory persistent store
- (MyEntity*)createTemporaryEntity
{
    MyEntity *newInstance = [MyEntity createEntity];
    NSManagedObjectContext *context = [NSManagedObjectContext defaultContext];
    NSPersistentStore *inMemoryStore = [(SampleAppDelegate *)[[UIApplication sharedApplication] delegate] inMemoryStore];
    [context assignObject:newInstance toPersistentStore:inMemoryStore];
    return newInstance;
}
@end

We’ve again use the MagicalRecord helpers in this second example to create an in-memory SQLite store on the default NSPersistentStoreCoordinator. Notice that we also must keep a reference to it in our app delegate. The reason for this reference is easy to understand once we get to creating temporary object instances in the createTemporaryEntity method on MyViewController. Creating an entity is as straight forward as always, however, we must then assign this new entity to the in-memory store using our saved reference. We must also assign our non-temporary objects to the fileBasedStore as well.

There is one rather large caveat to this solution–on iOS devices in particular. There is a finite limit to what you can store in memory during the lifetime of an app. Going over this limit can be grounds for termination by the system watchdog process. If you implement this solution and find that you are constantly hitting a memory limit in your app due to the temporary in memory store, a simple variant to this solution is to make this a regular disk based SQLite store, but make it a secondary store, one that can easily be deleted by some clean up code later on. By persisting the store to disk, you don’t have the same constant memory pressure, and you also still have a secondary store which only contains temporary instances of your objects.

Mark Entities for deletion, and delete later

In our third scenario, the Core Data stack remains the same as in our original simple Core Data stack setup. However, this situation is helpful when objects are being used by the UI. In some cases, references to managed objects in the UI may have a reason to delay the release of these objects. In order to avoid crashes due to deleting the persistence information out from under the object, we can simply say that entity should be deleted later. MagicalRecord provides a truncateAll (and the truncateAllInContext: variant) to remove all the instances of a particular type from the store. However, in this case, we need to have a more specific filter on a flag on the entity.

[MRCoreDataAction saveDataWithBlock:^(NSManagedObjectContext *localContext)
{
    NSPredicate *itemsToDeleteSearch = [NSPredicate predicateWithFormat:@"shouldBeDeleted = YES"];
    NSArray *itemsToDelete = [SampleEntity findAllWithPredicate:itemsToDeleteSearch];
    for (NSManagedObject *entity in itemsToDelete)
    {
        [entity deleteInContext:localContext];
    }
}];

In the case where the shouldBeDeleted flag is a transient attribute, using an NSPredicate search against the store will not work as this attribute is not even created in the SQLite schema for that entity. A variant on this solution is to hang on to those deleted references in an NSMutableArray (or NSMutableSet) and iterate on that collection at the appropriate time. However, this variant may backfire in the event of a crash or a forced termination. That is, in either of those scenarios, your collection of instances to delete will no longer be in memory.

On iOS 4 and above, applications can enter the “background”. This is a perfect time to perform this delayed purge as it can be done without the user really noticing any UI jerkiness caused by merges. Heck, if you make other apps slow down, the user may just blame that one! (Don’t do this…be a good app citizen).

Conclusion

I’ve just described three ways to deal with temporary Core Data objects. These are by no means the only ways to handle transient entities, however, these are relatively simple concepts. The code for each of these examples is also simple enough to retrofit into existing application code. If you have other solutions relating to the idea of temporary storage, I’d be happy to hear about them in the comments below!

Schedule and Additional Information

I am very pleased to announce that I will be speaking at this year’s 360 iDev conference in Denver, Colorado. Previously it was a foregone conclusion that I would be attending this conference since it was so close to my home. However, now that I am living in California, that is no longer the case.

However, I am happy to say that I will be attending this year despite a change in my home address. The speaker line up at 360 iDev keeps getting better each year and I am happy to be a part of this great lineup.

If you are on the fence about going to this conference then please take my advice; GO.

Not only is the speaker line up great but Denver has a great walking downtown and the after session events should be just as entertaining as the speakers themselves.

I look forward to seeing you there.

Schedule and Additional Information

I am very pleased to announce that I will be speaking in Brighton at the Update conference starting on September 5, 2011. The Update conference is a bit different than the normal “developer” conference that I attend and speak at. Instead of just or primarily focusing on the developer; a great deal of focus is put upon the user and the user experience. If you have been in attendance for any of Aral Balkan’s talks then you know very well what his focus is.

In addition to speaking at the conference I will be giving two one day workshops on Core Data. The first workshop will be an introduction to Core Data. I hesitate to call this a beginner workshop because you, the attendees, will be directing the workshop. If everyone wants to start at step one then we will. However throughout the day you will be setting the pace of what we learn and discuss. This format has worked very well in the past and the last time we did a workshop like this the day ended with some pretty advanced discussions.

The second session is labelled advanced and will be starting off with the assuming that we all know and understand the principals behind Core Data. In this session we will deep dive into some of the more complex areas of Core Data. Again, I have a list of topics but you, the attendees, drive the session. Therefore it is intentionally open ended so that you can come with your questions and problems and we can discuss them together. I plan on wrapping up the second workshop with some real world examples of edge cases and situations I have run into and how they were solved.

I invite you to join me in Brighton this September to attend Update 2011. I expect it to be a great time.

I was part of the original development team that wrote “The Daily” for News Corp.

When “The Daily” was released, we expected some issues with it. Every developer should expect issues with a 1.0 release. We knew that it was written under a tight deadline and that there were most certainly sharp edges that we had not identified. The application was about as well received as we could expect from the user base. Some people do not like Rupert Murdoch. Some people do not like his teams slant on the news. That was expected. He is good at polarizing his readership.

The response we got from the developer community was a complete surprise.

Back In The Day

One of the things that originally attracted me to being a Mac/Cocoa developer was the community. I remember, fondly, watching two direct competitors sitting across a table from one another trading code solutions. Both knew that the other person was going to add it to their code base and they shared that information anyway. Both developers knew that it was the other developer that was driving them to make a better product. There can be no first place without a second place. Having a close competitor helps drive us forward.

Likewise, going to a CocoaHeads meeting you would see senior developers, developers who have been doing this for decades, sitting next to developers who are just now wrapping their heads around retain/release. They weren’t pained but instead looking upon the younger developer with the fondness of remembering what it was like when they started. Perhaps a bit of envy of all the exciting things this new young developer is about to learn.

Introducing The iPhone

When the iPhone was introduced we were all very excited to start working with it. Cocoa on a mobile platform! We could not wait to get our hands on it. The community embraced it immediately.

At first you had to hack your phone to write code for it but we quickly got an SDK. The community consumed that SDK with a hunger. The forums were active, the boards were active, the blogs were hot. Everyone wanted to share the new cool thing they learned or share the pain they just went through so that, hopefully, no one else had to go through that pain. The community was alive and prosperous.

I do remember many conversations about what would happen to the community with this sudden influx of developers. Many conversations started over what to do about these developers invading our platform. When I was involved in those discussions my response was always the same; we would welcome them to the community and teach them how we work. We would show them how great it is over here and that we share our toys. We would welcome them knowing that they would become part of us; not the other way around.

For the most part that is what happened. A large portion of these developers joined us. They shared; many new blogs started up, new books were written and our NSCoder nights and CocoaHeads meetings grew. The community became stronger with this influx of new developers.

A Disturbing Trend

More recently, perhaps within the last year, there has been a disturbing trend in the community. Surprisingly, sadly, this trend has not come from the new developers but from some of the older grey beards. There has been a trend to “piss on” things written by other developers. A new app comes out, good or bad, and the claws come out. People are quick to blast it; the more press it gets, the more it gets blasted.

A new photo sharing idea that got a lot of VC money? Blast it!

A new game concept? Kill it!

An old idea rehashed in a new way? Destroy it! Burn them!

He got more press than me? He must die in a fire!

I would almost expect this from new developers who just joined the community. However it is not the new members of this community that are all aflame. Most likely they are looking at these ideas and seeing how they can apply them or if there is anything to learn from them.

Sadly though, it is the older, established, members of the community that are turning so hostile. Gone is the sharing and the live/let live attitude that once made this community so great. Quite a few members are just full of piss and vinegar.

You can say what you want about me. I do not hold myself above others and know that my code sucks more often than not. I tend to share that opinion of my code often. I do not share what I have learned to gain glory but to just enjoy the act of sharing. Sharing makes me feel good.

What saddens me is this new desire to attack things that are either new or just in the media. Does the application suck? Maybe. But to curse the developers who wrote it? Not cool.

We as developer must remember that we are not the target for 99% of the software that is written. We are not the final judge on what will or will not work. If anything, we are the last people that should have an opinion on what is good or bad. We should be the ones that step back and watch what the “normals” do with it. They determine the success or failure; not us.

Some Thoughts From The Trenches

Not everyone who reads this blog is an independent developer. Not everyone who reads this blog writes code to fit someone else’s goals so I would like to share a few points from someone who is an independent developer and writes code to meet the expectations from others.

Deadlines are a bitch

Rarely do I have the opportunity to set my own deadlines. Companies who hire development shops almost always have a deadline in mind before the first line of code is ever written. More often than not, they have a deadline in mind before they have all of the requirements defined.

We, as developers, do not get to move those deadlines without massive and dire consequences. Usually our only opportunity to move them is at the beginning and then by saying “No, Thank You” to the project. Moving a deadline mid project usually has financial consequences.

Non-Developers Think This Is Easy

Non-developers see that one new feature as trivial. They see that crash as obvious and do not understand what the hell is taking so long. Business people are non-developers. The people who sign checks are business people.

To go along with my point about deadlines, getting paid involves meeting the requirements of the project within the deadline. These are almost always at odds with each other. The requirements are far greater than can be fit within the deadline. The response from the business team is “add more people, make it happen”. We as developers know that adding more people will actually slow things down.

The First 90% Is Easy

Developers frequently look at a problem and in their first blush say “oh I see how that works, that is simple“. That is a trap my friends. Everything looks easy at first. My favorite recent example is the carousel of The Daily. It is trivial right? No, there are a huge number of dependencies and features and bells and whistle and edge cases that all work their way into that piece of the application. The core of moving images around on the screen is trivial. Anyone can do it in a couple of hours. However to complete all of the requirements? Months of work. Personally I am still not completely happy with it. Maybe with 6 more months of polish it will get to where I would like to see it.

Never underestimate the requirements that go into a piece of functionality. Don’t assume that the developers are incompetent if it doesn’t work or perform the way you expect it to. Chances are that there are requirements that you as a user are unaware of.

Software Development Is Not The Only Cost

This is my one hostile point in this post. Whoever thought the iOS developers got paid $30 Million on “The Daily” is a moron.

“The Daily” is a lot bigger than just writing iOS code. There is a huge editorial staff, there is a server team, there is office space, hardware, travel, the list goes on and on. The actual iOS code was a small fraction of the huge effort that was put into creating a digital only daily newspaper; the first of its kind by the way.

Doing Something First Sucks

Unless you are trying to get into some record book, the person who does something first is rarely remembered. He who does it best is often remembered fondly. When you are the first to do something you rarely are the best at it. This is just a simple fact of life. When you are the first to do something you are making shit up as you go along. You are guessing. You have no clue what is going to happen when you release. The first MP3 player? Shit. First Tablet? Horrible.

Being first means that you are going to be superseded. Either by improving on your design or by being replaced by another team that took your idea and ran with it. The iPod blew away the first MP3 player. The iPad blew away the first Tablet. I look forward to seeing who becomes the greatest digital only daily newspaper on a tablet. Maybe that will be The Daily via iteration or maybe it will be someone else.

Conclusion

“Be excellent to each other” — Bill & Ted’s Excellent Adventure

This should be the guiding principal of the Cocoa Developer Community in my not-so-humble opinion. It is one of the things that make it great. New people are welcomed and the older members are honored.

There is no reason to hate other development efforts. It does not matter if that developer is better or worse than you. It does not matter what that developer wrote. There is plenty of room for all of us.

Be excited by his or her success. His or her spotlight does not put you in darkness.

I look forward to seeing each of you next week in San Francisco. If you see me please feel free to say hi.

WWDC is once again upon us; be excellent to each other.

Publishing lead times being what they are, this post covers information not included in the book.

The Ugly/Crude Way

I’m going to assume that the incoming JSON more or less matches your managed object, i.e. that you have a JSON dictionary with keys that match the attribute names of your Core Data entities. Given a dictionary jsonDict that has been created by parsing incoming JSON data using a JSON library (TouchJSON and json-framework are good choices) and a managed object imaginatively called myObj, the dead simple approach is to just do something like:

[myObj setValue:[jsonDict objectForKey:@"name"] forKey:@"name"];
[myObj setValue:[jsonDict objectForKey:@"city"] forKey:@"city"];
[myObj setValue:[jsonDict objectForKey:@"phone"] forKey:@"phone"];

…and on and on and on ad nauseum. It works but it’s astoundingly ugly. It also makes maintenance more challenging, since any changes to the data structure require changes not only to your Core Data model and related managed object classes but also to your data import code.

The Easy/Dangerous Way

Cocoa provides a method that makes this much, much simpler. Thanks to key-value coding there’s a method in NSObject that reduces the above to a single line:

[myObj setValuesForKeysWithDictionary:jsonDict];

This will run through key/value pairs in jsonDict and set the same key/value pairs on myObj. The only caveat is that the key names must match, but that shouldn’t be a problem assuming you’ve named them the same in your data model.

Actually there’s one more caveat, and it’s a biggie. In this method, the keys contained in the dictionary determine what keys are used on the managed object. But what if the dictionary contains a key that’s not part of the entity description? Then your app crashes with an error about how the object is not key-value coding compliant for the key. Using this method here puts your app’s stability at the mercy of the web service you’re using. If you don’t have absolute control over the web service, you’re running serious risks using this method here. We can do better.

Inspecting Managed Objects

Since we’re using Core Data, we can use NSEntityDescription to inspect a managed object’s attributes and turn the logic above around. Instead of using the incoming dictionary to determine what key/value pairs to use, we can use the managed object.

You can look up the entity description for a managed object by asking for its entity. That gives you an NSEntityDescription which you can then ask for all kinds of useful information, like what attributes exist. That leads to a safer approach when converting from JSON:

NSDictionary *attributes = [[myObj entity] attributesByName];
for (NSString *attribute in attributes) {
    id value = [jsonDict objectForKey:attribute];
    if (value == nil) {
        continue;
    }
    [myObj setValue:value forKey:attribute];
}

This code iterates over the entity’s attributes, looks up a key in the dictionary for each of them, and applies that key/value pair to the managed object. That’s still nice and generic, and has the advantage that changes to incoming data won’t crash the app any more. It looks up values in the dictionary only for attributes that actually exist in the entity description. Extra dictionary keys are simply ignored.

The check for nil is there because the code does hit all of the entity’s attributes. If the entity has any attributes that aren’t present in the incoming data (a “favorites” flag, for example) the code would end up setting a nil value for the attribute. That could lead to accidentally wiping out data that you really want to keep.

Handling Broken and Inconsistent JSON

The JSON standard is quite clear about how to distinguish strings from numbers– basically, strings are surrounded by quotes and numbers are not. JSON web services however, are not always good about following this requirement. And even when they are, they are not always consistent from one record to another.

An example, similar to one I encountered recently: Size information for clothing. Sizes are provided by the web service and are typically something like “30-32″, “34-36″, etc. These are correctly quoted as strings in the JSON, and the app saves them as string attributes and displays them to the user as is.

But sometimes the size just has one number, e.g. “8″, “10″, etc. In this case the server drops the quotes, making them numbers. My JSON parser correctly produces an NSNumber. Only I want to save this in my entity’s string attribute! I can use Objective-C introspection to see if I received an NSString or an NSNumber from my JSON parser, but I also need to know what type the managed object expects for the property. I briefly considered breaking my JSON parser so that it would always return NSString, so at least I would know what to expect from it. Fortunately NSEntityDescription came to the rescue again.

Besides asking the entity description what its attribute names are, you can also inquire about the attribute types configured in the Core Data model. This is returned as an NSAttributeType. Using this, we can expand the code above to handle mismatched data types. It’s probably a good idea to abstract the code for easy reuse, too, so I’ll put it in a category on NSManagedObject:

@implementation NSManagedObject (safeSetValuesKeysWithDictionary)
 
- (void)safeSetValuesForKeysWithDictionary:(NSDictionary *)keyedValues
{
    NSDictionary *attributes = [[self entity] attributesByName];
    for (NSString *attribute in attributes) {
        id value = [keyedValues objectForKey:attribute];
        if (value == nil) {
            // Don't attempt to set nil, or you'll overwite values in self that aren't present in keyedValues
            continue;
        }
        NSAttributeType attributeType = [[attributes objectForKey:attribute] attributeType];
        if ((attributeType == NSStringAttributeType) && ([value isKindOfClass:[NSNumber class]])) {
            value = [value stringValue];
        } else if (((attributeType == NSInteger16AttributeType) || (attributeType == NSInteger32AttributeType) || (attributeType == NSInteger64AttributeType) || (attributeType == NSBooleanAttributeType)) && ([value isKindOfClass:[NSString class]])) {
            value = [NSNumber numberWithInteger:[value  integerValue]];
        } else if ((attributeType == NSFloatAttributeType) && ([value isKindOfClass:[NSString class]])) {
            value = [NSNumber numberWithDouble:[value doubleValue]];
        }
        [self setValue:value forKey:attribute];
    }
}
@end

This code inspects the types of both the value found in the dictionary we created from the incoming JSON and the attribute on the managed object, and if there’s a string/number mismatch it modifies the value to make sure it matches what’s expected.

This is getting pretty useful. Not only is it a generic JSON-to-NSManagedObject conversion, it also handles mismatches between numeric and string types without needing any entity-specific information. It could be even better, though.

Handling Dates

JSON does not have a date type, but dates are nevertheless common in JSON. They’re just represented as strings using one date format or another. Only you probably want an NSDate, not a string. NSDateFormatter is really useful here, but wouldn’t it be nice to make the date conversion generic as well, so you don’t need to special-case your date attribute? Can you see where I’m going with this?

Having written the category method above, it’s not much more work to add an optional NSDateFormatter argument, and then to use it whenever an entity’s attribute expects a date. This modified version adds that argument, and an extra case to the “else … if” chain:

@implementation NSManagedObject (safeSetValuesKeysWithDictionary)
 
- (void)safeSetValuesForKeysWithDictionary:(NSDictionary *)keyedValues dateFormatter:(NSDateFormatter *)dateFormatter
{
    NSDictionary *attributes = [[self entity] attributesByName];
    for (NSString *attribute in attributes) {
        id value = [keyedValues objectForKey:attribute];
        if (value == nil) {
            continue;
        }
        NSAttributeType attributeType = [[attributes objectForKey:attribute] attributeType];
        if ((attributeType == NSStringAttributeType) && ([value isKindOfClass:[NSNumber class]])) {
            value = [value stringValue];
        } else if (((attributeType == NSInteger16AttributeType) || (attributeType == NSInteger32AttributeType) || (attributeType == NSInteger64AttributeType) || (attributeType == NSBooleanAttributeType)) && ([value isKindOfClass:[NSString class]])) {
            value = [NSNumber numberWithInteger:[value integerValue]];
        } else if ((attributeType == NSFloatAttributeType) &&  ([value isKindOfClass:[NSString class]])) {
            value = [NSNumber numberWithDouble:[value doubleValue]];
        } else if ((attributeType == NSDateAttributeType) && ([value isKindOfClass:[NSString class]]) && (dateFormatter != nil)) {
            value = [dateFormatter dateFromString:value];
        }
        [self setValue:value forKey:attribute];
    }
}
@end

Conclusion

Saving JSON data to a managed object is one of those things that’s not as easy as it seems at first glance. Making it happen is easy enough, but making it happen safely in maintainable code can quickly get complicated. Fortunately, Core Data has your back and will help you work out what needs to happen along the way.

We are involved in a lot of contract work and frequently our clients will give us video files. These files can be demos, mock ups, advertisements, etc.

As a developer on the move, I like to keep these with me and on my iPad so that I can review them whenever I want.

Of course, Client A does not want Client B to be able to accidentally see their files. Lawsuits happen that way.

To make this even more fun, everyone wants to borrow your iPad. Everyone wants to touch it and play with it. This increases the risk of the videos getting into the wrong hands.

From this, SecurVid was born.

Features

SecurVid is designed to put videos behind a password lock. The videos are kept encrypted on disk and are not backed up to your computer that you sync your iPad with. This means that it is not designed to be a storage system for your videos but a protection for videos you need access to but don’t want everyone who borrows your iPad to see.

SecurVid has a lossy way to reset its password in case you forget it. Enter the wrong password three times and it will offer to reset for you. However that reset comes at a price; it will first delete all of your videos before offering to reset the password.

Once you are into SecurVid you can watch the videos you have already loaded naturally. You can also add more videos via iTunes as well as from any other application that is configured with file sharing. Therefore if a client emails you a video you can tap and press on the video file in mail on your iPad and load it into SecurVid.

Likewise you can export videos from SecurVid either for emailing or for loading into another application. You can even mark a video as “unprotected” and allow iTunes to see it next time you plug in your iPad.

The Future

This is the 1.0 release of SecurVid and has the core features that I/we needed in the application. However it is just the beginning for this application. As the application matures we will be adding additional features and continuing to release updates.

Cost

Because the 1.0 has only the core set of features we are offering it at an introductory price of $4.99. That price will remain until the next set of features are added. When that occurs we will increase the price up to the next appropriate level. However if you bought the 1.0 you will get these upgrades for free.

SecurVid HD is available on the App Store now.

Whenever most cocoa developers mention threading and Core Data in the same sentence, the reaction I see most often is that of mysticism and disbelief. For one, multithreaded programming in general is hard–hard to design correctly, hard to write correctly, and debugging threads is just asking for trouble. Introducing Core Data into that mix can seem like the straw that broke the camel’s back. However, by following a few simple rules and guidelines, and codifying them into a super simple pattern, one that may be familiar to you, we can achieve safer Core Data threading without the common headaches.

Another reminder: the examples in this blog post will refer to the open source project on Github, MagicalRecord, which is basically several helpful categories and classes I’ve written to extend the standard Core Data APIs.

What would Apple do?

Let’s start by looking up in the docs (see Concurrency with Core Data) to see what needs to happen in order to set up Core Data to pass data objects across threads and perform save operations in the background. Summarizing the main points from the recommended approach:

• Each thread should have its one NSManagedObjectContext
• NSManagedObjects themselves are not thread-safe
• NSManagedObjectIDs are thread-safe
• If you save in the background, you need to merge your changes to other contexts using one of the Core Data system notifications, NSManagedObjectContextDidSaveNotification.

How do we do this in code? The most common solution is to create a custom NSOperation, and make sure we create and set up our new context in the main method of the operation, like so:

#import "MGPCoreDataOperation.h"
 
@interface MGPCoreDataOperation ()
 
@property (nonatomic, retain) NSManagedObjectContext *context;
 
@end
 
@implementation MGPCoreDataOperation
 
- (void)main
{
	[self setContext:[NSManagedObjectContext context]];
 
	//perform my core data operations here
}
 
@end

Remember to create the NSManagedObjectContext in the main method of your custom NSOperation, and not the init method. The main is the only one of the two that is guaranteed to be called from another thread. Your secondary NSManagedObjectContext needs to be created inside the thread where it will be used.

This solution is fairly straightforward, however, let’s think beyond this class. Using this approach, while a valid solution, is somewhat cumbersome to reuse. For every save operation using this technique you will:

• need to subclass, or create this class via a template every time
• instantiate this objects on the main thread, and give it to an NSOperationQueue
• have to pass in object ids through a property

This solution is for the heavy lifting algorithms such as parsing data in the background and saving it in once the parse is done, and all on a background thread. Let’s try a solution that is a little lighter weight, a little easier on the eyes, and uses one of the more fun and recent additions to Objective C–blocks and GCD.

A small idea

Before we get started with our threading adventure, we need to first seed our minds with a small idea that will help us for the remainder of this post. In the context of threading and UI, we need to examine the concept of a Main Thread. The main thread is different from other threads in that it is considered the foreground thread, and is the thread which runs the main runloop, which in turn, processes messages sent from outside your app such as taps or clicks, and sends them to your application. From this example, let’s introduce the concept of a Main Managed Object Context. This context is meant to only be accessed from the same thread as the UI. This context is also always available, as is the Main Thread, and Main Runloop. In the MagicalRecord helpers, I have implemented a defaultContext class method that provides access from anywhere.

The default Xcode Core Data application template promotes this idea by virtue of putting a reused managed object context in the app delegate. Both the app delegate, and the managed object context are implied singletons. MagicalRecord also keeps it’s default context in a single place for easy reuse in your applications, and is also a singleton-like pattern. But, unlike the application templates, it keeps its reference away from your app delegate. Technically speaking, moving the shared NSManagedObjectContext out of the app delegate is merely a preference, but my preference is to keep the application delegate as free from actual application responsibility as possible.

This main context provides some neat features. First, it let’s us simplify the find methods described in my previous post, while still having the option for a context parameter. Second, this lets us get started with Core Data very quickly since we can do fetching from the main thread on the main context, and not have cross thread issues. And third, and most importantly, we have a single context to which any background changes are merged. And if those benefits weren’t helpful enough, changes to observed managed object instance can immediately be acted upon by any UI components using KVO.

Core Data, Blocks and GCD…oh my!

Multithreading in Core Data is important when it comes to saving data in the store. While complex fetches can certainly take quite a bit of time to return, most frequently the bottleneck is during a save. As such, the goal in this solution is to make background saving easy and safe.

This block based solution is inspired by one of the newer iOS APIs for animating views. Blocks are incredibly powerful, and for Core Animation code, have greatly simplified animation code. A simple animation now looks like this:

UIView *redView = ...;
[UIView animateWithDuration:5.0
		 animations:^{
			redView.alpha = 0.0;
		 }
 ];

Can we use this paradigm to save Core Data objects on a background thread? What if we had a save API resembling this animation API? Let’s first think about the synchronous (aka. easy) case. The API would probably look something like this:

+ (void) saveDataInContext:(void(^)(NSManagedObjectContext *context))saveBlock;

When using this API our code will resemble this sample:

PersonEntity *person = ...;
NSManagedObjectID *objectID = [person objectID];
[NSManagedObjectHelper saveDataInContext:^(NSManagedObjectContext *localContext){
	PersonEntity *localPerson = (PersonEntity *)[localContext objectWithID:objectID];
 
	//make my updates to localPerson here
	localPerson.name = @"IronMan";
	//...more changes
}];

While it’s not terribly important in this particular example because it’s all running in the same tread, I must remind you that NSManagedObjects are NOT thread-safe, but NSMangedObjectIDs are, so we need to pass an NSManagedObject’s object ID across the thread, or in this case, block boundary.

In order for this code sample to work, the implementation of the saveDataInContext method should be:

• creating a new NSManagedObjectContext instance, AND
• setting up the both contexts with the proper merge policy, AND
• setting up the main context to listen to the background context to merge on save,
• sending that context as the parameter into the block for use by other code.

This would make creating lightweight save or update operations as easy as creating a UIView block-based animation.

So, what would the body of this method look like? Let’s start the first item on the list.

+ (void)saveDataInContext:(void(^)(NSManagedObjectContext *context))saveBlock
{
	NSManagedObjectContext *context = [NSManagedObjectContext context];			//step 1
	[context setMergePolicy:NSMergeByPropertyObjectTrumpMergePolicy];				//step 2
	[defaultContext setMergePolicy:NSMergeObjectByPropertyStoreTrumpMergePolicy];
	[defaultContext observeContext:context];						//step 3
 
	block(context);										//step 4
	if ([context hasChanges])								//step 5
	{
		[context save];  //MagicalRecord will dump errors to the console with this save method
	}
}

This code uses some MagicalRecord helper methods, but the steps as implemented are:

• Step 1: Create the new context
• Step 2: Set the merge policy
• Step 3: Set up the notification
• Step 4: Call back our block with our newly created context that’s been set up properly.
• Step 5: Save the context if it has any changes

Notice also, that in order for a merge to occur automatically, one context has to be set to win during the merge. In this case, we want the background context to win by telling the main context that we want the information in our data store to be the correct answer in the case of a conflict. Also notice that much of the setup of a second context is as simple as calling a single method. We can now build upon this simple foundation and introduce background threading for our saves. Since we’re using blocks, we can also take advantage of the Grand Central Dispatch APIs.

A background operation using our synchronous method can easily be written using GCD:

+ (void)saveDataInBackgroundWithContext:(void(^)(NSManagedObjectContext *context))saveBlock
{
	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
		[self saveDataInContext:saveBlock];
	});
}

Going the Distance

But, can we do more? What is one neat feature of the UIView block animation APIs? Their ability to call a block when the animation has completed. How useful would it be to have a block of code called exactly after the changes to your object have been saved and merged? I thought so; let’s look again at a UIView animation API for inspiration:

+ (void)animateWithDuration:(NSTimeInterval)duration animations:(void (^)(void))animations completion:(void (^)(BOOL finished))completion

UIView *redView = ...;
[UIView animateWithDuration:5.0
		 animations:^{
			redView.alpha = 0.0;
		 }
		 completion:^(BOOL  completed){
			[redView removeFromSuperview];
			[redView release];
			redView = nil;
		 }
];

To be safe, let’s make sure our completion block is called on the main thread while we’re at it.

+ (void)saveDataInBackgroundWithContext:(void(^)(NSManagedObjectContext *context))saveBlock completion:(void(^)(void))completion
{
	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
		[self saveDataInContext:saveBlock];
 
		dispatch_sync(dispatch_get_main_queue(), ^{
			completion();
		});
	});
}

Now, you can save and reload, or update your data in a single place sort of like this:

 
NSArray *listOfPeople = ...;
[NSManagedObjectHelper saveDataInBackgroundWithContext:^(NSManagedObjectContext *localContext){
	for (NSDictionary *personInfo in listOfPeople)
	{
		PersonEntity *person = [PersonEntity createInContext:localContext];
		[person setValuesForKeysWithDictionary:personInfo];
	}
} completion:^{
	self.people = [PersonEntity findAll];
}];

This small block of code does a lot for you. It,

• creates and sets up a background Managed Object Context
• creates and sets up a background GCD queue
• saves the background context, which was set up to notify our Managed Object Context in the Main thread
• calls back your completion block on the main thread AFTER the changes have been pushed all the way to the persistent store

However, there is one more thing we could add to help keep our saving simple. As it is now, the background operations are performed on a global background queue. I’ve encountered times where it is necessary to have a single dedicated GCD queue only for Core Data operations. In MagicalRecord, we’ve defined a simple static function that allows access to a write only GCD queue.

static dispatch_queue_t coredata_background_save_queue;
 
dispatch_queue_t background_save_queue()
{
    if (coredata_background_save_queue == NULL)
    {
        coredata_background_save_queue = dispatch_queue_create("com.magicalpanda.coredata.backgroundsaves", 0);
    }
    return coredata_background_save_queue;
}

And this would only modify our background core data save method like so:

+ (void)saveDataInBackgroundWithContext:(void(^)(NSManagedObjectContext *context))saveBlock completion:(void(^)(void))completion
{
	dispatch_async(coredata_background_save_queue(), ^{
		[self saveDataInContext:saveBlock];
 
		dispatch_sync(dispatch_get_main_queue(), ^{
			completion();
		});
	});
}

Using blocks, GCD and a little bit of convention, we’ve helped to make multithreaded Core Data code much easier to write and stay consistent with Apple’s recommended methods. Not only that, but this solution consolodates much of the manual boilerplate setup required to properly save data in the background and safely update data throughout your applications.

Core Data, for both iPhone and Mac, is a very powerful framework for persisting your objects out of memory, and into a more permanent storage medium. With the enormous power of Core Data, it can be easy to slip into the trap of thinking that Core Data is very complex.

Easy Fetches

When I was first learning about Core Data, I naively thought that getting data from a data store would be easy. Core Data is a framework for fetching and persisting data, after all! I eventually learned that the minimum code required in order to fetch data from the store resembled this:

NSManagedObjectContext *context = [[[UIApplication sharedApplication] delegate] managedObjectContext];
NSFetchRequest *request = [[NSFetchRequest alloc] init];
 
NSEntityDescription *entity = [NSEntityDescription entityForName:@"MyEntity" inManagedObjectContext:context];
[request setEntity:entity];
 
NSPredicate *searchFilter = [NSPredicate predicateWithFormat:@"attribute = %@", searchingFor];
NSError *error = nil;
NSArray *results = [context executeFetchRequest:request error:&error];

This is quite a bit of code for pulling data out of a store, and this doesn’t even include error handling. I’ve often seen this code copied and pasted where needed in other projects. Some developers even use code snippets to generate these fetch requests. All that similar code can easily become a nightmare to maintain once your app starts to contain many entities. What I wanted was a little more automagic.

About a year ago, I ran across this post by Matt Gallagher on his terrific blog Cocoa with Love, which introduced the idea of the “one line fetch”. The blog post had a good idea, and a simple example of how to implement such an idea, however it occurred to me there were some ways I could enhance the example. The first was the use of the entity name in the method as a string parameter. This is required for the case that there are no custom NSManagedObject subclasses for your entities. And second, the example still required the helper methods to be part of your view controllers.

One reason to use helper methods to generate your fetch requests is that your code remains dry. That is, you remove duplication. Once you start using Core Data, you quickly realize that many requests start to look alike after a while. By consolidating this logic into a single method, or set of methods, we can be sure that optimizing one request can improve many requests. But perhaps more important is that by condensing the code, and removing the repetitive parts, you can easily see at a glance what the code intends to do, with the details only a method call away.

Dynamically Generating your Fetch Requests

In order to request data from your Core Data store, you must start with an NSFetchRequest. As the sample request above showed, we’ll need to at least set the entity for the data we’re intending to fetch.

I always use mogenerator to generate custom subclasses for my Core Data entities. With each custom subclass, you get some valuable information that can help ease the process of building helper methods to autogenerate fetch requests. The primary bit of useful information is a Class object that is related to your Entity. If your entity names and class names are identical, it’s fairly easy to derive a class method to create the correct NSEntityDescription at runtime by using the NSStringFromClass() function. However, with mogenerator, a convenience method that returns the entity description required for fetch requests is auto generated for you called entityInManagedObjectContext:.

When you have a class method, the self object refers to the current Class object, from which you can get a string to look up the proper entity description required for a lookup. This is a big step in autogenerating requests. In the case of mogenerated code, we can check if the current Class responds to a particular selector, entityInManagedObjectContext:, and call that instead. For now, assume that the following methods are class methods in an NSManagedObject subclass called MyEntity.

+ (NSEntityDescription *)entityDescriptionInContext:(NSManagedObjectContext *)context
{
    return [self respondsToSelector:@selector(entityInManagedObjectContext:)] ?
        [self performSelector:@selector(entityInManagedObjectContext:) withObject:context] :
        [NSEntityDescription entityForName:NSStringFromClass(self) inManagedObjectContext:context];
}

Using this method, we can dynamically determine the entity for which we should fetch data. So, let’s start by creating a class method to find all instances of a particular entity in the store:

+ (NSArray *)findAllObjects
{
    NSManagedObjectContext *context = [[[UIApplication sharedApplication] delegate] managedObjectContext];
    NSEntityDescription *entity = [self entityDescriptionInContext:context];
    NSFetchRequest *request = [[[NSFetchRequest alloc] init] autorelease];
    [request setEntity:entity];
    return [context executeFetchRequest:request error:nil];
}

And using this method from, for example a View Controller in our app, we could simply call:

[MyEntity findAllObjects];

The niceness of this syntax over the fully verbose glob is that at a glance I can quickly see what’s going on. In this case, I want all the instances of MyEntity in the store, in no particular order. Simple and to the point without having to decipher a series of NSFetchRequest customizations (which would be minimal for this example). In this case, we’re passing a message to the class method on MyEntity.

Next, we need to set some default error handling, and we have a simple way to find all objects in a store that are MyEntity instances.

+ (NSArray *)findAllObjects
{
    NSManagedObjectContext *context = [[[UIApplication sharedApplication] delegate] managedObjectContext];
    NSEntityDescription *entity = [self entityDescriptionInContext:context];
    NSFetchRequest *request = [[[NSFetchRequest alloc] init] autorelease];
    [request setEntity:entity];
 
    NSError *error = nil;
    NSArray *results = [context executeFetchRequest:request error:&error];
 
    if (error != nil)
    {
      //handle errors
    }
    return results;
}

I’m not explicitly adding error handling here, however inspecting the error object will give you some useful information to track down Core Data issues, such as validation errors, that might prevent your data from being saved. But there are two problems here. First, we don’t always want to use the default context from the Application Delegate. Second, and more importantly, we should be able to use this for other entities in the store.

Addressing the first issue–the matter of specifying the context–this is important because there will be cases where we need to perform some operation on this data in the background on another thread. The proper way to do that in Core Data is to create another context, so let’s make the context a method parameter:

+ (NSArray *)findAllObjectsInContext:(NSManagedObjectContext *)context
{
    ...
}

and we can still use the simple version of the find method like this:

+ (NSArray *) findAllObjects
{
    return [self findAllObjectsInContext:[[[UIApplication sharedApplication] delegate] managedObjectContext]];
}

So, now we have two methods that help find entities in our store, and the context is an injectable dependency. Fancy…and helpful for those cases involving threads.

The other problem was the matter of reusing this method across any custom NSManagedObject subclass. All objects that store data for you in Core Data are instances of or subclasses of NSManagedObject. The easiest way to reuse this method in this case is via a category. However, by placing these methods in a category of NSManagedObject, rather than the MyEntity subclass, all subclasses of NSManagedObject will inherit them at runtime. This means all entities in our store will have this nice helper method for retrieving all instances.

//NSManagedObject+EasyFetching.h
@interface NSManagedObject (EasyFetching)
 
+ (NSEntityDescription *)entityDescriptionInContext:(NSManagedObjectContext *)context;
+ (NSArray *)findAllObjects;
+ (NSArray *)findAllObjectsInContext:(NSManagedObjectContext *)context;
 
@end
 
//NSManagedObject+EasyFetching.m
@implementation NSManagedObject (EasyFetching)
 
+ (NSEntityDescription *)entityDescriptionInContext:(NSManagedObjectContext *)context;
{
    return [self respondsToSelector:@selector(entityInManagedObjectContext:)] ?
    [self performSelector:@selector(entityInManagedObjectContext:) withObject:context] :
    [NSEntityDescription entityForName:NSStringFromClass(self) inManagedObjectContext:context];
}
 
+ (NSArray *)findAllObjects;
{
    NSManagedObjectContext *context = [[[UIApplication sharedApplication] delegate] managedObjectContext];
    return [self findAllObjectsInContext:context];
}
 
+ (NSArray *)findAllObjectsInContext:(NSManagedObjectContext *)context;
{
    NSEntityDescription *entity = [self entityDescriptionInContext:context];
    NSFetchRequest *request = [[[NSFetchRequest alloc] init] autorelease];
    [request setEntity:entity];
    NSError *error = nil;
    NSArray *results = [context executeFetchRequest:request error:&error];
    if (error != nil)
    {
        //handle errors
    }
    return results;
}
@end

So now we have a class level find method on any NSManagedObject that will:

• create the fetch request based on the class,
• execute that request
• use the same error handling for all requests (this is handy for logging core data errors to the console)

And, in addition, we now have a place to put any custom helper methods. Since these methods are categories, they won’t interfere with any current fetching code already in your apps. These methods also have the handy side effect of being included in the type-ahead list in Xcode so they are there when you need them.

Next time, we’ll discuss how to handle other common request operations such as filtering, prefetching keys and even returning only one object.

Download Demo Project

The Trick Is In Listening To The View

Let me cut to the chase here and clue you in on what you need to do. The easiest way to take advantage of the CATiledLayer is to create a UIView based subclass and override the +layerClass class method to return a [CATiledLayer class].

+ layerClass
{
  return [CATiledLayer class];
}

Then, you just need to override drawRect: and draw what it tells you to draw. That’s it! Listen to the UIView. It’s telling you in drawRect: which rectangle it wants to draw. So while your user is scrolling a scroll view that contains your view, for example, drawRect: will be getting called continuously. You just need to calculate how that rectangle corresponds to the image you’re wanting to draw.

- (void)drawRect:(CGRect)rect
{
  // You cant get the current context if you need it.
   CGContextRef context = UIGraphicsGetCurrentContext();
 
  // Your drawing code here ...
}

I’ll show you a fuller implementation of drawRect: later. Feel free to skip there now or even download the demo project if you want to see the project in action. First I’m going to discuss how we get our tiles and truly take advantage of the memory use reduction we get from the CATiledLayer.

The Downside Is Tiling the Image

So, now that you’ve implemented a UIView derived subclass that overrides drawRect: performance is going to improve drastically, right? Well, unfortunately, not exactly. If you pull the entire image into memory with -imageNamed or even -imageWithContentsOfFile, you’re going to be up against the exact same memory problem you had before using a tiled layer. So what are the solutions? Well, if there were a way to map tiles to bytes on disk, that would be great, but unfortunately that is far more complicated and I’m not even sure it’s possible. In the end, we have to actually tile the image manually and store the images on disk to be loaded on demand.

So, the first question I asked when I realized this is can I do the tiling programmatically and write the files out to disk or do I need to slice them up in a photo editor manually? Just like everything in programming, there are tradeoffs between various solutions. If you have a different part of your workflow where it makes sense to tile the images before sending them to the device, I would choose that solution, however, that doesn’t seem terribly likely. In most cases you’re probably going to need to tile the images on device programmatically. If that’s true, then I suggest that you don’t just tile every image, but rather set a threshold size. If your images reach a certain dimension, only tile those since it will take some time and processing power to do so.

Of course, you’re going to want to do your tiling on a background thread as to not block your user interface, but here is some basic image tiling code that will write your image tiles to disk. You can place this in an NSOperation or use a block and run it on a background queue.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

- (void)saveTilesOfSize:(CGSize)size 
               forImage:(UIImage*)image 
            toDirectory:(NSString*)directoryPath 
            usingPrefix:(NSString*)prefix
{
  CGFloat cols = [image size].width / size.width;
  CGFloat rows = [image size].height / size.height;
 
  int fullColumns = floorf(cols);
  int fullRows = floorf(rows);
 
  CGFloat remainderWidth = [image size].width - 
                          (fullColumns * size.width);
  CGFloat remainderHeight = [image size].height - 
                          (fullRows * size.height);
 
 
  if (cols > fullColumns) fullColumns++;
  if (rows > fullRows) fullRows++;
 
  CGImageRef fullImage = [image CGImage];
 
  for (int y = 0; y < fullRows; ++y) {
    for (int x = 0; x < fullColumns; ++x) {
      CGSize tileSize = size;
      if (x + 1 == fullColumns && remainderWidth > 0) {
        // Last column
        tileSize.width = remainderWidth;
      }
      if (y + 1 == fullRows && remainderHeight > 0) {
        // Last row
        tileSize.height = remainderHeight;
      }
 
      CGImageRef tileImage = CGImageCreateWithImageInRect(fullImage, 
                                        (CGRect){{x*size.width, y*size.height}, 
                                          tileSize});
      NSData *imageData = UIImagePNGRepresentation([UIImage imageWithCGImage:tileImage]);
      NSString *path = [NSString stringWithFormat:@"%@/%@%d_%d.png", 
                        directoryPath, prefix, x, y];
      [imageData writeToFile:path atomically:NO];
    }
  }    
}

Let me walk you through this a little bit. We pass in the size of the tiles we want to break our image into. We pass in the image itself, the directory we want to save it to, and finally a prefix that we will use as a unique identifier for the file names for the image in question. We need this so that we can retrieve them again later when we’re ready display them.

The first thing we do is calculate the number of tiles we’re going to have in columns and rows by dividing the total width by the tile width and the total height by the tile height. The result is a floating point number. We then get the number of columns and rows that are full sized tiles by using floorf(). We then calculate the width of the last column and the height of last row. Next we iterate through the entire grid of what will soon be tiled images, columns per row. Then we extract the image data at the rect in question and write its contents to disk using an NSData. The filename format we’re using takes into account the x and y positions in our grid of tiles such that an image would have its tiles named:

<directory>/<prefix>x_y.png

Where directory and prefix are the strings passed into the function. The .png extension here is really just for clarity. It is completely unnecessary.

Implementing Draw Rect

The first implementation of CATiledLayer expected that you would create a delegate for your layer and then override

- (void)drawLayer:(CALayer*)theLayer 
            inContext:(CGContextRef)theContext

This was error prone for several reasons. The trouble was you couldn’t use both UIKit and Core Graphics calls to draw in the layer which people would often do. They would soon start asking why their app was crashing and would discover that the problem could be solved by only drawing using only Core Graphics.

Now, in iOS4, things have gotten much easier. You can simply override drawRect: in your UIView subclass and everything works correctly. To draw my layer correctly, I adapted the drawing code you find in the PhotoScroller sample code from the WWDC 2010 sessions (Link opens iTunes). It looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

- (void)drawRect:(CGRect)rect {
 	CGContextRef context = UIGraphicsGetCurrentContext();
 
  CGSize tileSize = (CGSize){256, 256};
 
  int firstCol = floorf(CGRectGetMinX(rect) / tileSize.width);
  int lastCol = floorf((CGRectGetMaxX(rect)-1) / tileSize.width);
  int firstRow = floorf(CGRectGetMinY(rect) / tileSize.height);
  int lastRow = floorf((CGRectGetMaxY(rect)-1) / tileSize.height);
 
  for (int row = firstRow; row <= lastRow; row++) {
    for (int col = firstCol; col <= lastCol; col++) {
      UIImage *tile = [self tileAtCol:col row:row];
 
      CGRect tileRect = CGRectMake(tileSize.width * col, 
                         tileSize.height * row,
                         tileSize.width, tileSize.height);
 
      tileRect = CGRectIntersection(self.bounds, tileRect);
 
      [tile drawInRect:tileRect];
 
      // Draw a white line around the tile border so 
      // we can see it
      [[UIColor whiteColor] set];
      CGContextSetLineWidth(context, 6.0);
      CGContextStrokeRect(context, tileRect);
    }
  }
}

Our view port in the app is set by the frame of the scroll view. This view is a lot larger than a single tile, which we have set to the default size of 256 x 256. That means we need to find all of the tiles that need to be drawn for displaying in the view port. So, we calculate the first column, last column, first row, and last row. This tells us where within the image we need to start and stop drawing. It also tells us which tiles we need to load. Once we’ve got all of these rows and columns calculated, we can then iterate through them and grab the tile for the current row and column using the same filename format we used to save the files in the first place. The code to load the image tiles looks like this:

1
2
3
4
5

- (UIImage*)tileAtCol:(int)col row:(int)row
{
  NSString *path = [NSString stringWithFormat:@"%@/%@%d_%d.png", tileDirectory, tileTag, col, row];
  return [UIImage imageWithContentsOfFile:path];  
}

Notice we’re using -imageWithContentsOfFile rather than -imageNamed, since -imageNamed actually caches the image in memory–which we don’t want. If we used that, we would be right back at our memory usage issue after scrolling around for a few minutes.

Conclusion

Using the CATiledLayer makes a lot of sense when memory is of the essence, which it is much of the time when doing iOS development. Examples from Apple and other places do a good job showing how you can use a tiled layer for use with PDFs, but if you want to tile an image, things are a little more complicated. I hope this post has served to help you better understand this powerful Core Animation layer. Until next time.

Download Demo Project

Notes on the Demo Project

When running on the device the image I am tiling is just too large to be tiled completely before the app uses up too much memory and is killed. You will probably want to run in the simulator to see the tiling finish. As noted in the code comments, you un-comment the tiling code and do a first run with the app allowing it to finish tiling. Then, stop the app and re-comment that tiling code and it will just load and display the tiles it created without having to go through that tiling again. I suppose I could have made the tiling code get called in response to a button tap or something like that, but I’ll leave that as an exercise for the reader. Contact me through the comments below if you have any problems or questions.

1. In your Xcode project, create a Shell Script target called GenGitVersion.
2. Insert the following script in GenGitVersion’s Run Script phase, replacing the path to your git executable if need be:

   1 2 3 4

   git=/usr/local/git/bin/git version=`$git describe` echo "#define GIT_VERSION $version" > InfoPlist.h touch Info.plist

3. Make GenGitVersion a dependency of your main target.
4. Add “InfoPlist.h” to your .gitignore file.
5. In your main target’s build settings:
1. Turn on Preprocess Info.plist File
2. Set Info.plist Preprocessor Prefix File to InfoPlist.h
6. In your app’s Info.plist set the Bundle versions string, short to GIT_VERSION

Now each time you build the main target, the version will be populated in the build’s Info.plist.

Examples

• If you tag a commit, the version returned by git describe is the tag name.
If the current commit isn’t tagged, you’ll get the most recent tag plus the number of additional commits and an abbreviated commit name. Here’s a 1.0b1 tag with 2 additional commits: 1.0b1-2-g3925f3b. This can be a good way to identify a private developer build.
• You can optionally add --dirty to git describe and the version will have -dirty appended if there are uncommitted changes to your working tree.
• git describe will fail if there are no tags in the current working tree.

The documentation on Core Data for the iPhone has lead to some confusion about how best to use Core Data on a Cocoa Touch device. One particular section seems to be the most confusing, specifically:

A view controller typically shouldn’t retrieve the context from a global object such as the application delegate. This tends to make the application architecture rigid. Neither should a view controller typically create a context for its own use. This may mean that operations performed using the controller’s context aren’t registered with other contexts, so different view controllers will have different perspectives on the data.

When you create a view controller, you pass it a context. You pass an existing context, or (in a situation where you want the new controller to manage a discrete set of edits) a new context that you create for it. It’s typically the responsibility of the application delegate to create a context to pass to the first view controller that’s displayed.

The idea behind this section is the issue of rigidity. Ideally, each view controller should be an island on its own. It should not rely on its parent, nor should it rely on the Application Delegate. Once a view controller is pushed onto the screen it should ideally be its own master.

Why Rigidity is bad

It is fairly common when designing a Cocoa Touch application to “hard code” everything. Take the following navigation controller design:

When this design, it is common to code each view controller and make it “aware” of its parent. In that design, it would be common to see view controller B call methods or call back (to its delegate) view controller A. While there is nothing technically wrong with this design, it is very rigid. It is nearly impossible to either move view controller B to another location in the stack or to reuse view controller B somewhere else. This is the trap that the documentation is trying to help new developers avoid.

Solution One

Again using a standard/normal navigation controller design, it is expected that the detail flows from left to right. The left most (or root) view controller contains the most vague information and the right most (or deepest) view controller contains the greatest detail.

In this case then the best solution is to use a UIFetchedResultsController. This controller can be considered a thin layer between the view controllers and the Core Data bits. The advantage is that the UIFetchedResultsController is designed to work with tables. The other advantage is that your least detailed view (the root most likely) can listen as the delegate of the UIFetchedResultsController for changes and update itself.

In this design, however, instead of passing around a context, you would hand off just the entity that the child view controller needs to know about. The Core Data Recipes example provided by Apple illustrates this design quite well.

How does this break rigidity? Each view controller, from the root on down, only knows what is passed into it. The root gets the UIFetchedResultsController passed into it. The child views only get the items it cares about passed into it. None of them care what their parent view controller is. There is no call back to a parent.

Solution two

What happens when we don’t have a typical navigation controller design? Perhaps a child view can pop up a modal view that displays different information. Perhaps a child view, for whatever reason needs to access information that cannot be directly passed into it every time.

In these cases there are a few different options.

View already has a NSManagedObject

Following our example above, lets say that view controller C needs to create a new object. Perhaps it is a detail view of a recipe and the user wants to add a new recipe type (perhaps she is a vegan and just discovered there is no vegan type in the list). In this case we have passed in an entity (the recipe) but not a reference to the NSManagedObjectContext. Fortunately this solution is easy to fix. The NSManagedObject retains a reference to its NSManagedObjectContext internally and we can access it. Therefore we can easily retrieve the NSManagedObjectContext from the NSManagedObject and create the new Type entity and pass it to the modal child or whatever our design calls for.

This again avoids rigidity because the view controller that represents the entity does not need to call up to a parent object or the UIApplication delegate. It is self contained and only manages view controllers that are down stream from it.

View does not have a NSManagedObject

In this situation things are slightly more complicated. In this case we want to create a @property for the NSManagedObjectContext and require that our creator set the property.

@interface MyViewController : ViewController
{
    NSManagedObjectContext *moc;
}

@property (nonatomic, retain) NSManagedObjectContext *moc;

@end

Again, the view controller is an island of its own because it does not care where that NSManagedObjectContext came from. All it knows is that it is required for the view to function. It does not care of it is a new NSManagedObjectContext specifically created for its use (perhaps for a cancelable edit tree) or if it is the same NSManagedObjectContext that has been passed around since the launch of the application. All it knows is that it has the elements it needs to perform its function.

By making the NSManagedObjectContext a settable property we can also transplant the view easily. If, at some point in the project lifecycle, we decide that it makes more sense to have the following design:

Taking from Apple’s Recipes Application, perhaps we decide that moving from the table view directly to the image of the recipe is more pleasing to the users and that when they want to see how to make it they can “flip” the image over and see the detail.

Making this change with each view controller being an island is quite simple. We just rearrange the views without having to worry too much about breaking the application.

Solution three

Up until now we have been looking at just a navigation controller design. But what about tab bars? In the situation of a tab bar we again want to avoid rigidity because it is even more common that tabs will get moved around.

The solution to this is to again use a @property for the NSManagedObjectContext and require that the creator set this property before the view is displayed on screen. If you are creating the tabs in code this is trivial because you are already calling init on the view controller and you can add one more line of code after the init to set the property.

If the user interface is being developed mostly in Interface Builder it is slightly more tricky. Personally I am not a fan of creating navigation controllers or tab bar controllers in Interface Builder. However if that is the design then I would recommend referencing the view controllers as properties and passing along the context upon initialization of the application. It may be possible to do this entirely in Interface Builder but I am not comfortable enough to recommend that as a solution.

Conclusion

The overall idea behind this article is to keep each view controller separate from anything up stream or in a different silo. This will make the design far more flexible in the long run. Any time that you feel the “need” to pass in a parent view controller to a child view controller, reconsider the design. Consider using a @protocol delegate design or NSNotification calls instead. Keep each view controller as its own private island.

As I promised in the sessions, here are the slides and sample code.

• NSFetchedResultsController
• Importing and Exporting Efficiently
• Importing Example Code

And the ZSContextWatcher is located in our open source git hub project.

Thank you to all of the attendees for having me and I look forward to seeing everyone at the next conference.