Scorebook v1.6 shipped a little over a week ago, and it finally includes all my sync code. When I cracked the project back open late last year I decided to turn off the engine that I'd put together just to get the app back in shippable condition. But I also knew that I wanted to ship the sync shortly thereafter. Well, it took about 6 weeks for me to be comfortable enough to get it out the door.

Honestly it's a little embarrassing how long it took to get this feature out. I started writing about it over 4 years ago (if you're interested in reading through the journey, they are curated here). I thought I was going to get the feature shipped by WWDC 2015. Welp. It's out there now, and that's the important thing.

Back when I was building the feature I knew I wanted it to be as seamless as possible. A modal on startup asking the user if they want to turn it on, and then a simple switch in the Setting screen. Everything else should be handle-able by my sync code. And because their data lives in iCloud I don't have to worry myself about encrypting or having any access to user data. Mission accomplished on both fronts.

As I revisited the feature when getting back into the app there were a few deprecations that had come in since iOS 8 but the code I wrote back then was still good. I did it all in Objective-C so there weren't any crazy language changes to migrate through (thank God I didn't adopt Swift 1!).

Now I get to look forward to building truly new features. I know I want to tackle dynamic type and accessibility, and might do that next. Then come the summer I want to rethink what the gameplay setup screen looks like as well as the scorecard screen. I know the whole UI could use some updating since it's very much from the iOS 7 era of design. But for most of that I'll hold off to see what Apple has in store for us at WWDC.

Check out Scorebook 1.6 in the App Store!

Last night I gave my first Xcoders talk (and first tech talk in general), going over how I’ve brought CloudKit into Scorebook. The response was overwhelmingly positive from the group and I’m really happy about that. The talk had most all of the content that I’ve written so far in this series, but also went pretty in depth on CloudKit in general and has my upload and download processes as well as lots of code samples.

The slides can be found here.

Some notes:

• I’m bummed that I haven’t shipped the Scorebook update yet, but I’m planning to in the next month or at the latest by the end of July.
• I didn’t talk about the user switching things from my last post. Mostly because I haven’t finished the implementation and I didn’t want to talk about it in that state. I’ll keep writing about it here as I make progress though.

There was video recorded as well, and once that is posted to the Xcoders Vimeo account I’ll be sure to update this post with a link to it.

Update: Here is the video on Vimeo

Lastly, I love talking about this stuff. If you have questions hit me up on Twitter or via email.

I’m feeling really confident in the things that I’ve put into sync in Scorebook. But now I’m starting to handle edge cases. Things like what happens when a user uses Scorebook without a current iCloud account, or how do I handle it when they log out and/or login again as a different user?

So I came up with a handy flowchart that I threw at Twitter and the Core Intuition Slack channel. Check it out:

I got some feedback on Slack from a guy who is doing something similar to what I am, but he’s accounting for the need to reset either the local store from the cloud or vice versa. The assumption is that at some point one or the other will become wrong and need to go back to square one. I may be overconfident here, but I’m leaning against doing that. My desired UI for syncing is a single switch on the settings screen. I think that’s how I’m going to launch, and I’ll have to see what the feedback is from users.

I’ve started writing the code to handle all of this and so far it’s not as bad as I thought it would be. I need to do a lot of testing still, though (and I’ve yet to start in on merging).

Do you see anything in the chart that might need to be changed? I’d love to hear about it!

(Thanks to David Hoang for the encouragement to post this)

I thought I was close to being done, and then I remembered that I haven’t implemented responding to deletions from CloudKit locally. Here’s how the process works:

1. Delete the object from Core Data & save the context
2. Respond to the context did save notification and send CloudKit CKRecordID objects representing the deleted record to CloudKit
3. CloudKit will send a notification to all subscribed devices letting them know of the deletion, with the corresponding CKRecordID object.
4. Figure out what entity that object belongs to, and delete the instance from the database
5. Save the updated database.

Pretty easy, right? Except, consider this: there isn’t a way to know the recordType of the CKRecordID coming down. The recordType is the database equivalent of a table, and it’s a property on CKRecord, not CKRecordID. When I’m creating CKRecords from my managed objects, I set the recordType property to a string representing my entity. That keeps things clean (i.e. my SBGame instances would all have record types of SBGame).

I haven’t talked about uniquing objects in CloudKit yet, so let’s go back to the beginning there. When you create a CKRecord you can specify it’s unique ID (made up of a recordName property which is a string, and a zone if it’s going to a custom zone). Those 2 factors combine with the CKRecord’s recordType property to create a primary key. Good database practice so far. You can also choose not to specify a recordID at the CKRecord’s creation, and it will be created when saved to the server.

For ease, I have decided to create a unique ID as a UUID string that is set on -awakeFromInsert on all of my entities. That way I always know what their unique ID will be, and I don’t have to worry about saving them back to the database when they are saved to the cloud.

The first step to determining what record type the CKRecordID I’m responsible for deleting, I made a change to each entity’s -recordName property, so that it includes the entity name as the first part of the record name. Then it’s followed by a delimiter (I’m using “|~|”) and then the UUID. Here’s how that now looks:

- (void)awakeFromInsert
{
    [super awakeFromInsert];
    self.ckRecordName = [NSString stringWithFormat:@"SBGame|~|%@", [[NSUUID UUID] UUIDString]];
}

A sample recordName for an SBGame instance would be “SBGame||386c1919-5f25-4be2-975f-5b34506c51db”, with || being the delimiter between the entity name and the UUID. A bit hacky, but it works.

On the downloader I explode the string into an array using he same delimiter and grab the first object, which will be “SBGame”. Once I have the entity and the recordName to search for, I can put this in a fetch request to grab the object that needs deletion. Here’s what that looks like:

NSString *recordName = recordID.recordName;
NSString *recordType = [[recordName componentsSeparatedByString:@"|~|"] firstObject];

NSPredicate *predicate = [NSPredicate predicateWithFormat:@"ckRecordName = %@", recordName];
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:recordType];
fetchRequest.predicate = predicate;

NSArray *foundObjects = [backgroundContext executeFetchRequest:fetchRequest error:nil];
id foundObject = [foundObjects firstObject];
if (foundObject != nil) {
    [backgroundContext deleteObject:foundObject];
}

In my limited initial testing this is working well so far, and it’s a simple solution to the problem.

In my usage of CloudKit I had to determine early on how I was going to upload batches of records and deal with batches coming down. I’m dealing entirely with CKRecord instances, so all of my entities will need to know how to handle those; both in how to create one from themselves, and how to turn turn one into itself.

I didn’t have any clever idea at first, so I punted. For saving records to CloudKit, I’m monitoring the NSManagedObjectContextDidSaveNotification (which isn’t verbose enough) and I get NSSets of managed objects. Here’s what my initial implementation looked like:

- (CKRecord *)ckRecordForManagedObject:(NSManagedObject *)managedObject
{
    CKRecordZoneID *userZone = [[SBSyncController sharedSyncController] userRecordZone];
    CKRecord *recordToReturn;

    if ([managedObject isKindOfClass:[SBPerson class]]) {
        recordToReturn = [(SBPerson *)managedObject cloudKitRecordInRecordZone:userZone];
    } else if ([managedObject isKindOfClass:[SBGame class]]) {
        recordToReturn = [(SBGame *)managedObject cloudKitRecordInRecordZone:userZone];
    } else if ([managedObject isKindOfClass:[SBMatch class]]) {
        recordToReturn = [(SBMatch *)managedObject cloudKitRecordInRecordZone:userZone];
    } else if ([managedObject isKindOfClass:[SBPlayer class]]) {
        recordToReturn = [(SBPlayer *)managedObject cloudKitRecordInRecordZone:userZone];
    } else if ([managedObject isKindOfClass:[SBScore class]]) {
        recordToReturn = [(SBScore *)managedObject cloudKitRecordInRecordZone:userZone];
    } else if ([managedObject isKindOfClass:[SBMatchImage class]]) {
        recordToReturn = [(SBMatchImage *)managedObject cloudKitRecordInRecordZone:userZone];
    } else if ([managedObject isKindOfClass:[SBMatchLocation class]]) {
        recordToReturn = [(SBMatchLocation *)managedObject cloudKitRecordInRecordZone:userZone];
    }

    return recordToReturn;
}

Yuck.

And similarly, I captured the -recordType property of an incoming CKRecord, then ran -isEqualToString: against it to determine which entity it represented. And then something interesting happened.

I got to the place where I was working through how to handle conflicts and needed a -modificationTimestamp property to get that done, and realized that I could use a protocol to declare uniform conformance across all of my entities. I could make sure that the class conforms to the protocol, and then set the property without having to do the ugliness of something like the above snippet.

And thus, SBCloudKitCompatible was born. This protocol defines 4 things in the interface:

@protocol SBCloudKitCompatible <NSObject>
@property (nonatomic, strong) NSString *ckRecordName;
@property (nonatomic, strong) NSDate *modificationDate;

- (CKRecord *)cloudKitRecordInRecordZone:(CKRecordZoneID *)zone;
+ (NSManagedObject *)managedObjectFromRecord:(CKRecord *)record context:(NSManagedObjectContext *)context;
@end

By conforming to this protocol, I’ve been able to cut out a bunch of code. Here’s what the snippet above now looks like:

- (CKRecord *)ckRecordForManagedObject:(NSManagedObject *)managedObject
{
    CKRecord *recordToReturn = nil;

    if ([managedObject conformsToProtocol:@protocol(SBCloudKitCompatible)]) {
        id<SBCloudKitCompatible> object = (id<SBCloudKitCompatible>)managedObject;
        recordToReturn = [object cloudKitRecordInRecordZone:self.zoneID];
    }

    return recordToReturn;
}

I think we can agree that it looks way, way better.

When I decided to begin sync implementation with CloudKit, I just started going. I watched the WWDC videos (mostly) and read through the getting started, then started coding.

That wasn't the best tactic. Turns out.

There are a couple of questions that I should have asked myself. Do I want to have record changes pushed to me automatically? Do I want atomic operations so that all of the items have to succeed?

(As a slight aside, take note that neither of those things can be done on the public database, this is only for private data)

Well, I do want at least the record changes. So I need to implement a custom record zone. This is done with a CKRecordZone object, and is kind of like the equivalent of a database schema. Instead of using the default zone (which can't have push notifications or atomic actions), I create my own like this:

CKRecordZoneID *zoneID = [[CKRecordZoneID alloc] initWithZoneName:@"ScorebookData" ownerName:CKOwnerDefaultName];
CKRecordZone *zone = [[CKRecordZone alloc] initWithZoneID:zoneID];

The zone name can be "ASCII characters and must not exceed 255 characters". The zone name needs to be unique within the user's private database (which makes using a single string as the zone name possible). The owner name can just have the string constant CKOwnerDefaultName, which I'm pretty sure gets changed server-side to be some sort of user identifier. The docs aren't super clear about that part, but the method notes say:

To specify the current user, use the CKOwnerDefaultName constant.

Poking inside that constant at runtime expands it to __defaultOwner__. CloudKit seems to know what to do, though.

The next part I had to figure out was how do I get records into this zone? You don't save anything to the zone object, but instead use a different initializer on the CKRecordID objects. So I added a paramteter to my -cloudKitRecord methods in my model categories to take a zone. Now the methods are - (CKRecord *)cloudKitRecordInRecordZone:(CKRecordZoneID *)zone, and the implememtation uses the -initWithRecordName:zoneID initializer to create the record ID.

The thing I'm going to talk about next time is setting this whole stack up. I'm coming to realize that CloudKit, like Core Data, is a stack. I'm building a sync controller (which is what injects the CKRecordZoneID into the model objects' zone parameter) and that is working well so far. Stay tuned.

I want to finish out my schema bit with a little bit on relationships. CloudKit fully supports relationships so you can associate your data with one record type to another. However it’s not exactly like you would expect in a relational database. CloudKit supports 1:1 and 1:many relationships, but not many:many. That’s not a problem for me but it may be for some.

In designing these relationships, the big thing to know going in is that you create the reference from the foreign-key side, not the primary side. So if you have a product that is becoming a line item, where the left side might be the product with many line items, you create the relationship on the right side with the line item. See the hand-crafted image below for clarity.

Creating a relationship is straightforward, and you use a CKReference object to do so. The designated initializer is -initWithRecordID:action: and you can send in the record ID object of the record to reference. If you don’t have the record ID, you can send in the whole record to the -initWithRecord:action: method. I don’t know which is better, and if you don’t have a record ID then that’s the one to use. I was going to flip back and forth, but since I decided to make the record ID at creation time I stick with the initWithRecordID:action: method instead.

The action part of the initializer refers to delete rules. Do you want the child to be deleted when the parent goes away? If so you can pass in CKReferenceActionDeleteSelf, and if not then you’ll want to use CKReferenceActionNone. It’s pretty straightforward.

From there in my CloudKit record creation method, the reference becomes just another key on the CKRecord object. So it gets a key, with the reference as a value, and that’s it. It looks something like this:

CKRecord *record = [[CKRecord alloc] initWithRecordType:@“Player”];
CKRecordID *personID = [[CKRecordID alloc] initWithRecordName:self.person.ckRecordName;
CKReference *personReference = [[CKReference alloc] initWithRecordID:personID action:CKReferenceActionDeleteSelf];
record[@“person”] = personReference;

Save the record to a database and that’s all there is to it.

Building a schema on CloudKit is pretty straightforward. You get a CKRecord object and treat it like a dictionary. There are a few data types they can take (NSString, NSNumber, NSData to name a few) and it’s a key/value pair to get the data on the record. Once you have a record created, you save it to the database and your RecordType (i.e. a database table) is created. Simple. But there is a trick associated with related data.

One of the data types you can attach is a CKReference. This is how you glue 2 records together. CloudKit supports 1:Many relationships (not many:many though). The way you do the relating is through the child record, not the parent. This tripped me up a little bit because with my background in FileMaker I would typically go parent to child (like with a portal object on a layout).

Let me back up for a moment. I made a design decision early on to isolate my networking files as much as possible. It could be a hedge or just wanting to have all the syncing code in one place, it jut felt right. I put a new folder in my Model folder called Sync and in it I have categories on all my model objects called <objectName>+CloudKit.h/m. I want each entity to know how to turn itself into a CKRecord and how to turn a CKRecord into itself. That feels like the right way to go.

Inside each category I have a method called -cloudKitRecord that returns a CKRecord. Here’s a basic implementation:

- (CKRecord *)cloudKitRecord
{
    CKRecordID *recordID = [[CKRecordID alloc] initWithRecordName:self.ckRecordId];
    CKRecord *personRecord = [[CKRecord alloc] initWithRecordType:[SBPerson entityName]
                                                         recordID:recordID];

    personRecord[SBPersonFirstNameKEY] = self.firstName;
    personRecord[SBPersonLastNameKEY] = self.lastName;
    personRecord[SBPersonEmailAddressKEY] = self.emailAddress;

    if (self.imageURL) {
        CKAsset *imageAsset = [[CKAsset alloc] initWithFileURL:[self urlForImage]];
        personRecord[SBPersonAvatarKEY] = imageAsset;
    }

    return personRecord;
}

A few things to note here:

• I’m overriding -awakeFromInsert on all of my managed objects to create a -ckRecordId string property (Just a random UUID) that I can use as my CloudKit record ID. This is the unique identifier across the system. I started out with something much more complicated but landed here for now.
• I have used Accessorizer (MAS link) to create string constants for all of my managed object properties. See this talk from Paul Gorake for more Core Data amazingness.
• You can see my new CKAsset code at work here. I’m creating a new image asset with the URL to the image on disk, then you just use the asset as a property on the CKRecord. CloudKit handles the rest, which is really nice.

On the flipside, I have methods in the category that will take a CKRecord and create the Core Data managed object out of it. I haven’t gotten to the downstream sync just yet (I just ran into the need for CKRecordZones that I’ll probably write about next). Once I start the downstream I’ll put up a post about it too.

I’m going to come back to the CKReference stuff in my next post, since this one got a bit too long. Stay tuned.

After thinking through my options last time I landed on option 2. Instead of storing the binary data within Core Data, I'm putting it in the file system instead. Here's how it's working:

• The user picks an image that they want to save
• The object associating with the image determines its filename and is responsible for all saving and retrieving of file data

I put the saving methods in a category on the 3 entities that save images. I couldn't stomach the idea of importing UIKit into actual model classes. This is more of a style thing than not, so it could have gone either way.

When I was originally putting this together I stored the absolute filepath to each file. This would work until I relaunched and then it broke. Turns out the simulator's documents path isn't always guaranteed to be the same file path every time. The workaround is to store the file's path relative to my documents directory. When I need to retrieve the image, the object responsible grabs the current document directory and smashes it up with the relative file path.

I'm not super happy that this is something I had to do in the first place, but I am satisfied with how it turned out. Now I get to move on to the grittier details of the syncing process.

I'm working on the CloudKit schema for Scorebook and am coming upon a question right away. Scorebook allows users to add pictures as avatars, or game thumbnails, and to attach to game plays. I'm storing these in Core Data as binary data. All cool on the device. I'm also using the option to allow external storage in the managed object model.

CloudKit offers to store these files as CKAssets attached to my CKRecord. The problem is the init method for a CKAsset is initWithFileURL:, and initWithData: is not an option. So, how do I get to the files in the database?

My first inclination is to export the NSData as a file and return its resulting file:/// URL. I don't like this because of the additional storage that is necessary. I also don't like this because of possible overhead in processing. This method could drain users' batteries much faster. I'll also have to account for cleanup of the files once the sync is complete.

The other option, that could be something of a rabbit hole, is to rewrite all of those binary data fields to point to a location on disk for the files. This is starting to sound like it might be the winner. The tricky part I think will be migrating all the current image assets out of the database and to their own files on disk, then initiating the first sync.

Which sounds better to you? Is there something I'm missing?

Update I just filed rdar://19915193 asking for an initializer to work with NSData objects for CKAssets.