Apple Networking Feedback

Recently Quinn, an engineer on the Developer Technical Support team at Apple, posted a request for feedback on Apple’s networking APIs. Here are his questions and my answers:

  1. If your product is still using the deprecated NSURLConnection or NSURLDownload APIs, is that because:
    • (A) You just haven’t got around to adopting NSURLSession
    • (B) NSURLSession is missing some feature that you need
    • (C) Other

    For B and C, please provide some details.

    Answer: There are times I’ve longed for the NSURLConnection API. In many respects, I find its execution model to be simpler, where a single connection has a single corresponding delegate. That’s really simple to follow and is the main thing I miss about the NSURLConnection API. However, I do not miss its reliance on a runloop; I’ve had situations where I’ve had to manually spin up a background thread and pump its runloop in order to keep NSURLConnection off the main thread.

  2. If your product is still using the deprecated CFHTTPStream API, is that because:

    • (A) You just haven’t got around to adopting NSURLSession
    • (B) NSURLSession is missing some feature that you need
    • (C) Other

    For B and C, please provide some details.

    Answer: Not applicable.

  3. If your product is still using the FTP protocol, is that because:

    • (A) It’s a legacy feature you haven’t got around to removing
    • (B) Your product interacts with hardware that only supports FTP
    • (C) FTP is an absolute requirement of the folks who pay the bills
    • (D) Other

    For B, C and D, please provide some details.

    Answer: Not applicable. Was it ever applicable to anyone?

  4. If your product is still using the FTP protocol, do you use:

    • (A) NSURLSession
    • (B) NSURLConnection / NSURLDownload
    • (C) CFFTPStream directly
    • (D) A third-party library that uses CFFTPStream
    • (E) A third-party library with its own core FTP code
    • (F) Other

    For D, E or F, please provide some details.

    Answer: Not applicable.

  5. If your product uses a third-party library layered on top of NSURLSession, did you choose that library because:

    • (A) It provides specific functionality you need
    • (B) It has a nicer programming interface
    • (C) Other

    In all cases, please let us know which library you’re using. In case C, please provide some details.

    Answer: (A) and (B).

    Taking AlamoFire as an example, it meets (A) because NSURLSession and friends don’t provide anything in the way of multi-part encoding. You occasionally come across the need for building up a form submission, and having to manually do that all yourself is extremely annoying, especially if that form submission involves streaming files off disk. AlamoFire’s form encoding isn’t perfect (since you can’t really compose NSInputStreams, it essentially just re-builds the entire form submission as a file on disk and then provides a single stream to that), but it’s infinitely better than a non-existent API from NSURLSession and having to roll your own encoder.

    If also meets (B) because AlamoFire generally has a shorter API than NSURLSession provides. And, as I’ll mention at the end, it uses Result<T>, which is amazing.

  6. If your product uses a third-party HTTP[S] library that is not layered on top of NSURLSession (one that uses its own core HTTP code) is that because:

    • (A) It helps with cross-platform compatibility
    • (B) It’s intrinsic to your development environment
    • (C) NSURLSession is missing some feature that you need
    • (D) The third-party library performs better than NSURLSession
    • (E) The third-party library is more reliable than NSURLSession
    • (F) Other

    For C, D, E and F, please provide some details.

    Answer: In the past I’ve used non-NSURLSession HTTP libraries mainly because of (A); when working for a company that offers their product on multiple platforms, it’s pretty common to have a bunch of the core product logic built in cross-platform frameworks (mainly C++).

    Otherwise, I rarely come across a situation where I need to “re-invent” the networking wheel.

  7. If your product uses the HTTP[S] protocol but does not use HTTP/2, is that because:

    • (A) You use HTTP, not HTTPS
    • (B) Your product talks to a variety of HTTP servers, some of which support HTTP/2 and some of which don’t
    • (C) Your product talks to a specific HTTPS server that’s outside of your control
    • (D) Your product talks to a specific HTTPS server that you control but you haven’t yet updated to use HTTP/2
    • (E) Other

    For E, please provide some details.

    Answer: (E): I’ve never had to care about HTTP 1.1 vs HTTP 2

  8. For NSURLSession [*], what are the enhancement requests or bugs that you’d most like to see addressed? Please list up to five in order from most to least desired. Ideally we’d like a list of bug numbers, with all the details in the corresponding bug report.

    [*] This includes the core NSURLSession API and all its related technologies, including:

    • Async API structure (delegates, blocked-based convenience APIs, and so on)
    • Swift integration
    • Requests and responses
    • Caching
    • Cookies
    • Authentication (including credentials and their storage)
    • Background sessions
    • Core HTTP and HTTP/2 protocol implementations
    • Debugging tools
    • Security (HTTPS)
    • Performance
    • Task metrics
    • Task prioritisation
    • Stream tasks

    Answer: I generally don’t have many complaints about the networking stack. Mainly I just want it to be simpler, but if I had to pick some…

    1. Swift integration (mainly better type-safety)
    2. Debugging tools (see answer to #16)
  9. If your app uses the reachability API (SCNetworkReachability, either directly or via a wrapper like the Reachability sample code), is that because:
    • (A) Someone said you needed reachability and you’re not entirely sure why
    • (B) You just haven’t got around to adopting the new waitsForConnectivity support in NSURLSession
    • (C) You want to preflight network requests, that is, you don’t make a request until reachability indicates that it might work
    • (D) You want to know when it’s a good time to retry a failed request
    • (E) You’re using SCNetworkReachabilityCreateWithAddressPair to monitor the state of a specific connection
    • (F) You want to know what type of networking is available, for example, WWAN or Wi-Fi
    • (G) You’re using it solely to update your app’s user interface
    • (H) Other

    For C, we’d really like some details on why you’re preflighting network requests.

    For F, G and H, please provide some general details.

    Answer: (B), (D), (F), and (G).

    I like control. By manually using SCNetworkReachability, I get the control to manually retry requests, update my app user-interface, proactively disable app functionality (why should I keep a sync engine turned on if I “know” the network is off?), and tune certain kinds of requests to the networking interfaces I feel are more appropriate for the app’s functionality (ex: when on WWAN, restrict communication to app-critical things; save WiFi for proactively downloading caches, etc).

    However, the reachability API is archaic. It desperately needs some Swifty love.

  10. If your product works directly on top of the TCP protocol, what API do you use:

    • (A) BSD Sockets
    • (B) CFSocket
    • (C) NSStream
    • (D) NSURLSessionTask
    • (E) NWTCPConnection
    • (F) A third-party library
    • (G) Other

    For F, please let us know which library you’re using. For G, please provide some details.

    Answer: Not applicable.

  11. If you answered the previous question, please provide information about your TLS (aka SSL) use:

    • (A) You don’t need TLS
    • (B) You use the TLS implementation provided by the API you’re using (for example, in NSURLSessionTask you can enable TLS by calling -startSecureConnection)
    • (C) You use Secure Transport to implement TLS
    • (D) You use OpenSSL to implement TLS
    • (E) You use some other third-party TLS library

    For D, please explain why you use OpenSSL. For E, please provide some details.

    Answer: Not applicable, although there are totally valid reasons for bundling OpenSSL in your app because the iOS Security APIs don’t provide enough functionality.

  12. If your product uses BSD Sockets for networking, is that because:

    • (A) It helps with cross-platform compatibility
    • (B) You have existing code that you don’t want to rewrite
    • (C) You’re using UDP, not TCP
    • (D) You’re using one of the more obscure features of BSD Sockets (raw IP, routing sockets, and so on) for which there is no equivalent higher-level API
    • (E) The equivalent higher-level API would otherwise be appropriate but is missing some small feature that you need
    • (F) Your code performs better than the equivalent high-level API
    • (G) Your code is more reliable than the equivalent high-level API
    • (H) Other

    For D through H, please provide some details.

    Answer: Not applicable

  13. Are you interested in adopting the QUIC protocol?

    Answer: I don’t know what QUIC is.

  14. Are there any other existing or future networking protocols that would be useful to your product? Remember that the focus here is on commonly-used user-space networking APIs, so please restrict yourself to protocols that make sense in that context.

    Answer: Web sockets. If you’re connecting to a realtime web service, it’s pretty common to have that implemented via websockets. In the past I’ve poked around at SocketRocket and Starscream. It’d be very nice if that were built-in.

  15. When debugging problems with the networking features of your product, which tools do you typically use?

    • (A) Wireshark
    • (B) Some other packet trace tool
    • (C) A debugging HTTP proxy
    • (D) CFNetwork diagnostic logging
    • (E) sysdiagnose logs
    • (F) Other

    For B, C and F, please provide details.


    (C): Charles Proxy

    (D): I put this in here because I explicitly don’t use it. There’s no obvious way to configure the verbosity of its logging, and it seems like every other release of iOS plagues your console with random spew from the networking layers that is difficult to disable. It’d be nice if this were useful in any degree, but honestly right now it’s an utter waste of console space and just obfuscates my logs.

    (F): Paw ← this is like the best thing ever.

  16. In the context of debugging problems with the networking features of your product, is there a specific tool that you’d like Apple to provide?

    Answer: I want an Instruments template that does HTTP inspection. There are templates for doing the byte-level traffic analysis, but usually I don’t care that much about it. I want a tool that shows me the actual HTTP requests and responses, like what I get in Charles. I want to see the hosts I’m connecting to, the raw HTTP request, various views of the bodies (raw, formatted, interpreted as JSON, interpreted as form encoding, etc), the headers, the latency, etc. And I want to see all this information for the response.

    Ideally, there would be a debug gauge in Xcode that shows a high-level view of this information, and then if I needed deeper analysis I would pop open Instruments to get it.

Other Thoughts: (These don’t particularly fit in to the questions above, so I’m putting them here at the bottom)

There are a couple things that I find annoying about working with NSURLSession in Swift:

  1. It’s a URL session, and not an HTTP session. This means I have to do lots of silly as? casting to make sure things are HTTPURLResponses before proceeding. I would really love to see a specialization of the API that is focused on only the HTTP use-case. I don’t think I’ve ever really needed anything except an HTTP session, and having to dig around the API to figure out what’s HTTP-specific and not and do the type-safe guarding is annoying.

  2. The Swift API desperately needs a Result<T> type. Getting a callback that is a (URLResponse?, Data?, Error?) is supremely annoying to deal with. I’d much rather have a Result<(HTTPURLResponse, Data?)>. That would be way easier to work with.

  3. It would be nice (but not required) to have some sort of protocol-defined serialization/deserialization support for bodies in requests and responses. If I know that I’m going to get back XML or JSON or an Image, it’d be great if I could have a generic way to simply say “execute this request and give me back the image that comes back in the body”.

Sometimes I hate being a programmer

There are two things that make me hate being a programmer.

Too many ideas, not enough time

I hate being a programmer when I have an idea for an app (or any sort of code) but no time to write it. I’ve written before about my dream app, but it kills me that I don’t have enough time to write it. There’s a particular version of solitaire that I really love playing that is devilishly difficult and really interesting, but the app is totally buggy and it drives me nuts. I could rewrite it, because the logic of Solitaire isn’t that difficult, but I don’t have enough time. I want to write a Mac app for browsing and posting on the Swift forums. I want to write a better version of the system Keychain Access app. I think the Wallet app on iOS desperately needs me to rewrite it. I have about half a dozen ideas for apps around tooling and supplementing the developer experience.

And I don’t have time to write them.

This kills me. It makes me hate being a programmer because I feel like I never have time to write the things I want to write.

I have to write this again?

I hate being a programmer when I’m faced with the prospect of writing some code that just isn’t interesting to write. Or, maybe it’s interesting, but I also am aware of how huge the problem space is. My classic example of this is local persistence. The sheer drudgery of writing a persistence layer for your data model is positively soul-sucking. Throw in syncing over a network and offline mode and you’re staring down the barrel of implementing vector clocks and CRDTs and crazy logic to resume stuff on app launch and why am I in this profession again?


But then.

Sometimes you write some code that just takes your breath away. You write something so profoundly elegant that you wonder how you could ever write any code better than this ever again. And then it’s worth it.

Swift Protocols Wishlist

If I were Supreme Swift Potentate, there are a few things I’d change about how Swift deals with protocols, and how this gets manifest in the standard library.

In no particular order, here they are:

  1. If you’re adopting a protocol (especially one from the standard library), the tools don’t provide any help whatsoever in knowing what you actually have to implement. Because of this, it’s pretty much flat-out impossible¹ to make your own value-typed data structure (like, say, an AVL tree or whatever) that conforms to all of the expected data structure-y protocols.

  2. I wish protocols were generic instead of having associated types. For example, I wish I could declare:

    func processTheseIntegers(_ integers: Collection<Int>) { ... }

    Instead of having to make the whole thing generic:

    func processTheseIntegers<C: Collection>(_ integers: C) where C.Element == Int { ... }
  3. We don’t have generic protocols, because instead we have these things called Protocols with Associated Types (“PATs”). PATs are things that look great at first blush. Just take a look at how succinct this is!
    protocol Requestable {
        associatedtype ResponseType: InitializableFromData

    It’s so easy declare that “I have this request, and it has a related response type, and this response value has to conform to this protocol. Awesome!


    The rules around arrays and stuff in Swift mean you can’t create an array of anything that happens to be Requestable:

     var currentRequests: Array<Requestable> // not allowed

    In order to do something like this, you have to create type erasers or propagate the associated type of the protocol out as a generic. It’s maddening and it sucks the life out of you. Basically any time you find yourself using a PAT, you know that you’re either going to have to create a type eraser, or you’re going to have to make things generic that have no business being generic. They’re these parasitic things that end up infesting your code in ways you really don’t want.

  4. I wish the compiler generated type erasers for me. In the case of the Requestable stuff above, the common way to work around the problem of heterogenous arrays is to make them homogenous using type erasers. A type eraser is a value that hides some of this stuff and “homogenizes” the types to the type system. For examples of this, see pretty much anything in the standard library whose name starts with Any, like AnyHashable. Writing type erasers manually is an exercise in pain. You have to create a couple of layers of indirection and do some weird things that I can never remember off the top of my head and always have to google. It’s grunt work, and the compiler should do it for me.

  5. I wish there were a way to say “this protocol can only be adopted by value types”. We have the opposite, to say “this protocol can only be adopted by reference types”, which is great for defining weak var delegate: MyDelegateProtocol?.

    But there are times when you want a protocol-typed value, but don’t want to allow mutability. Normally in Swift you define immutability with let and var. However, you can’t do that with protocols. With a protocol, everything is a var, because it can be adopted by a reference type, and reference types have no restrictions on internal mutability. This is inherently unsafe, if you allow for things to change out from underneath you. This is why Array and String and everything became value types in Swift, because their internal-mutability creates for some nasty bugs in Objective-C, where they’re reference types.²

  6. This one is pretty niche, but occasionally you deal with a lot of single-requirement protocols, like this:

    protocol YearHaving { var year: Int { get } }
    protocol MonthHaving { var month: Int { get } }
    protocol DayHaving { var day: Int { get } }

    It’d be nice if there were a shorthand way to define a struct that just has all of those fields. Maybe something like this:

    auto struct YearMonthDay: YearHaving, MonthHaving, DayHaving { }

    That way you wouldn’t have to re-declare all of the properties you’re supposed to have, and they’d all come in as let properties (because they only have { get }), and then you get the default compiler-provided initializer of .init(year: Int, month: Int, day: Int). Yeah this is pretty specific, but when you need it, you need it.

  7. These last ones are all related, and they’re all really big. I wish I could provide default implementations of protocol methods that assign to self. For example:

    protocol URLRequestable {
        var url: URL { get }

    There are a bunch of different kinds of URLs. It’d be nice if I could do:

    extension URLRequestable {
        init(url: URL) {
            switch url.scheme {
                case "mailto": self = EmailRequest(url: url)
                case "ftp": self = FTPRequest(url: url)
                case "http": self = HTTPRequest(url: url)
                default: self = URLRequest(url: url)

    This is related to a concept in Objective-C called “class clusters“. A class cluster is basically an interface for a type, with a bunch of varying implementations that depend on certain conditions. To me, defining a class cluster’s interface via a protocol feels far more “swifty” than using an abstract superclass (like Objective-C), but there’s also no way to make it truly swifty and use an initializer to create your cluster instance; you have to use a static factory method if you want to do that.

  8. I wish I could extend a protocol to conform to another protocol. For example:

    protocol URLRequestable {
        var url: URL { get }

    If I know that all URLRequestable things have a protocol, then I should be able to do:

    extension URLRequestable: Hashable {
        static func ==(lhs: URLRequestable, rhs: URLRequestable) -> Bool { return lhs.url == rhs.url }
        var hashValue: Int { return url.hashValue }

    That would be super useful, because I could get incoming URLRequestable values, and then toss them in a Set to help ensure I don’t execute duplicate requests. Sadly, you cannot do this. I don’t know why. (Usually when I ask, I get told something intensely inscrutable that includes the word “existential” in it)

  9. Adding these all up leads me to my last wish: I wish that the API defined by the standard library was actually all just protocols. I wish String and Array and Dictionary and Set were protocols. If you had protocol clusters, you could still construct things like you do now, but you’d just get back private implementations of protocol String or whatever. This would mean that, when I ask a Dictionary<String, Foo> for its .keys, I could just get back a Set<String> and not a Dictionary<String, Foo>.Keys. Similarly, when I ask for a .lazy.something, I don’t have to get back some weird LazyRandomAccessCollection or whatever; I could just get an Array<Something> that happens to have a lazy implementation.

I look at these things on this list and think “wow, Swift would be awesome if it had all of this”. I really really wish it did; especially the last one. And maybe someday it’ll have a couple of these, but I get depressed when I think that it’s probably highly unlikely it’ll ever have all of them.

¹ – yeah, I know it’s not impossible. But doing it “right” requires way too much work to even figure out what “right” means.

² – yeah, I know you can make a struct that is internally mutable by having it wrap a reference type.

ⁿ – I posted about this on Twitter here.

The 2018c Timezone Database Update

I subscribe to a couple of mildly interesting and low-volume email lists. One such list is the tz-announce list hosted by ICANN. It’s the list for people who care way too much about timezones.

The other day I got an email that the 2018c release of the database was available. There are a couple of interesting changes in it, which I thought would be fun to look at and consider the implications.

  1. São Tomé and Príncipe switched from +00 to +01 on 2018-01-01 at 01:00.

    So, in this range of hours, an hour will be missing in the middle of it.

  2. Starting in 2018 southern Brazil will begin DST on November’s first Sunday instead of October’s third Sunday.

    Oh man. Daylight Saving Time. I really don’t get why this is still a thing.

  3. Japanese DST transitions (1948-1951) were Sundays at 00:00, not Saturdays or Sundays at 02:00.

    These first three changes are to the past and future, which brings up some really interesting stuff when it comes to storing timestamps. Any time you change the past (or the future), you deal with the possibility of invalidating already-saved information.

    Let’s consider the Brazil change as an example. Brazil is notorious among chronologists for performing its Daylight Saving Time jumps at midnight, which means that on the “spring forward” jumps, midnight doesn’t exist. And, like all DST jumps, the leap back produces an hour that occurs twice.

    In this case, Brazil will be “springing forward” (because even though it’s November, that’s Spring in southern Brazil). Therefore, midnight isn’t going to exist.

    Now, what happens if you run a scheduling service and you had scheduled something to happen at midnight on November 4th, 2018? Suddenly, that time doesn’t exist anymore. When you scheduled that information, “November 4th 2018 @ 00:00:00 AM” was a totally legitimate date. Now it’s not.

    Similarly, if I happened to be storing a date relating to a midnight in Japan after the end of World War 2, there’s a chance that that date is now invalid.

    Time zones are already crazy complicated. Changing past and future time zone transitions just compounds the effect.

  4. A discrepancy of 4s in timestamps before 1931 in South Sudan has been corrected. The ‘backzone’ and ‘’ files did not agree with the ‘africa’ and ‘’ files.

    I guess a bug was fixed in the file? And it was 4 seconds off? Ack.

  5. The abbreviation invented for Bolivia Summer Time (1931-2) is now BST instead of BOST, to be more consistent with the convention used for Latvian Summer Time (1918-9) and for British Summer Time.

    I hope you don’t store dates using the abbreviated timezone name. If you do, and you happened to be storing a date in Bolivia Summer Time from between 1931 and 1932 (all 3 of you), then this just broke.

So there you have it. Not very many changes, but definitely things of consequence. If you’re interested, I recommend subscribing to tz-announce. There’s only a few emails a year and it’s always interesting to see political and historical actions affecting technology.

Simplifying Swift framework development

I’ve developed a handy trick when writing frameworks in Swift that makes the overall process a little bit nicer, and it’s just adding a single file to your framework.

Let’s say you’re building CoreAwesome.framework for inclusion in your app, or publishing to Github, or whatever. There are a couple things you end up doing a lot:

First, you end up having lots of import OtherFramework statements scattered throughout your .swift files, so that portions of your framework can have access to pieces of functionality provided by dependencies (whether system frameworks or whatever). I find that sort of repetition (import Foundation, anyone?) to be pretty annoying.

Second, you don’t always have a good way of getting access to the Bundle that corresponds to the framework easily. You need this particularly if you’re loading bundle-specified resources.

So, based on these two things, I’ve developed this pattern, which is kind of like a swift framework pre-compiled header:

When I create CoreAwesome.framework, I get CoreAwesome.h, and that’s about it. So I immediately add CoreAwesome.swift at the top level next to the .h, and put this in it:

// CoreAwesome.swift
@_exported import Foundation
@_exported import DependencyA
@_exported import DependencyB

public let CoreAwesome = Bundle(for: CoreAwesomeMarker.self)

private class CoreAwesomeMarker { }

First, there’s this weird @_exported thing. The underscore indicates we need to be a bit wary of it, because it’s not a modifier you’re really supposed to use. But if you do…

@_exported will make an import-ed module visible to the entire module into which its been imported. This means you don’t have to import Dependency in every file you need it. You just @_exported that dependency once, and you’re good to go in any file in that module.

This is especially nice if DependencyA defines public operators, which aren’t always imported the same way that symbols are. That’s a topic for another day.

Second, I define a public constant that is the name of the framework, and whose value is the Bundle for that framework. I use the class-based look up (ie, find the bundle that declares this class), because it’s one of the few convenient Bundle initializers that doesn’t return a Bundle?, and thus I don’t have to deal with unwrapping. And then I use a special marker class for making that lookup resilient in the face of other functionality changes.

With the constant in-hand, I can easily load resources:

let resource = CoreAwesome.url(forResource: "Foo", withExtension: "plist")

This reads pretty naturally, and the entire file makes developing my framework just a little bit easier.

Update: I neglected to mention that it was Joe Fabisevich who clued me in to the @_exported trick. Thanks, Joe!!

Update #2: Both Harlan Haskins and Kevin Ballard pointed out that the constant to access the framework’s bundle will conflict with a module-qualified declaration. Like, if you have two modules that declare a Foo, then you need to disambiguate which one you want by doing Module1.Foo vs Module2.Foo. However, if Module1 is the name for a Bundle instance, this breaks.

Solving this problem is left as an exercise for the reader. 😉

Reading your own entitlements

When you’re writing an iOS or macOS app, you typically don’t need to dynamically know what your own entitlements are. However, there are a couple of rare circumstances when it could be Nice To Have.

I recently came across one of those situations. Like most developers, I have a set of core libraries I maintain that I use in my apps. These libraries tend to contain all of the common pieces of code I’ve found to be helpful. It’s like my own private SDK.

For example, I have a Sandbox type that represents a set of on-disk locations:

public class Sandbox {

    public static let currentProcess: Sandbox

    public let documents: AbsolutePath
    public let caches: AbsolutePath
    public let support: AbsolutePath
    public let temporary: AbsolutePath

    public init(documents: AbsolutePath, caches: AbsolutePath, support: AbsolutePath, defaults: UserDefaults)
    public convenience init?(groupIdentifier: String)

I got thinking that it’d be nice to add a static default property on Sandbox that would contain the “default” Sandbox. In a situation where my app doesn’t use a shared group container, the default sandbox would be the current process’s sandbox. If I do have one (or more) shared group containers, then the default sandbox would be the first one listed. This is similar to how CKContainer.default behaves. You could imagine wanting similar behavior if you have a wrapper around the Keychain APIs, for example.

However, entitlements end up getting embedded inside your app binary. They are not a copied-in resource, but are rather a blob of data stuck in your executable file.

After lots of googling and asking friends, I eventually found two resources that were exceptionally helpful:

1️⃣ The first was this answer by Cédric Luthi. In it, he shows how to use the dyld APIs to find your executable image, and then iterate through the sections defined in that image until you find the one you’re looking for. In his case, he wanted the LC_UUID section. In order to read your own entitlements, you want the LC_CODE_SIGNATURE section.

Once you’ve found the LC_CODE_SIGNATURE section, you know roughly where in your executable file the entitlements are located. However, the resulting __LINKEDIT section has a largely undocumented format, and it wasn’t until Daniel Jalkut suggested I find the codesign source that I made any progress.

2️⃣ After some targeted googling, I found this source code on This file appears to be a decent amount of the source for the codesign utility, but the really awesome bit is that lc_load_sig function. There, finally, is how to poke around in that special __LINKEDIT section and interpret what’s going on.

Armed with these two pieces of data, we can now build some code that will read the entitlements blob out of your own executable (and once you have the blob, you can run it through NSPropertyListSerialization to parse and introspect it):

#import <Foundation/Foundation.h>
#import <mach-o/dyld.h>
 * Structure of an embedded-signature MultiBlob (called a SuperBlob in the codesign source)
typedef struct __BlobIndex {
    uint32_t type;                   /* type of entry */
    uint32_t offset;                 /* offset of entry */
} CS_Blob;

typedef struct __MultiBlob {
    uint32_t magic;                  /* magic number */
    uint32_t length;                 /* total length of SuperBlob */
    uint32_t count;                  /* number of index entries following */
    CS_Blob index[];                 /* (count) entries */
    /* followed by Blobs in no particular order as indicated by offsets in index */
} CS_MultiBlob;

extern NSData *EntitlementsData(void) {

    // iterate through the headers to find the executable, since only the executable has the entitlements
    const struct mach_header *executableHeader = NULL;
    for (uint32_t i = 0; i < _dyld_image_count() && executableHeader == NULL; i++) {
        const struct mach_header *header = _dyld_get_image_header(i);
        if (header->filetype == MH_EXECUTE) { executableHeader = header; }

    if (executableHeader == NULL) { return nil; }

    // find if it's a 64-bit executable or not
    BOOL is64bit = executableHeader->magic == MH_MAGIC_64 || executableHeader->magic == MH_CIGAM_64;
    uintptr_t cursor = (uintptr_t)executableHeader + (is64bit ? sizeof(struct mach_header_64) : sizeof(struct mach_header));

    // iterate through the dyld commands to find the "LC_CODE_SIGNATURE" command
    const struct segment_command *segmentCommand = NULL;
    for (uint32_t i = 0; i < executableHeader->ncmds; i++, cursor += segmentCommand->cmdsize) {
        segmentCommand = (struct segment_command *)cursor;
        if (segmentCommand->cmd != LC_CODE_SIGNATURE) { continue; }

        const struct linkedit_data_command *dataCommand = (const struct linkedit_data_command *)segmentCommand;

        // jump to the offset specified by the command
        uintptr_t dataStart = (uintptr_t)executableHeader + dataCommand->dataoff;
        CS_MultiBlob *multiBlob = (CS_MultiBlob *)dataStart;
        if (ntohl(multiBlob->magic) != 0xfade0cc0) { return nil; }

        // iterate through the blobs in this segment until we find the one with the appropriate magic value
        uint32_t count = ntohl(multiBlob->count);
        for (int i = 0; i < count; i++) {
            uintptr_t blobBytes = dataStart + ntohl(multiBlob->index[i].offset);
            uint32_t blobMagic = ntohl(*(uint32_t *)blobBytes);
            if (blobMagic != 0xfade7171) { continue; }

            // the first 4 bytes are the magic
            // the next 4 are the length
            // after that is the encoded plist
            uint32_t blobLength = ntohl(*(uint32_t *)(blobBytes + 4));
            return [NSData dataWithBytes:(const void *)(blobBytes + 8) length:(blobLength - 8)];

    return nil;


It turns out that the code above only works when you’re deploying to a device. If you’re just building for simulator, then the entitlements actually go in a different location; they’re in the __TEXT segment in a section called __entitlements, and they’re the sole contents of that section. This means reading them is very easy:

NSData *ReadEntitlementsFromTEXT(const struct mach_header *executable) {
    uint32_t dataOffset;
    uint64_t dataLength;

    BOOL is64bit = executable->magic == MH_MAGIC_64 || executable->magic == MH_CIGAM_64;
    if (is64bit) {
        const struct section_64 *section = getsectbynamefromheader_64((const struct mach_header_64 *)executable, "__TEXT", "__entitlements");
        dataOffset = section->offset;
        dataLength = section->size;
    } else {
        const struct section *section = getsectbynamefromheader(executable, "__TEXT", "__entitlements");
        dataOffset = section->offset;
        dataLength = (uint64_t)section->size;

    uintptr_t dataStart = (uintptr_t)executable + dataOffset;
    return [NSData dataWithBytes:(const void *)dataStart length:dataLength];

Misusing enums

This is a response to Matt Diephouse’s article, which is itself a response to John Sundell’s article. You should go read these first.

Matt starts off by saying:

Earlier this week, John Sundell wrote a nice article about building an enum-based analytics system in Swift. He included many fine suggestions, but I believe he’s wrong about one point: that enums are the right choice.

Therefore, I’ll start similarly:

Earlier this week, Matt Diephouse wrote a nice article about building a struct-based analytics system in Swift. He included many fine suggestions, but I believe he’s wrong about one point: that structs are the right choice.

As the examples in both articles show, analytic events can have different payloads of information that they’re going to capture. Matt uses a metadata property (a Dictionary<String, String>) to capture this, but this approach imposes a significant burden at the callsite. In order to properly fill out an AnalyticEvent struct, the callsite must know how to destructure the current model/event information and package it up inside the metadata property.

This could be fine if you’re dealing with a very small app, but given that we’re at the point where we’re putting in analytics, the chances that this app is small is pretty remote.

Additionally, it’s often not the case that analytics are defined at the application level. They’re typically brought in as part of an underlying framework. This means that the top-most levels of abstraction in your app (the app itself) have to know intimate, implementation details about how levels far below (the raw analytic communication protocol) work and expect their data to be formatted. Now, it’s possible that the metadata is a very loose bucket of “anything goes here”, but that still requires a lot of extra code at the callsite.

And when you’re putting in analytics, you want them to be as least invasive as possible, because they’re not the point of your app.

So, in my opinion, the better approach would be to define your analytics by a protocol, like this:

protocol AnalyticEvent {
    var name: String { get }
    var payload: Dictionary<String, String> { get }

The analytics capture mechanism would then only accept values of type AnalyticEvent (the protocol), and not define concrete types itself.

Here are some of the benefits we get by taking this approach:

Semantically appropriate event definition

Each layer of your app can define its own types (whether an enum, class, or struct) that conform to the AnalyticEvent protocol. Perhaps your Networking layer defines events that capture how often it has to hit the network versus how often it has to hit the cache. It can define those in its own custom NetworkingEvent: AnalyticEvent type.

Minimized callsites

By having each layer define its own type, it can create minimized initializers for its custom AnalyticEvent that would be impractical to do with an enum, and cumbersome with a struct. For example, if I had a UserSessionEvent: AnalyticEvent, I could create an initializer that takes a LoginFailureReason as the parameter, and then the initializer turns it in to the privately-known name and payload.

Type checking events

I could create a whole bunch of extensions on an AnalyticsEvent struct to do the custom initializer for the struct that is contextually appropriate for the callsite, but that explodes the numbers of initializers I have to sort through when creating an AnalyticsEvent. The autocomplete would show me every possible initializer for every possible event type, which is a total pain.

On the other hand, by requiring a custom type that adopts the AnalyticsEvent protocol, I can narrow my focus down to only the autocompletion results that are possible for a UserSessionEvent or a NetworkingEvent or a AwesomeCarouselWidgetInteractionEvent, etc.

With this sort of type-checking in place, refactoring these events also becomes easy. I can search the codebase for just NetworkingEvent to see every place where that type of event is getting generated.

Easy extensibility

Since AnalyticEvent is a protocol, adding a new kind of event is trivial. I don’t have to add a new case to an enum. I don’t have to litter stringly-typed event names throughout my code. I don’t have to add another extension to a struct. The protocol makes it easy to isolate concerns to their respective layers.

So, next time you reach for an enum, ask yourself whether you’re going to switch over it. If you’re not, you’re probably better off with a protocol instead.


It should go without saying that I really appreciate both Matt and John discussing these patterns in a public forum. In no way should my comments be construed as any sort of personal attack or judgement of Matt or John.

Level up your debugging skills

Finding the root cause of an error in your app can often feel very intimidating, whether you’re brand-new to programming or you’ve been building coding for decades. Debugging problems can be extremely time consuming. Where do you start looking? How do you know if what you think is the problem is actually the problem?

While there is no one-size-fits-all approach to debugging, there is a major guiding principle that can be extremely helpful in determining the root cause of an error, and that is to apply the scientific method to your analysis. The Scientific Method is a process we use to learn new things, analyze why things happen, and correct misconceptions. In simple terms, the steps involved are:

  1. Observe
  2. Ask questions
  3. Hypothesize
  4. Test


There are many ways to observe problems in your app. Maybe it crashed. Maybe it didn’t do something you were expecting it to. Maybe your users or QA testers observed some incorrect behavior. Maybe you saw something yourself. Regardless of how it happened, debugging always starts with an observed issue.

Ask Questions

Questions form the basis of debugging (and indeed, all learning in general). Depending on your goals, deadlines, or other constraints and circumstances, your questions may range from “why is this happening and how do I fix the root cause?” to “how do I hide the symptoms of this issue?”

There are no wrong questions here. Some questions may ultimately lead to irrelevant information, but the process of asking these questions helps train your mind to ask better questions in the future.

It’s also possible that your observations lead you to areas where you don’t have the required domain knowledge to even know what questions to ask. In situations like this, you should talk to other developers and ask “what questions should I be asking?”. Try to not ask for the answer, but instead ask for the question. Getting an answer gives you a single point of reference. But getting a question gives you a trajectory for future learning.


Once you’ve formed a question, next comes a very fun part: you get to make up an answer! The answer you invent can be anything, as long as it is testable. For example, you could hypothesize that the reason your app crashes is because the Moon was exactly at its orbital apogee. This is unlikely to be the root cause of your app’s crash (unless you’re writing an astronomy app?), but the point is that your hypothesis can be entirely made up.

Over time, experience will teach you to recognize patterns. For example, experienced Objective-C developers have long recognized that an EXC_BAD_ACCESS crash is likely an error related to memory management. Swift developers know what a crash due to force-unwrapping nil looks like. The more practice you get at debugging, the easier and quicker this process will become.


After you’ve formed a hypothesis, now you get to perform an experiment. In order to analyze the hypothesis, you need to devise a way to either prove or disprove your theory. Sometimes, this is really trivial: run your app again, set a breakpoint, and check to see if your variable is nil when you try to unwrap it.

Sometimes the process is a lot more complex. This is where the developer tools can be invaluable.

At the most fundamental level, we have the ability to print messages to the console. This is often referred to as “caveman debugging“, because it’s using the simplest and most fundamental tools. Sometimes this is good enough. But print() and NSLog() can be tedious tools to work with.

Fortunately, Xcode provides some really useful debugging tools beyond print() and NSLog(), like breakpoints and watchpoints. The debug gauges in the Debug Navigator can help you observe the state of your app. If you see memory usage continuously increasing, then you have Observed Something and may need to consider Asking More Questions. You also have more advanced tools available, like the LLDB console, everything in Instruments, and even hyper-specialized tools like dtrace.

These tools help you analyze the results of your experiment. The experiment you construct should ideal follow proper procedures and allow you to test your variables in isolation, as well as having a control.

Ideally, these experiments and analytic tools help you prove your hypothesis correct. If they do, then it’s time to start figuring out how to solve the problem. Often, testing proves our hypothesis false. We prove that what we thought was the issue wasn’t actually the problem. When this happens, we go back to step one: we’re still observing the problem, which means we need to ask new questions, form new hypotheses, and devise new tests to examine the theories.

And of course, these tests you devise should ideally be captured in the unit tests of your app, to help you guarantee that the problem doesn’t resurface in the future as other code changes!

Parting Thoughts

This pattern of Observe-Question-Hypothesize-Test is extremely powerful. It doesn’t always work for every kind of problem (such as problems that are inherently not reproducible), but when it’s applicable it is an excellent way to organize your thoughts and know that you’re on the right path towards solving your app’s problems and becoming a better and wiser developer.

So the next time you’re stuck on a bug and don’t know how to proceed, consider applying the scientific method.

My Rating System

Recently I’ve posted a couple of tweets rating some movies I’ve seen, and I almost always get asked about my rating system, because it’s a little unusual.

I rate things out of seven stars. This is something I learned from my dad, and while it sounds weird at first, it actually makes a whole lot of sense.

We’re all fairly used to a five-star rating system, but it ends up not being quite specific enough. For example, 3/5 stars would be an average rating (because 3 is in the middle of the 1-5 range). 5/5, being 100%, is “perfect”. But that means there’s only one gradation left between the two to express the concept of something that’s “better than average” but not “perfect”: 4/5. That’s not enough.

Thus, the 7-star scale.

With a 7-star scale, you still have a definitive middle value (4/7), but you have three values on each side of the middle to express the “low”, “medium”, and “high” values of positive and negative.


★☆☆☆☆☆☆ – downright terrible
★★☆☆☆☆☆ – really bad
★★★☆☆☆☆ – bad
★★★★☆☆☆ – average
★★★★★☆☆ – good
★★★★★★☆ – great
★★★★★★★ – truly excellent

(I suppose there is, in theory, a 0/7 rating, but I’ve yet to come across something so bad that I would rate it that low. And no, I don’t want suggestions on what that might be. I also can’t think of many 7/7 movies I’ve seen. Wonder Woman might be close, though.)

When I talk about movies with my friends, we agree that this sort of granularity is nice. For example, in my tweets above, I can clearly indicate that The Foreigner was (in my opinion) a better movie than Blade Runner: 2049, but they were both still good, while not being on-par with Thor: Ragnarok.

This then leads to the next question: why not a 9-star scale? It has the same benefit of a definitive middle value (5/9), but you start then getting in to the existential questions of “how is 6/9 different from 7/9, or 7/9 from 8/9?”. The nice thing about the 7-star scale is you have the “Low-Medium-High” range for all parts of the scale.

So, that’s why I use a 7-star scale. And that’s also probably a whole lot more than you ever really wanted to know.

A Better MVC, Part 4: Future Directions

Part 4 in a series on “fixing” Model-View-Controller:

  1. The Problems
  2. Fixing Encapsulation
  3. Fixing Massive View Controller
  4. Future Directions

There are other ways you can apply these principles to writing more maintainable apps.

View Controller-based Cells

One that I’m really interested in and am actively researching is how to use view controllers as cell content.

Cells can be really complicated things, architecturally. If you have any sort of dynamic content in a cell, you’re often faced with weird situations where you wonder “where does this logic belong?”. Do you put image loading in your view subclass? Allow your cell to handle complex business logic? Tack on additions to the list’s delegate protocol so you can message things back out (which then just complicates your list view controller)?

Using view controllers as cells helps answer these questions. It is natural to place this sort of code in the view’s controller, and the fact that the view happens to be a cell doesn’t really make a difference to the underlying principle.

Small finite lists

It’s really easy to make a view controller a cell when you have a list of a finite size. You have your “ListViewController”, and you give it an array of view controllers. It adds them as children, and then pulls out the appropriate one when its time to show a cell, and sticks the view controller’s view inside the cell’s contentView.

You can then apply the same principles of “flow view controllers” and “mini view controllers” to the content of the cell, and use child view controllers to manage portions or variations of the cell. I used to write a Mac app where I used this approach, and I could sometimes get upwards of 15 view controllers in a single cell. Granted, the cells were pretty complicated in what they could do, but none of these view controllers was longer than 100 lines of code. It made debugging issues trivially easy.

At this level of granularity in a finite list, you also probably aren’t very worried about view lifecycle callbacks, because you can pre-allocate everything.

Huge finite lists

Once you start moving past the point where you can pre-allocate everything, you have two main approaches you can take. The first approach (“huge finite lists”) is like what the current UITableView and UICollectionView API offer: you know up-front how many items are in a section. In this case, your ListViewController would likely have a similar looking datasource API as UITableView, where it progressively asks for view controllers and then uses the underlying “willBeginDisplayingCell:” etc callbacks to notify on lifecycle.

At this level you might also want to start thinking about view controller reuse, but I would probably avoid thinking about that unless I measured and determined that view controller deallocation and initialization was a measurable bottleneck.

Infinite lists

Then there’s the problem of infinity. A great example of this is the main screen of the Reddit app. You can scroll forever, and the content will just keep loading and loading… there’s no way to know up-front how much content there is. With this, you’ll be looking at loading in “slices” of the infinite view controller sequence, or taking a page (haha) out of UIPageViewController‘s book and using the “doubly-linked list” sort of API (“what’s the view controller after A? What’s the view controller after B? etc).

You’ll still have the underlying callbacks to help manage view lifecycle, and you’ll also still have to consider view controller reuse as an option.


As you get in to this style of programming, you’ll find that you end up developing a decent amount of boilerplate around embedding view controllers. That is to be expected, because the view controller containment APIs tend to offer the bare minimum to do what you need.

As you find situations where the API can be improved, please request that these improvements be made in the system frameworks.


There’s a lot to digest in these posts. Some people may scream in horror at the idea of “view controllers as cells”, but it’s an idea worth exploring.

So, the huge TL;DR of this is:

  • Decompose your UI using view controllers
  • Use view controllers to manage sequence
  • View controllers don’t have to fill the screen

I’d love to hear your thoughts on this topic. Hit me up on twitter or via email and let’s chat!