HTTP in Swift, Part 10: Cancellation

Part 10 in a series on building a Swift HTTP framework:

Cancelling an in-progress request is an important feature of any networking library, and it’s something we’ll want to support in this framework as well.

The Setup

In order to support cancellation, we’ll need to make the last major change to the API we’ve built so far, which looks like this:

open class HTTPLoader {

    func load(request: HTTPRequest, completion: @escaping (HTTPResult) -> Void)
    func reset(with group: DispatchGroup)

}

The limitation we see with this is that once we’ve started loading a request, we have no way to refer to that “execution” of the request; recall that an HTTPRequest is a value type, so it may be duplicated and copied around an infinite number of times.

So, we shall need to introduce some state to keep track of the task of loading and completing an HTTPRequest. Taking a cue from URLSession, I call this an HTTPTask:

public class HTTPTask {
    public var id: UUID { request.id }
    private var request: HTTPRequest
    private let completion: (HTTPResult) -> Void

    public init(request: HTTPRequest, completion: @escaping (HTTPResult) -> Void) {
        self.request = request
        self.completion = completion
    }

    public func cancel() {
        // TODO
    }

    public func complete(with result: HTTPResult) {
        completion(result)
    }
}

Unsurprisingly, we’ll need to change HTTPLoader to use this:

open class HTTPLoader {
    ...
    open func load(task: HTTPTask) {
        if let next = nextLoader {
            next.load(task: task)
        } else {
            // a convenience method to construct an HTTPError 
            // and then call .complete with the error in an HTTPResult
            task.fail(.cannotConnect)
        }
    } 
    ...
}

Constructing a task might be a bit verbose for clients, so we’ll keep the original method around as a convenience:

extension HTTPLoader {
    ...
    public func load(request: HTTPRequest, completion: @escaping (HTTPResult) -> Void) -> HTTPTask {
        let task = HTTPTask(request: request, completion: completion)
        load(task: task)
        return task
    } 
    ...
}

That’s the basic infrastructure. Now let’s talk about cancellation.

The elephant in the room where it happened

Cancellation is an enormously complicated topic. On the surface it seems pretty simple, but even a quick look below quickly turns messy. First off, what does cancellation actually mean? If I have some sort of request and I “cancel” it, what’s the expected behavior?

If I cancel a request before I pass it to a loader, should the completion handler fire? Why or why not? If I pass a cancelled request to a loader, should the loader try to load it? Why or why not?

If I cancel a request after I’ve started loading it but before it hits the terminal loader, should that be recognized by the current loader? Should a cancelled request be passed further on down the chain? If not, who’s responsible for invoking the completion handler, if it’s not the last loader?

If I cancel a request after it arrives at the terminal loader, should it stop the outgoing network connection? What if I’ve already started receiving a response? What if I’ve already got the response but haven’t started executing the completion handlers yet?

If I cancel a request after the completion handlers have executed, should anything happen? Why or why not?

And how do I do all of this while still allowing for thread safety?

These are complicated questions with even more complicated answers, and in no way do I claim to have all the answers, nor do I even claim to have good code to try and implement these answers. Implementing a correct cancellation scheme is notoriously difficult; ask any developer who’s tried to implement their own NSOperation subclass.

As I explain the concepts around cancellation in our networking library, please understand that the code and concepts are incomplete. I warned you about this in the first post. Thus, there will be a lot of // TODO: comments in the code.

The 'Get out of jail free' card from monopoly with the description that 'this code is left as an exercise to the reader'

Reacting to cancellation

So we’ve got this cancel() method now on our HTTPTask, but we need a way for various loaders to react to its invocation. Basically, we need a list of closures to run when a task gets cancelled. Let’s add an array of “cancellation callbacks” to the task for this purpose:

public class HTTPTask {
    ...
    private var cancellationHandlers = Array<() -> Void>()

    public func addCancellationHandler(_ handler: @escaping () -> Void>) {
        // TODO: make this thread-safe
        // TODO: what if this was already cancelled?
        // TODO: what if this is already finished but was not cancelled before finishing?
        cancellationHandlers.append(handler)
    } 

    public func cancel() {
        // TODO: toggle some state to indicate that "isCancelled == true"
        // TODO: make this thread-safe
        let handlers = cancellationHandlers
        cancellationHandlers = []

        // invoke each handler in reverse order
        handlers.reversed().forEach { $0() }
    }
}

Food for thought: Cancellation handlers should be invoked in LIFO order. Why?

In our loader for interacting with URLSession, we can now cancel our URLSessionDataTask if cancel() is invoked on the HTTPTask:

public class URLSessionLoader: HTTPLoader {
    ...

    open func load(task: HTTPTask) {
        ... // constructing the URLRequest from the HTTPRequest
        let dataTask = self.session.dataTask(with: urlRequest) { ... }

        // if the HTTPTask is cancelled, also cancel the dataTask
        task.addCancellationHandler { dataTask.cancel() }
        dataTask.resume()
    }
}

This gives us the basics of cancellation. If we cancel after a task has reached the terminal loader, it will cancel the underlying URLSessionDataTask and allow the URLSession response mechanism to dictate the subsequent behavior: we’ll get a URLError back with the .cancelled code.

As it currently stands, if we cancel a request before it reaches the terminal loader, nothing happens. And if we cancel a request after it finishes loading, again nothing will happen.

The “correct” behavior is a complicated interplay of what your needs are, coupled with what is reasonable to implement. A “100%” correct solution will require some extremely careful work involving synchronization primitives (such as an NSRecursiveLock) and very careful state management.

And it should go without saying, that no solution for proper cancellation is correct unless it is also accompanied by copious amounts of unit tests. Congratulations! You have fallen off the map; here be dragons.

Caveat Implementor: Let the implementor beware!

An Auto-cancelling Loader

We’ll wave our hands at this point and assume that our cancellation logic is “good enough”. To be honest, a naïve solution will likely be “good enough” for a decent majority of cases, so even this simple array of “cancellation handlers” will suffice for a while. So let’s forge ahead and build a loader based on cancellation.

We’ve established previously that we need the ability to “reset” a loader chain to provide semantics of “starting over from scratch”. Part of “starting over” would be to cancel any in-flight requests that we have; we can’t “start over” and still have remnants of our previous stack still going on.

The loader we build will therefore tie “cancellation” in with the concept of “resetting”: when the loader gets a call to reset(), it’ll immediately cancel() any in-progress requests and only allow resetting to finish once all of those requests have completed.

This means we’ll need to keep track of any requests that pass through us, and forget about them when they finish:

public class Autocancel: HTTPLoader {
    private let queue = DispatchQueue(label: "AutocancelLoader")
    private var currentTasks = [UUID: HTTPTask]()
    
    public override func load(task: HTTPTask) {
        queue.sync {
            let id = task.id
            currentTasks[id] = task
            task.addCompletionHandler { _ in
                self.queue.sync {
                    self.currentTasks[id] = nil
                }
            }
        }
        
        super.load(task: task)
    }
}

When a task comes, we’ll add it to a dictionary of known tasks; we’ll look it up based on the task’s identifier. Then when the task finishes, we’ll remove it from our dictionary. In this manner, we’ll have an always up-to-date mapping of tasks that are ongoing but have not completed yet.

Our loader also needs to react to the reset() method:

public class Autocancel: HTTPLoader {
    ...
    public override func reset(with group: DispatchGroup) {
        group.enter() // indicate that we have work to do
        queue.async {
            // get the list of current tasks
            let copy = self.tasks
            self.tasks = [:]
            DispatchQueue.global(qos: .userInitiated).async {
                for task in copy.values {
                    // cancel the task
                    group.enter()
                    task.addCompletionHandler { _ in group.leave() }
                    task.cancel()
                }
                group.leave()
            }
        }
        
        nextLoader?.reset(with: group)
    }
    
}

This logic is a little subtle, so I’ll explain:

When the reset() call comes in, we immediately enter the DispatchGroup to indicate that we have some amount of work to perform. Then we’ll grab the list of current tasks (ie, whatever’s in the dictionary).

For each task, we enter the DispatchGroup again to tie the lifetime of that particular task to the overall reset request. When the task is “done”, that task will leave the group. We then instruct the task to cancel().

After we’re done instructing each task to cancel, we leave the DispatchGroup to correctly balance out our initial enter() call.

This implementation is a prime example of the advantage of using a DispatchGroup as the coordinating mechanism for resetting. We cannot know at compile time which task will finish first, or if there are even any tasks to cancel at all. If we were using a single completion handler as the way to signal “done resetting”, we would have a very difficult time implementing this method correctly. Since we’re using a DispatchGroup, all we have to do instead is enter() and leave() as many times as needed.

These two methods mean that when this loader is included in our chain, we will automatically cancel all in-flight requests as part of the overall “reset” command, and resetting will not complete until after all the in-flight requests are finished. Neat!

// A networking chain that:
// - prevents you from resetting while a reset command is in progress
// - automatically cancels in-flight requests when asked to reset
// - updates requests with missing server information with default or per-request server environment information
// - executes all of this on a URLSession

let chain = resetGuard --> autocancel --> applyEnvironment --> ... --> urlSessionLoader

In the next post, we’ll be taking a look at how to automatically throttle outgoing requests so we don’t accidentally DDOS our servers.

The Setup

The elephant in the room where it happened

Reacting to cancellation

An Auto-cancelling Loader

Related️️ Posts️