HTTP in Swift, Part 3: Request Bodies

Part 3 in a series on building a Swift HTTP framework:

Before moving on to sending our HTTPRequest values, let’s make an improvement to our struct. Last time, we ended up with this basic definition:

public struct HTTPRequest {
    private var urlComponents = URLComponents()
    public var method: HTTPMethod = .get
    public var headers: [String: String] = [:]
    public var body: Data?
}

In this post, we’re going to take a look at that body property and improve it.

Generalizing Bodies

As we learned in the introduction to HTTP, a request’s body is raw binary data, which we have expressed here. However, there are several standard formats for this data that are commonly used when communicating with web APIs, such as JSON and form submissions.

Instead of requiring clients of this code to manually construct the binary representation of their submission data, we can generalize it to a form of “a thing that gives us the data”.

Since we aren’t going to place any restrictions on the algorithm used to construct the data, it makes sense to define this functionality via a protocol and not a concrete type:

public protocol HTTPBody { }

Next, we need a way to get the Data out of one of these values, and optionally report an error if something goes wrong:

public protocol HTTPBody { 
    func encode() throws -> Data 
}

We could stop at this point, but there are two other pieces of information that would be nice to have:

public protocol HTTPBody { 
    var isEmpty: Bool { get }
    var additionalHeaders: [String: String] { get } 
    func encode() throws -> Data 
}

If we can quickly know that a body is empty, then we can save ourselves the trouble of attempting to retrieve any encoded data and dealing with either an error or an empty Data value. Additionally, some kinds of bodies work in conjunction with headers in the request. For example, when we encode a value as JSON, we’d like a way to specify the Content-Type: application/json header automatically, without also having to manually specify this on the request. To that end, we’ll allow these types to declare additional headers that will end up as part of the final request. To simplify adoption even further, we can provide a default implementation for these:

extension HTTPBody {
    public var isEmpty: Bool { return false }
    public var additionalHeaders: [String: String] { return [:] }
}

Finally, we can update our type to use this new protocol:

public struct HTTPRequest {
    private var urlComponents = URLComponents()
    public var method: HTTPMethod = .get
    public var headers: [String: String] = [:]
    public var body: HTTPBody?
}

EmptyBody

The simplest kind of HTTPBody is “no body” at all. With this protocol, defining an empty body is simple:

public struct EmptyBody: HTTPBody {
    public let isEmpty = true

    public init() { }
    public func encode() throws -> Data { Data() }
}

We can even go so far as to make this the default body value, removing the need for the optionality of the property entirely:

public struct HTTPRequest {
    private var urlComponents = URLComponents()
    public var method: HTTPMethod = .get
    public var headers: [String: String] = [:]
    public var body: HTTPBody = EmptyBody()
}

DataBody

The next obvious kind of body to implement would be one that returns any Data value it was given. This would be used in cases where we don’t necessarily have an HTTPBody implementation, but perhaps we already have the Data value itself to send.

The definition is trivial:

public struct DataBody: HTTPBody {    
    private let data: Data
    
    public var isEmpty: Bool { data.isEmpty }
    public var additionalHeaders: [String: String]
    
    public init(_ data: Data, additionalHeaders: [String: String] = [:]) {
        self.data = data
        self.additionalHeaders = additionalHeaders
    }
    
    public func encode() throws -> Data { data }    
}

With this, we can easily wrap an existing Data value into an HTTPBody for our request:

let otherData: Data = ...
var request = HTTPRequest()
request.body = DataBody(otherData)

JSONBody

Encoding a value as JSON is an incredibly common task when sending network requests. Making an HTTPBody to take care of this for us is now easy:

public struct JSONBody: HTTPBody {
    public let isEmpty: Bool = false
    public var additionalHeaders = [
        "Content-Type": "application/json; charset=utf-8"
    ]
    
    private let encode: () throws -> Data
    
    public init<T: Encodable>(_ value: T, encoder: JSONEncoder = JSONEncoder()) {
        self.encode = { try encoder.encode(value) }
    }
    
    public func encode() throws -> Data { return try encode() }
}

First, we assume that whatever value we get will result in at least something, because even an empty string encodes to a non-empty JSON value. Thus, isEmpty = false.

Next, most servers want a Content-Type of application/json when receiving a JSON body, so we’ll assume that’s the common case and default that value in the additionalHeaders. However, we’ll leave the property as var in case there’s a rare situation in which a client wouldn’t want this.

For encoding, we need to accept some generic value (the thing to encode), but it’d be nice to not have to make the entire struct generic to the encoded type. We can avoid the type’s generic parameter by limiting it to the initializer, and then capturing the generic value in a closure.

We also need a way to provide a custom JSONEncoder, so that clients get an opportunity to fiddle with things like the .keyEncodingStrategy or whatever. But, we’ll provide a default encoder to simplify usage.

Finally, the encode() method itself simply becomes an invocation of the closure we created, which captures the generic value and runs it through the JSONEncoder.

Using one of these looks like this:

struct PagingParameters: Encodable {
    let page: Int
    let number: Int
}
let parameters = PagingParameters(page: 0, number: 10)

var request = HTTPRequest()
request.body = JSONBody(parameters)

With this, the body will get automatically encoded as {"page":0,"number":10}, and our final request will have the proper Content-Type header.

Food for thought: The Combine framework defines a TopLevelEncoder protocol (which may eventually move into the standard library).

How would you alter JSONBody so that you could also provide a custom encoder that conforms to TopLevelEncoder?

FormBody

The last kind of body we’ll look at in this post is a body to represent a basic form submission. We’ll be saving file uploads for a future post when we talk specifically about multipart form uploads.

Form submission bodies end up as roughly-URL-encoded key-value pairs, such as name=Arthur&age=42.

We’ll start out with the same basic structure as our HTTPBody implementations:

public struct FormBody: HTTPBody {
    public var isEmpty: Bool { values.isEmpty }
    public let additionalHeaders = [
        "Content-Type": "application/x-www-form-urlencoded; charset=utf-8"
    ]
    
    private let values: [URLQueryItem]
    
    public init(_ values: [URLQueryItem]) {
        self.values = values
    }
    
    public init(_ values: [String: String]) {
        let queryItems = values.map { URLQueryItem(name: $0.key, value: $0.value) }
        self.init(queryItems)
    }
    
    public func encode() throws -> Data {
        let pieces = values.map { /* TODO */ }
        let bodyString = pieces.joined(separator: "&")
        return Data(bodyString.utf8)
    }
}

Like before, we have a custom Content-Type header to apply to the request. We also expose a couple of initializers so that clients can describe the values in a way that makes sense to them. We’ve also stubbed out most of the encode() method, leaving out the actual encoding of a URLQueryItem values.

Encoding the name and value is, unfortunately, a little ambiguous. If you go read through the ancient specifications on form submissions, you’ll see things referring to “newline normalization” and encoding spaces as +. We could go through the effort of digging around and finding out what those things mean, but in practice, web servers tend to handle anything that’s percent-encoded just fine, even spaces. We’ll take a shortcut and assume that this will be true. We’ll also make the blanket assumption that alphanumeric characters are fine within a name and value, and that everything else should be encoded:

private func urlEncode(_ string: String) -> String {
    let allowedCharacters = CharacterSet.alphanumerics
    return string.addingPercentEncoding(withAllowedCharacters: allowedCharacters) ?? ""
}

Combining the name and the value happens with a = character:

private func urlEncode(_ queryItem: URLQueryItem) -> String {
    let name = urlEncode(queryItem.name)
    let value = urlEncode(queryItem.value ?? "")
    return "\(name)=\(value)"
}

And with this, we can resolve that /* TODO */ comment:

public struct FormBody: HTTPBody {
    public var isEmpty: Bool { values.isEmpty }
    public let additionalHeaders = [
        "Content-Type": "application/x-www-form-urlencoded; charset=utf-8"
    ]
    
    private let values: [URLQueryItem]
    
    public init(_ values: [URLQueryItem]) {
        self.values = values
    }
    
    public init(_ values: [String: String]) {
        let queryItems = values.map { URLQueryItem(name: $0.key, value: $0.value) }
        self.init(queryItems)
    }
    
    public func encode() throws -> Data {
        let pieces = values.map(self.urlEncode)
        let bodyString = pieces.joined(separator: "&")
        return Data(bodyString.utf8)
    }

    private func urlEncode(_ queryItem: URLQueryItem) -> String {
        let name = urlEncode(queryItem.name)
        let value = urlEncode(queryItem.value ?? "")
        return "\(name)=\(value)"
    }

    private func urlEncode(_ string: String) -> String {
        let allowedCharacters = CharacterSet.alphanumerics
        return string.addingPercentEncoding(withAllowedCharacters: allowedCharacters) ?? ""
    }
}

As before, using this becomes straight-forward:

var request = HTTPRequest()
request.body = FormBody(["greeting": "Hello, ", "target": "🌎"])
// the body is encoded as:
// greeting=Hello%2C%20&target=%F0%9F%8C%8E

Food for thought: If you come across a situation where a server does not correctly decode %20 as a space, which parts would you need to change, and how would you change them?

Other Bodies

The formats of bodies you can send in an HTTP request are infinitely varied. I’ve already mentioned that we’ll take a closer look at multipart requests in the future, but this HTTPBody approach works for just about every kind of body you’ll come across.

Food for thought: sometimes, it’s impractical to load an entire request body into memory before sending (such as when uploading a multi-megabyte file). In these cases, we’d likely want to use an InputStream to represent the encoded form of the body instead of a Data. How would you change these types to use InputStream instead?

In the next post, we’ll describe the HTTP request loading abstraction layer and implementing it with URLSession.