When discussing the topic of unit testing and automated tests in general, it’s incredibly common to use the term testable code — code that’s synchronous, predictable, and always produces the same output for a given set of inputs. Networking code, however, tends to fall on the complete opposite end of that spectrum.

What makes networking code so difficult to test is both the fact that it’s inherently asynchronous, and that it relies on several external factors — such as a solid internet connection, servers, and many other kinds of systems that are involved in performing, loading, and decoding the result of a given network request.

While we’ve already taken a look at several different ways to test asynchronous code in general, this week, let’s focus on how to test networking code in particular — using Foundation’s URLSession API, as well as the Endpoint system from last month’s “Creating generic networking APIs in Swift”, as a starting point.

This article is going to assume that you’ve read the above mentioned article about generic networking APIs (since the following code samples are going to build directly on the code introduced in that article), and that you’re familiar with the basics of unit testing.

Verifying request generation logic

When starting to write a brand new suite of tests, it’s typically a good idea to start from the bottom, and figure out how to reliably test the most foundational logic first, before moving on to testing higher-level APIs.

In this case, let’s start by verifying that the way we generate URLRequest values from a given Endpoint works as expected under a few different conditions. To make those tests as predictable as possible, let’s create a stubbed version of our EndpointKind protocol that doesn’t modify our requests in any way:

extension EndpointKinds {
    enum Stub: EndpointKind {
        static func prepare(_ request: inout URLRequest,
                            with data: Void) {
            // No-op
        }
    }
}

What makes a stub different from a mock is that a stub is simply used to fulfill one of our API requirements, without modifying our code’s behavior in any way.

With the above in place, we can actually already write our first test — in which we’ll verify that the correct request URL is generated for a basic endpoint that doesn’t contain any query items or HTTP headers:

class EndpointTests: XCTestCase {
    func testBasicRequestGeneration() {
        let endpoint = Endpoint<EndpointKinds.Stub, String>(path: "path")
        let request = endpoint.makeRequest(with: ())
        XCTAssertEqual(request?.url, URL(string: "https://api.myapp.com/path"))
    }
}

That’s a great start, but there are a few ways that the above code could be improved.

For starters, we currently could end up getting false positives if both the generated URLRequest and our expected URL turned out to be nil (which might seem unlikely, but when writing tests it’s typically a good idea to eliminate all sources of potential ambiguity).

Second, we’re currently assuming that our request’s host will always be api.myapp.com, which might not be the case — since modern apps often support multiple networking environments (such as staging and production), and our app might also need to talk to several servers with different host addresses.

Let’s start by addressing the first issue by using the built-in XCTUnwrap function to verify that our generated request didn’t end up being nil. We’ll also take this opportunity to create a typealias for our specialized Endpoint type as well — which gives us the following implementation:

class EndpointTests: XCTestCase {
    typealias StubbedEndpoint = Endpoint<EndpointKinds.Stub, String>

    func testBasicRequestGeneration() throws {
        let endpoint = StubbedEndpoint(path: "path")
        let request = try XCTUnwrap(endpoint.makeRequest(with: ()))
        XCTAssertEqual(request.url, URL(string: "https://api.myapp.com/path"))
    }
}

Next, to make it possible to control what exact host that a given request should target, let’s introduce a dedicated URLHost type — which will essentially be a simple wrapper around a raw String value:

struct URLHost: RawRepresentable {
    var rawValue: String
}

A major benefit of implementing dedicated types for specific kinds of values is that doing so lets us encapsulate our most common variations using enum-like static properties. For example, in this case we might create such properties for our staging and production hosts, as well as a convenience API called default that automatically resolves the right host depending on whether our app is running in DEBUG mode or not:

extension URLHost {
    static var staging: Self {
        URLHost(rawValue: "staging.api.myapp.com")
    }

    static var production: Self {
        URLHost(rawValue: "api.myapp.com")
    }

    static var `default`: Self {
        #if DEBUG
        return staging
        #else
        return production
        #endif
    }
}

With the above in place, let’s now update our Endpoint type’s URLRequest generation method by enabling a URLHost value to be injected into it (while keeping default as the, well, default):

extension Endpoint {
    func makeRequest(with data: Kind.RequestData,
                     host: URLHost = .default) -> URLRequest? {
        var components = URLComponents()
        components.scheme = "https"
        components.host = host.rawValue
        components.path = "/" + path
        components.queryItems = queryItems.isEmpty ? nil : queryItems

        guard let url = components.url else {
            return nil
        }

        var request = URLRequest(url: url)
        Kind.prepare(&request, with: data)
        return request
    }
}

To avoid having to manually write each URL that we’ll use for verification within our tests, let’s also extend URLHost with a test-specific convenience API that’ll enable us to easily generate an expected URL for a given path — like this:

extension URLHost {
    func expectedURL(withPath path: String) throws -> URL {
        let url = URL(string: "https://" + rawValue + "/" + path)
        return try XCTUnwrap(url)
    }
}

Not only does the above set of additions make our production code more capable and dynamic (since we’re now automatically supporting both staging and production environments, as well as custom hosts), we’re now able to make our initial test much easier to read — while also making it slightly more robust and predictable as well:

class EndpointTests: XCTestCase {
    typealias StubbedEndpoint = Endpoint<EndpointKinds.Stub, String>

    let host = URLHost(rawValue: "test")

    func testBasicRequestGeneration() throws {
        let endpoint = StubbedEndpoint(path: "path")
        let request = endpoint.makeRequest(with: (), host: host)

        try XCTAssertEqual(
            request?.url,
            host.expectedURL(withPath: "path")
        )
    }
}

While we did just spend a bit of time and effort on building a few test-specific convenience APIs, and to make our production code more dynamic, the benefit of doing that work is that we can now start writing many kinds of Endpoint verification tests very quickly. For example, here’s one that verifies the URLRequest generated for an endpoint that uses query items:

class EndpointTests: XCTestCase {
    ...

    func testGeneratingRequestWithQueryItems() throws {
        let endpoint = StubbedEndpoint(path: "path", queryItems: [
            URLQueryItem(name: "a", value: "1"),
            URLQueryItem(name: "b", value: "2")
        ])

        let request = endpoint.makeRequest(with: (), host: host)

        try XCTAssertEqual(
            request?.url,
            host.expectedURL(withPath: "path?a=1&b=2")
        )
    }
}

We might also want to add a few tests that’ll use some of our real endpoints as well, just to make sure that those are working correctly. It’s probably not worth testing all of our endpoints this way (it’s important to be pragmatic when writing tests, after all), but we might choose to do something like test one of our endpoints that require authentication — which’ll let us verify that the correct Authorization header was added when generating a request:

class EndpointTests: XCTestCase {
    ...

    func testAddingAccessTokenToPrivateEndpoint() throws {
        let endpoint = Endpoint.search(for: "query")
        let token = AccessToken(rawValue: "12345")
        let request = endpoint.makeRequest(with: token, host: host)

        try XCTAssertEqual(
            request?.url,
            host.expectedURL(withPath: "search?q=query")
        )

        XCTAssertEqual(request?.allHTTPHeaderFields, [
            "Authorization": "Bearer 12345"
        ])
    }
}

While there are certainly more tests that we could’ve written to cover other aspects of our Endpoint-related logic, the above three tests already give us a quite broad coverage, and will likely help us identify any substantial regressions to that logic in the future.

Integration tests instead of abstractions

Now let’s move on to testing our actual networking code. As a reminder, we’re currently using Foundation’s URLSession API in combination with a few Combine operators to build our core networking pipeline — which looks like this:

extension URLSession {
    func publisher<K, R>(
        for endpoint: Endpoint<K, R>,
        using requestData: K.RequestData,
        decoder: JSONDecoder = .init()
    ) -> AnyPublisher<R, Error> {
        guard let request = endpoint.makeRequest(with: requestData) else {
            return Fail(
                error: InvalidEndpointError(endpoint: endpoint)
            ).eraseToAnyPublisher()
        }

        return dataTaskPublisher(for: request)
            .map(\.data)
            .decode(type: NetworkResponse<R>.self, decoder: decoder)
            .map(\.result)
            .eraseToAnyPublisher()
    }
}

Successfully testing the above code is definitely not very straightforward, as it uses a series of system APIs that would each need to be mocked, stubbed, or controlled in some other manner in order for our tests to become completely predictable.

A very common technique used in situations like this is to abstract each system API and asynchronous operation behind a protocol, and while that’s certainly a fine approach in many cases, let’s use a somewhat different strategy this time.

Internally, URLSession uses a class called URLProtocol to actually perform our various network calls, and the system also offers complete support for building our own custom implementations using that class as well. That means that we’ll be able to completely replace the default HTTP networking stack with a mocked version within our tests, without having to modify the above Combine pipeline at all — essentially giving us an integration test that verifies how our own custom logic integrates with system components like URLSession and Combine.

One downside of URLProtocol from a testing perspective, however, is that it relies heavily on static methods, which means that we’ll have to implement our various mocks as separate types. To make that a bit easier, let’s start by introducing a MockURLResponder protocol — which will essentially let us write various “mock servers” that respond to a given request by either returning Data, or by throwing an error:

protocol MockURLResponder {
    static func respond(to request: URLRequest) throws -> Data
}

With the above in place, let’s now implement our custom URL protocol, which requires us to override four methods from the abstract URLProtocol base class — like this:

class MockURLProtocol<Responder: MockURLResponder>: URLProtocol {
    override class func canInit(with request: URLRequest) -> Bool {
        true
    }

    override class func canonicalRequest(for request: URLRequest) -> URLRequest {
        request
    }

    override func startLoading() {
        guard let client = client else { return }

        do {
            // Here we try to get data from our responder type, and
            // we then send that data, as well as a HTTP response,
            // to our client. If any of those operations fail,
            // we send an error instead:
            let data = try Responder.respond(to: request)
            let response = try XCTUnwrap(HTTPURLResponse(
                url: XCTUnwrap(request.url),
                statusCode: 200,
                httpVersion: "HTTP/1.1",
                headerFields: nil
            ))

            client.urlProtocol(self,
                didReceive: response,
                cacheStoragePolicy: .notAllowed
            )
            client.urlProtocol(self, didLoad: data)
        } catch {
            client.urlProtocol(self, didFailWithError: error)
        }

        client.urlProtocolDidFinishLoading(self)
    }

    override func stopLoading() {
        // Required method, implement as a no-op.
    }
}

The next thing that we’ll need is a way to tell a given URLSession instance to use our new mock protocol, rather than its default HTTP-based implementation. One way to do that would be to create a convenience initializer that takes a MockURLResponder-conforming type, and then creates a URLSession instance that uses a MockURLProtocol specialized with that responder type. We’ll also need to register that specialization with URLProtocol itself — which gives us the following implementation:

extension URLSession {
    convenience init<T: MockURLResponder>(mockResponder: T.Type) {
        let config = URLSessionConfiguration.ephemeral
        config.protocolClasses = [MockURLProtocol<T>.self]
        self.init(configuration: config)
        URLProtocol.registerClass(MockURLProtocol<T>.self)
    }
}

Typically, it’s not considered a good practice to have initializers generate side-effects (such as the above URLProtocol registration), as the act of simply creating an object should ideally not cause any other logic to be triggered. However, in this particular case, we might choose to optimize for ease of use (rather than for complete architectural correctness) as always having to manually register each mock URL protocol would be quite inconvenient.

With all of those underlying pieces in place, let’s now go ahead and create our first concrete implementation of our MockURLResponder protocol — for example one that responds with an encoded Item model:

extension Item {
    enum MockDataURLResponder: MockURLResponder {
        static let item = Item(title: "Title", description: "Description")

        static func respond(to request: URLRequest) throws -> Data {
            let response = NetworkResponse(result: item)
            return try JSONEncoder().encode(response)
        }
    }
}

Before we start writing our tests, however, let’s add one final piece of infrastructure that’ll make it much easier to synchronously wait for a given Combine publisher to complete (since that’s what our networking API is based on).

To do that, let’s follow an approach that’s very similar to what we did in “Useful APIs when writing scripts and tools in Swift” — only this time we’ll use XCTest’s built-in expectation system to implement our waiting logic, rather than using a Grand Central Dispatch semaphore:

extension XCTestCase {
    func awaitCompletion<T: Publisher>(
        of publisher: T,
        timeout: TimeInterval = 10
    ) throws -> [T.Output] {
        // An expectation lets us await the result of an asynchronous
        // operation in a synchronous manner:
        let expectation = self.expectation(
            description: "Awaiting publisher completion"
        )

        var completion: Subscribers.Completion<T.Failure>?
        var output = [T.Output]()

        let cancellable = publisher.sink {
            completion = $0
            expectation.fulfill()
        } receiveValue: {
            output.append($0)
        }

        // Our test execution will stop at this point until our
        // expectation has been fulfilled, or until the given timeout
        // interval has elapsed:
        waitForExpectations(timeout: timeout)

        switch completion {
        case .failure(let error):
            throw error
        case .finished:
            return output
        case nil:
            // If we enter this code path, then our test has
            // already been marked as failing, since our
            // expectation was never fullfilled.
            cancellable.cancel()
            return []
        }
    }
}

Just like when testing our Endpoint system earlier, we’ve now spent a fair amount of effort building out a quite complex set of underlying infrastructure, but all of that work is about to pay off in a big way — since we’ll now be able to test a wide range of networking scenarios with very few lines of code. For example, here’s how we could test that we’re able to successfully load and decode the response for a given request:

class NetworkIntegrationTests: XCTestCase {
    func testSuccessfullyPerformingRequest() throws {
        let session = URLSession(mockResponder: Item.MockDataURLResponder.self)
        let accessToken = AccessToken(rawValue: "12345")

        let publisher = session.publisher(for: .latestItem, using: accessToken)
        let result = try awaitCompletion(of: publisher)

        XCTAssertEqual(result, [Item.MockDataURLResponder.item])
    }
}

Really nice! As a second and final example, here’s how we could verify that our networking pipeline correctly produces a top-level error when a given request ended up failing:

enum MockErrorURLResponder: MockURLResponder {
    static func respond(to request: URLRequest) throws -> Data {
        throw URLError(.badServerResponse)
    }
}

class NetworkIntegrationTests: XCTestCase {
    ...

    func testFailingWhenEncounteringError() throws {
        let session = URLSession(mockResponder: MockErrorURLResponder.self)
        let accessToken = AccessToken(rawValue: "12345")

        let publisher = session.publisher(for: .latestItem, using: accessToken)
        XCTAssertThrowsError(try awaitCompletion(of: publisher))
    }
}

We could then continue expanding the above test case to include other scenarios as well — for example in order to make sure that our networking pipeline correctly handles JSON decoding failures, or to verify any custom error handling logic that we might have in place.

Conclusion

While there’s definitely a very large number of different approaches that we can take when testing our networking code and its associated logic, basing those tests on our actual pipelines and operations tends to make them more realistic, and in turn, more likely to catch bugs and regressions.

The techniques used in this article might initially seem rather complicated, but since they’re all based on core system networking APIs (like URLRequest, URLSession, and URLProtocol), they’re likely going to be easier to maintain that a complex series of interconnected protocols that are only really there to enable testability.

But, like always, I encourage you to experiment with various approaches in order to find one that fits your particular project the best. Either way, I hope that you found this article interesting, and if you have any questions, comments or other kinds of feedback — feel free to reach out via either Twitter or email.

Thanks for reading! 🚀