Almost all modern apps use JSON in one way or another. Whether it's for configurations, to store local data or to download information over the network - JSON is everywhere.

Since we often use JSON to such a heavy extent, errors in the way we deal with it (especially in our core data models) could pretty much render our app unusable. It's therefor super important that we have solid tests in place to make sure that never happens - and that those tests cover as many cases as possible.

This week, let's take a look at how we can set up our JSON mapping tests to make them a lot more robust and future proof, and how we can use them to perform end-to-end testing.

The problem

Most of the time when we are writing tests we stick to unit testing, that are verifying that a single unit of our app works as it should in isolation. These tests are super valuable, and make it easier to iterate on separate parts of our app without having to rewrite the entire test suite.

However, when it comes to JSON mapping, we are faced with a bit of a challenge. If we test our models' JSON mapping code completely in isolation - we actually have no idea whether it'll actually work once the app is run and it hits the real network. Let's take a look at an example.

Let's say that we have the following user model in our code:

struct User {
    let name: String
    let age: Int
}

Which we will express using the following JSON from our backend:

{
    "name": "John",
    "age": 29
}

Now let's write a test that verifies that our User model can be initialized using the above JSON. To do that, we bundle the JSON in our test target, load it in a test case, and finally verify that a User instance was successfully created. Like this:

class UserTests: XCTestCase {
    func testJSONMapping() throws {
        let bundle = Bundle(for: type(of: self))

        guard let url = bundle.url(forResource: "User", withExtension: "json") else {
            XCTFail("Missing file: User.json")
            return
        }

        let json = try Data(contentsOf: url)
        let user: User = try unbox(data: json)

        XCTAssertEqual(user.name, "John")
        XCTAssertEqual(user.age, 29)
    }
}

I'm using Unbox here, but feel free to substitute it with whichever JSON mapper you prefer.

Alright, all good so far 👌. But here comes the problem - what happens if the JSON that our backend sends changes in any way? That should ideally not happen (we should make sure to use proper versioning and integration tests so that a new version of the backend never breaks any client), but we all know - these things do happen from time to time.

The problem is; since we have bundled a JSON file in our tests, they will keep passing, and we'll think that everything is fine - but then once we start requesting real data from the backend - things will start to break 💥.

End-to-end

This is a situation where end-to-end tests become really valuable. The idea with that kind of tests is to perform testing that covers our entire stack vertically. In our case, it means that we'd want to test that data successfully travels all the way from our backend system, into our app and that it finally gets successfully mapped into a model.

Typically, we use UI Testing to perform end-to-end testing (check out "Getting started with Xcode UI testing in Swift" if you want to learn more), but those can be very expensive to run. We don't necessarily want to run through all of our UI in order to test our JSON mapping, and it becomes quite tricky to try to verify that our data is correct in a UI test too.

But it turns out, there's a nice middle ground solution that we can use 👍

A single source of truth

Like in most programming situations, being able to maintain a single source of truth makes things a lot simpler and less error prone (in fact, just last week - in "Modelling state in Swift" - we took a look at how single sources of truth give us much easier state management). So if we could base our client-side JSON mapping tests on real data from the backend, we would have a single (and auto-updating) source of truth.

But we don't want to make requests to our backend service every time we run our tests. That would again make them too expensive and slow to run (bye bye TDD(ish) workflows 😢). My solution to this problem is to write a script that automatically downloads the latest JSON from the server once per day, and then stores it for all test runs. Let's take a look at how to implement that.

The script

Let's start with the script. For that I'll use Marathon, which is a tool that enables you to easily write and run scripts using Swift. As an example, let's write a script that each day downloads some output from the GitHub search API.

If you have Marathon installed, you can simply run the following command to get started:

$ marathon create DownloadJSON

The above will create a DownloadJSON.swift script, and open Xcode to let you start coding. Now, for the actual script:

import Foundation
import Files // marathon:https://github.com/JohnSundell/Files.git

// We store all of our scripts in a 'Scripts' folder
let scriptsFolder = try Folder.current.subfolder(named: "Scripts")
let currentTimestamp = Date().timeIntervalSinceReferenceDate

// To only run this script once per day, verify against a .lastRun file
if let lastRunFile = try? scriptsFolder.file(named: ".lastRun") {
    let lastRunTimestamp = try TimeInterval(lastRunFile.readAsInt())
    let oneDayInterval: TimeInterval = 60 * 60 * 24
    let timestampDelta = currentTimestamp - lastRunTimestamp

    // Less than one day has passed, no need to run the script
    if timestampDelta < oneDayInterval {
        exit(0)
    }
}

// Download JSON
let query = "language:swift"
let url = URL(string: "https://api.github.com/search/repositories?q=\(query)")!
let json = try Data(contentsOf: url)

// Write JSON file
let resourceFolder = try Folder.current.subfolder(atPath: "Tests/Resources")
try resourceFolder.createFile(named: "User.json", contents: json)

// Save a new version of .lastRun
let lastRunFile = try scriptsFolder.createFile(named: ".lastRun")
try lastRunFile.write(string: String(Int(currentTimestamp)))

To get this script to run every time the tests are run, add a new Run script phase to your test target (click your project in Xcode's project navigator, select your test target, then go to Build phases and click the '+' symbol). Place it as early as possible in the build, preferably right after Target Dependencies. Name it "Download JSON" and add the following content:

marathon run Scripts/DownloadJSON

Now, run your tests - and you'll see a TestResources folder with a GitHubSearch.json file in it appear in the root folder of your project. Simply drag it into Xcode, and it to your test target, and we now have an auto-updating real JSON file that we can write tests against! 🎉

Let's write a test

With all the infrastructure we need in place, let's now write a test. We'll create a simple Repository model to represent a repository contained in the GitHub search results:

struct Repository {
    let id: Int
    let name: String
}

extension Repository: Unboxable {
    init(unboxer: Unboxer) throws {
        id = try unboxer.unbox(key: "id")
        name = try unboxer.unbox(key: "name")
    }
}

And then a test that verifies that our downloaded JSON file is indeed compatible with our model code:

class RepositoryTests: XCTestCase {
    func testJSONMapping() throws {
        let bundle = Bundle(for: type(of: self))

        guard let url = bundle.url(forResource: "GitHubSearch", withExtension: "json") else {
            XCTFail("Missing file: GitHubSearch.json")
            return
        }

        let json = try Data(contentsOf: url)
        let repositories: [Repository] = try unbox(data: json, atKeyPath: "items")

        // We can't make any assumptions about the data here, since it can update
        // at any time. We'll simply verify that the data is there.
        XCTAssertFalse(repositories.isEmpty)
    }
}

We now have an end-to-end test in place, that runs fast, auto-updates and makes sure that our JSON mapping code will always work against real data from our backend! 👍

Conclusion

Both unit tests and end-to-end tests have great value - the unit tests verify that specific pieces of data get mapped correctly, while the end-to-end tests verify that things work when used with real data.

My advice is to use both, as well as adding UI tests for your most important user flows. Deploying these kind of tests tactically can really help you avoid painful situations & bugs, and gives you more confidence in releasing new versions of your app.

What do you think? Do you already use something similar to this kind of end-to-end JSON mapping tests in your projects, or is it something you'll try out? Let me know, along with any questions, comments or feedback that you might have - on Twitter @johnsundell.

Thanks for reading 🚀