When writing any kind of automated tests, it can sometimes be tricky to achieve a balance between making tests run efficiently and predictably, while still exercising the right code paths under realistic conditions. On one hand, we don’t want our tests to take too long to run, or to start failing because of external conditions we can’t control (such as network availability) — but on the other hand, if we make our tests too artificial, then what are we really testing?

One way to get closer to that kind of balance is to use multiple kinds of automated tests — each with a different level of artificialness — which lets us verify that our code works as expected from different perspectives. This week, let’s take a look at one such kind of tests, which is all about verifying how real instances of various objects interact with each other — commonly known as integration tests.

The integration of multiple units

Any code base can usually be separated into different parts — both by high-level features, such as Search, Login, Networking, and so on — and at a much lower level, by looking at individual classes and functions. A common way to test all those different parts is to do unit testing, and test each part (or unit) in isolation.

Usually when writing such tests, we want our units to be as small as possible (an individual class from a feature, rather than the feature as a whole), and we also want to limit the knowledge that unit has about the outside world (through techniques like dependency injection and mocking). The benefit of unit tests is that they can often be kept quite simple, and can be executed in a very predictable (although often quite artificial) environment.

When writing integration tests on the other hand, we want to verify how multiple units actually interact (or integrate) with each other, which requires us to change our tactics a bit when it comes to how we deal with a given object’s dependencies. Rather than mocking everything outside of a single unit, we instead use real objects to perform our tests and verify their outcomes. The benefit is that we might be able to catch bugs and problems that only occur once that real integration happens.

To better illustrate the difference between unit tests and integration tests — let’s start by taking a look at an example.

A logical unit

Let’s say that we’re working on an app that contains a Database class, which is an abstraction we’ve created on top of whichever database our app uses — for example Core Data, or SQLite. It exposes APIs for storing and loading different database entities, like this:

class Database {
    func store<T: DatabaseEntity>(_ entity: T) throws {
        ...
    }

    func loadEntity<T: DatabaseEntity>(withID id: T.ID) throws -> T {
        ...
    }
}

Another one of our classes, FriendManager — which is responsible for managing the currently logged in user’s friends — then uses our Database to store and load Friend models:

class FriendManager {
    private let database: Database
    private let dateProvider: () -> Date

    // Our dependencies get injected as part of the initializer
    init(database: Database,
         dateProvider: @escaping () -> Date = Date.init) {
        self.database = database
        self.dateProvider = dateProvider
    }

    func becomeFriends(with user: User) throws {
        // When a friend was added, it gets stored in the database
        let date = dateProvider()
        let friend = Friend(user: user, friendshipDate: date)
        try database.store(friend)
    }

    func isFriends(with user: User) -> Bool {
        // When checking whether the current user is friends with
        // another user, we attempt to load a corresponding
        // entity from our database.
        let id = Friend.ID.make(for: user)
        let friend = try? database.loadEntity(withID: id) as Friend
        return friend != nil
    }
}

Looking at the above implementation, it might seem like we’ve done everything right — we use dependency injection (including passing Date.init as a first class function acting as our dateProvider), we’re not using singletons or any other global state — so we should be ready to start writing unit tests for this class, right?

Well, if we wanted to write a pure unit test, we’d need a way to mock all of our dependencies. In the case of dateProvider, that’s easy — since we can just inject a function that always returns the same date, but in the case of our database, we’ll need to introduce some form of abstraction to make it mockable.

One way to do that would be to have FriendManager depend on a database protocol rather than on the concrete Database type. To make that happen, let’s use the technique from “Separation of concerns using protocols in Swift“, and split our database’s public API into two protocols — which we then combine into one using a typealias:

protocol WritableDatabase {
    func store<T: DatabaseEntity>(_ entity: T) throws
}

protocol ReadableDatabase {
    func loadEntity<T: DatabaseEntity>(withID id: T.ID) throws -> T
}

typealias DatabaseProtocol = WritableDatabase & ReadableDatabase

With the above in place, we can make Database conform to DatabaseProtocol by simply declaring conformance to it in an extension (since our database class already implements all of our above two protocols’ requirements):

extension Database: DatabaseProtocol {}

If we then update our FriendManager class to accept any instance conforming to DatabaseProtocol, instead of using the concrete Database implementation, then we’re ready to start mocking and unit testing.

For example, here’s how we might write a unit test that uses a DatabaseMock to verify that adding a friend works as expected:

class FriendManagerTests: XCTestCase {
    func testAddingFriend() throws {
        let database = DatabaseMock()
        let manager = FriendManager(database: database)
        let user = User(id: 7, name: "John")

        // Verify our initial state so we don't end up with false
        // positives in case the value was already in the database.
        XCTAssertFalse(manager.isFriends(with: user))

        // Perform our action, and use the FriendManager's own
        // API to verify that it returns the right value.
        try manager.becomeFriends(with: user)
        XCTAssertTrue(manager.isFriends(with: user))
    }
}

For more information about mocking — check out “Mocking in Swift“.

One key thing above, is that we’re using our friend manager’s own APIs to both perform our test action and to verify the outcome of that action. While we could’ve also used our mock to perform verifications, letting mocks play as much of an idle role as possible is usually a good way to avoid that our mocking code grows too complex, or that we end up relying too much on it (rather than using the actual APIs we want to test).

It’s integration time!

Now let’s verify the same functionality as above, but using an integration test instead. This time around, rather than mocking Database, we’ll instead test how our real database integrates with our friend manager. While that’ll require a bit more setup in our test, it’ll let us verify how our FriendManager behaves under more real conditions.

The first thing we’ll do, is to make sure that we’re using a database that doesn’t contain any previous data (that could generate false positives, and cause flakiness depending on what entries that were already in the database when the test started). To make that happen, we’re going to borrow the makeTemporaryFilePathForTest function from “Mock-free unit tests in Swift”, which’ll let us create a clean underlying file for our database each time a test is run:

func makeTemporaryFilePathForTest(
    named testName: StaticString = #function,
    suffix: String = ""
) -> String {
    let path = NSTemporaryDirectory() + "\(testName)" + suffix
    try? FileManager.default.removeItem(atPath: path)
    return path
}

Using the above function, we’ll now be able to create a predictable instance of Database, that we can then inject into FriendManager — and write our integration test based on those two objects:

class FriendManagerIntegrationTests: XCTestCase {
    func testAddingFriendToDatabase() throws {
        // Setting up our real database using a temporary file.
        let databasePath = makeTemporaryFilePathForTest(suffix: ".db")
        let database = Database(filePath: databasePath)

        // Using the above database, plus a date provider function
        // that always returns the same date, we can now create
        // an instance of our manager.
        let date = Date()
        let manager = FriendManager(database: database, dateProvider: { date })

        // Creating instances of the models we'll use for verification
        let user = User(id: 7, name: "John")
        let friend = Friend(user: user, friendshipDate: date)

        // Just like in our unit test, we want to verify that
        // our initial state is what we expect — just to make
        // sure that our database is indeed empty.
        XCTAssertNil(try? database.loadEntity(withID: friend.id) as Friend)

        // Performing our test action
        try manager.becomeFriends(with: user)

        // Verifying the outcome of the above action
        let loadedFriend = try database.loadEntity(withID: friend.id) as Friend
        XCTAssertEqual(loadedFriend, friend)
    }
}

Besides the fact that we’re now using our real database, rather than a mocked one, there’s another key difference between the way our integration test is set up compared to our unit test from before. Instead of using our friend manager’s isFriends(with:) API to verify the outcome of our test action, we instead use the underlying database — and assert that the correct entry was added.

The reason for that comes down to what we’re actually testing above — which isn’t that FriendManager returns the correct value for a given API call (that’s what our initial unit test was for), but instead that FriendManager correctly uses its integration with our Database class.

Not necessarily mock-free

While our above integration test involved two objects, without the use of any mocks, that’s not necessarily the case for all integration tests. More than two objects can be involved in the integration that’s being tested, and certain other layers may need to be mocked in order for the test to be practical.

Let’s take a look at a second integration test. This time we want to test the integration between three different objects — a UserLoader (which will be our main test subject), as well as a Cache and an APIClient, that our user loader will use to perform its work. Again, our goal is to use as few mocks as possible, and neither of the objects that are being integration tested will be mocked — but we do want to mock API client’s underlying networking.

The reason for that is both that we don’t want to require Internet access in order for our test to work, and also that relying on real network calls will make our test much slower, while possibly introducing unpredictability as well.

Here’s what that second integration test looks like:

class UserLoaderIntegrationTests: XCTestCase {
    func testCachingLoadedUser() throws {
        let user = User(id: 7, name: "John")
        let data = try JSONEncoder().encode(user)
        let cacheKey = "\(user.id)"

        // Here we use a mocked version of our Networking API,
        // while using real objects for our API client, cache
        // and user loader.
        let networking = NetworkingMock(result: .success(data))
        let apiClient = APIClient(networking: networking)
        let cache = Cache<User>()
        let loader = UserLoader(apiClient: apiClient, cache: cache)

        // Again verifying that our initial state is what we
        // expect it to be.
        XCTAssertNil(cache.value(forKey: cacheKey))

        var result: Result<User, UserLoader.Error>?

        loader.loadUser(withID: user.id) {
            result = $0
        }

        // Verifying the test's outcome by inspecting the cache
        // and actual loaded result, rather than relying on mocks.
        let loadedUser = try result?.get()
        let cachedUser = cache.value(forKey: cacheKey)

        XCTAssertEqual(loadedUser, user)
        XCTAssertEqual(cachedUser, user)
    }
}

For more information about testing the above kind of asynchronous code, check out “Unit testing asynchronous Swift code”.

While there’s no strict rule as to exactly how many objects that can partake in an integration test, in order to keep tests focused and easier to debug, it’s usually a good idea to try to keep that number as small as possible. The larger the number of objects, the harder it often becomes to pinpoint exactly what went wrong in case a test starts failing.

Conclusion

Both unit tests and integration tests, along with other kinds of automated testing (such as UI tests), can be incredibly valuable — in order to detect regressions, verify bug fixes, or to document the intent of a given algorithm or system. While the overwhelming majority of most code bases’ tests will probably still be unit tests, adding a few integration tests to verify the relationships between key objects can often go a long way to improving an app’s stability.

As for the exact difference between a unit test and integration test — sometimes it can be tricky to draw the line — especially if unit tests are written in a more mock-free manner, and different people will likely provide different definitions as to what an integration test is. Here’s my take:

A unit test tests a single unit. During a unit test, all external dependencies and values are either mocked, stubbed, or abstracted using functions — and the object’s own APIs (or mocks) are used for verification.

An integration test tests the integration between multiple code units (such as different functions and types). No test subject should be mocked, even though underlying dependencies that are not being tested may be mocked. The subject which is at the bottom of the stack will be used for verification.

But at the end of the day, what exactly we call our tests isn’t the most important — it’s how they, as tools, can help us build better software.

Questions, feedback or comments? Contact me, or reach out on Twitter @johnsundell.

Thanks for reading! 🚀