Articles, podcasts and news about Swift development, by John Sundell.

Building async and concurrent versions of forEach and map

Published on 10 Nov 2021
Discover page available: Concurrency

If we think about it, so much of the code that we write on a daily basis essentially consists of a series of data transformations. We take data in one shape or form — whether that’s actual model data, network responses, or things like user input or other events — we then run our logic on it, and ultimately end up with some form of output data that we then save, send, or display to the user.

In this article, let’s take a look at how we can utilize Swift’s built-in concurrency system when performing such data transformations using functions like forEach and map.

Synchronous transformations

Let’s start by taking a look at an example, in which we’ve built a MovieListViewController that enables the user to mark certain movies as favorites. When that happens, our view controller uses the standard library’s forEach API to iterate over the user-selected index paths, and then calls a FavoritesManager in order to mark the movies corresponding to those index paths as favorites:

class MovieListViewController: UIViewController {
    private var movies: [Movie]
    private let favoritesManager: FavoritesManager
    private lazy var tableView = UITableView()
    ...

    func markSelectedMoviesAsFavorites() {
        tableView.indexPathsForSelectedRows?.forEach { indexPath in
            let movie = movies[indexPath.row]
            favoritesManager.markMovieAsFavorite(movie)
        }
    }
}

Besides enabling movies to be marked as favorites, our FavoritesManager also has a method that lets us load all of the Movie models that the user has previously added to their favorites — which uses the built-in map method to transform a Set of movie IDs into actual models by retrieving them from a Database:

class FavoritesManager {
    private let database: Database
    private var favoriteIDs = Set<Movie.ID>()
    ...
    
    func loadFavorites() throws -> [Movie] {
        try favoriteIDs.map { id in
            try database.loadMovie(withID: id)
        }
    }

    func markMovieAsFavorite(_ movie: Movie) {
        ...
    }
}

So far so good. But now let’s take a look at what would happen if we wanted to start using Swift’s new concurrency system to perform the above operations in an asynchronous, non-blocking manner.

Going asynchronous

To get started, let’s say that we wanted to make our FavoritesManager and Database types perform their work asynchronously, for example like this:

class FavoritesManager {
    ...
    
    func markMovieAsFavorite(_ movie: Movie) async {
        ...
    }

    func loadFavorites() async throws -> [Movie] {
        try await favoriteIDs.map { id in
            try await database.loadMovie(withID: id)
        }
    }
}

However, it turns out that the above implementation doesn’t actually compile, since the standard library’s version of map doesn’t (yet) support async closures. Thankfully, that’s something that we can add support for ourselves, thanks to the power of Swift’s extensions.

In Swift, most of the operations that we commonly perform on various collections (including APIs like forEach, map, flatMap, and so on) are implemented using Sequence protocol extensions — which in turn is a protocol that all built-in collections (like Array, Set, and Dictionary) conform to. It’s also the protocol that’s used to drive Swift’s for loops.

So, to build an async version of map, all that we’d have to do is to extend the Sequence protocol ourselves in order to add that new method:

extension Sequence {
    func asyncMap<T>(
        _ transform: (Element) async throws -> T
    ) async rethrows -> [T] {
        var values = [T]()

        for element in self {
            try await values.append(transform(element))
        }

        return values
    }
}

Note how the above method is marked with the rethrows keyword. That tells the Swift compiler to only treat our new method as throwing if the closure passed into it also throws, which gives us the flexibility to throw errors when needed, without requiring us to always call our new method with try, even when its closure doesn’t throw.

With the above in place, we can now go back and replace our loadFavorites method’s use of map with asyncMap, and our new, asynchronous FavoritesManager implementation now successfully compiles and is ready to be used:

class FavoritesManager {
    ...

    func loadFavorites() async throws -> [Movie] {
        try await favoriteIDs.asyncMap { id in
            try await database.loadMovie(withID: id)
        }
    }
}

Next, let’s say that we also wanted to turn the forEach iteration within MovieListViewController asynchronous as well, since that would let our UI remain completely responsive while our FavoritesManager is busy marking the user’s selected movies as favorites.

To do that, we’re also going to need an async version of forEach, which can be implemented in a very similar way to how we built asyncMap:

extension Sequence {
    func asyncForEach(
        _ operation: (Element) async throws -> Void
    ) async rethrows {
        for element in self {
            try await operation(element)
        }
    }
}

Before we start using our new asyncForEach method within our view controller, however, there’s one thing that we’ll need to consider — since our call to mark the user’s selected movies as favorites will now be performed asynchronously, that means that our array of movies could possibly be mutated while our iteration is being performed.

So, to prevent bugs or crashes from happening in that kind of situation, let’s capture our movies array by value when we call asyncForEach, so that we’ll keep operating on the same array throughout our entire iteration:

class MovieListViewController: UIViewController {
    ...

    func markSelectedMoviesAsFavorites() {
        Task {
            await tableView.indexPathsForSelectedRows?.asyncForEach { 
                [movies] indexPath in

                let movie = movies[indexPath.row]
                await favoritesManager.markMovieAsFavorite(movie)
            }
        }
    }
}

To learn more about closure capturing in Swift, check out “Swift’s closure capturing mechanics”.

With those changes in place, we’ve now successfully turned two potentially heavy data transformation operations into asynchronous, non-blocking calls — which in turn will make our UI remain responsive regardless of how long those calls take to complete.

Concurrency

There’s one more thing that we could do to potentially speed up the above operations, though, and that’s to make them execute in parallel, rather than in sequence. Because even though our operations are now asynchronous in relation to other code, they’re actually still completely sequential in terms of how they’re executed — given that both asyncForEach and asyncMap always await the result of one operation before continuing with the next.

So, to enable us to perform a given forEach call in a way that executes all of its operations in parallel, let’s write a new, concurrent version of it that utilizes the built-in TaskGroup API:

extension Sequence {
    func concurrentForEach(
        _ operation: @escaping (Element) async -> Void
    ) async {
        // A task group automatically waits for all of its
        // sub-tasks to complete, while also performing those
        // tasks in parallel:
        await withTaskGroup(of: Void.self) { group in
            for element in self {
                group.addTask {
    await operation(element)
}
            }
        }
    }
}

To learn more about TaskGroup, check out “Using Swift’s concurrency system to run multiple tasks in parallel”.

Now, we’re definitely not going to want to perform all of our forEach calls concurrently (because we have to remember that spreading things out over multiple concurrent tasks and threads also comes with its own overhead and delays). However, in the case of marking movies as favorites within our MovieListViewController, we might be able to achieve a speed gain if the user ends up selecting a large number of movies, which we would now be able to iterate over concurrently.

So let’s replace our previous use of asyncForEach with a call to our new concurrentForEach method — like this:

class MovieListViewController: UIViewController {
    ...

    func markSelectedMoviesAsFavorites() {
        Task {
            await tableView.indexPathsForSelectedRows?.concurrentForEach {
                [movies, favoritesManager] indexPath in

                let movie = movies[indexPath.row]
                await favoritesManager.markMovieAsFavorite(movie)
            }
        }
    }
}

Note how we’re now also capturing a reference to our FavoritesManager within our closure. That’s because we’re now dealing with an escaping closure, which requires us to make all self-related captures explicit. So, rather than capturing self, we instead only capture the actual object that we need to perform our operation.

With the above change in place, our favorite marking logic will now operate on multiple movies in parallel when needed, which could potentially give us a significant speed boost. Of course (like with most things related to performance), whether we want to use asyncForEach or concurrentForEach is something that we’ll have to decide based on real-life performance metrics within each specific situation, as there’s no universal rule as to whether something could benefit from being executed concurrently.

Another thing that’s important to consider before making a given set of operations execute concurrently is whether our underlying data supports concurrent access. We’ll take a much closer look at that topic — including how we can use Swift’s actor types to enforce serial access to mutable state — in upcoming articles.

Moving on to the mapping operation within our FavoritesManager, let’s now take a look at how we could also enable that iteration to be performed concurrently as well. That’ll require a new, concurrent version of map, which we’ll call concurrentMap.

However, unlike concurrentForEach, this new method won’t use a TaskGroup, since that API doesn’t give us any guarantees in terms of completion order, meaning that we’d potentially end up with an output array that has a different order than our input. So, instead, we’ll start by creating an async Task for each element, and we’ll then use our previous asyncMap method to await the results of each of those tasks in-order:

extension Sequence {
    func concurrentMap<T>(
        _ transform: @escaping (Element) async throws -> T
    ) async throws -> [T] {
        let tasks = map { element in
            Task {
    try await transform(element)
}
        }

        return try await tasks.asyncMap { task in
            try await task.value
        }
    }
}

An alternative approach to the above would be to use a TaskGroup in combination with a dictionary that’d keep track of our output order. However, not only would that code be more complicated, it also wouldn’t give us any significant performance gains over our current implementation.

Using our new concurrentMap method, we’re now able to have our FavoritesManager load all favorites in a completely concurrent manner — which will make that operation scale really nicely as the user adds more and more movies to their list of favorites:

class FavoritesManager {
    ...

    func loadFavorites() async throws -> [Movie] {
        try await favoriteIDs.concurrentMap { [database] id in
            try await database.loadMovie(withID: id)
        }
    }
}

With that last piece in place, we’ve now not only given our code an async makeover, but we’ve also made it execute concurrently as well — and in the process we’ve built four completely reusable Sequence APIs — asyncForEach, concurrentForEach, asyncMap, and concurrentMap.

Conclusion

I hope that this article has given you a few new insights into how certain Swift iterations can be turned into async or concurrent operations using Swift’s new concurrency system. Again, it’s always important to consider when (and when not) to deploy concurrency, and to try to base those decisions on real-life performance metrics. After all, completely synchronous code will likely always remain the simplest to implement, debug, and maintain.

If you’re interested in using the new forEach and map variants introduced in this article, as well as similar versions of compactMap and flatMap — then check out CollectionConcurrencyKit, a new open source Swift package that adds those APIs to all Swift collections. And if you have any questions, comments, or feedback, then feel free to reach out via email.

Thanks for reading!