Articles, podcasts and news about Swift development, by John Sundell.

Ignoring invalid JSON elements when using Codable

Published on 22 Feb 2021
Discover page available: Codable

By default, encoding or decoding an array using Swift’s built-in Codable API is an all-or-nothing type of deal. Either all elements will be successfully handled, or an error will be thrown, which is arguably a good default, since it ensures a high level of data consistency.

However, sometimes we might want to tweak that behavior so that invalid elements will be ignored, rather than resulting in our entire coding process failing. For example, let’s say that we’re working with a JSON-based web API that returns collections of items that we’re currently modeling in Swift like this:

struct Item: Codable {
    var name: String
    var value: Int
}

extension Item {
    struct Collection: Codable {
        var items: [Item]
    }
}

Now let’s say that the web API that we’re working with occasionally returns responses such as the following, which includes a null value where our Swift code is expecting an Int:

{
    "items": [
        {
            "name": "One",
            "value": 1
        },
        {
            "name": "Two",
            "value": 2
        },
        {
            "name": "Three",
            "value": null
        }
    ]
}

If we try to decode the above response into an instance of our Item.Collection model, then the entire decoding process will fail, even if the majority of our items did contain perfectly valid data.

The above might seem like a somewhat contrived example, but it’s incredibly common to encounter malformed or inconsistent JSON formats in the wild, and we might not always be able to adjust those formats to neatly fit Swift’s very static nature.

Of course, one potential solution would be to simply make our value property optional (Int?), but doing so could introduce all sorts of complexities across our code base, since we’d now have to unwrap those values every time that we wish to use them as concrete, non-optional Int values.

Another way to solve our problem could be to define default values for the properties that we’re expecting to potentially be null, missing, or invalid — which would be a fine solution in situations when we still want to keep any elements that contained invalid data, but let’s say that this is not one of those situations.

So, instead, let’s take a look at how we could ignore all invalid elements when decoding any Decodable array, without having to make any major modifications to how our data is structured within our Swift types.

Building a lossy codable list type

What we’re essentially looking to do is to change our decoding process from being very strict to instead becoming “lossy”. To get started, let’s introduce a generic LossyCodableList type which will act as a thin wrapper around an array of Element values:

struct LossyCodableList<Element> {
    var elements: [Element]
}

Note how we didn’t immediately make our new type conform to Codable, and that’s because we’d like it to conditionally support either Decodable, Encodable, or both, depending on what Element type that it’s being used with. After all, not all types will be codable both ways, and by declaring our Codable conformances separately we’ll make our new LossyCodableList type as flexible as possible.

Let’s start with Decodable, which we’ll conform to by using an intermediate ElementWrapper type to decode each element in an optional manner. We’ll then use compactMap to discard all nil elements, which will give us our final array — like this:

extension LossyCodableList: Decodable where Element: Decodable {
    private struct ElementWrapper: Decodable {
        var element: Element?

        init(from decoder: Decoder) throws {
            let container = try decoder.singleValueContainer()
            element = try? container.decode(Element.self)
        }
    }

    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        let wrappers = try container.decode([ElementWrapper].self)
        elements = wrappers.compactMap(\.element)
    }
}

To learn more about the above way of conforming to protocols, check out “Conditional conformances in Swift”.

Next, Encodable, which might not be something that every project needs, but it could still come in handy in case we also want to give our encoding process the same lossy behavior:

extension LossyCodableList: Encodable where Element: Encodable {
    func encode(to encoder: Encoder) throws {
        var container = encoder.unkeyedContainer()

        for element in elements {
            try? container.encode(element)
        }
    }
}

With the above in place, we’ll now be able to automatically discard all invalid Item values simply by making our nested Collection type use our new LossyCodableList — like this:

extension Item {
    struct Collection: Codable {
        var items: LossyCodableList<Item>
    }
}

Making our list type transparent

One quite major downside with the above approach, however, is that we now always have to use items.elements to access our actual item values, which isn’t ideal. It would arguably be much better if we could turn our usage of LossyCodableList into a completely transparent implementation detail, so that we could keep accessing our items property as a simple array of values.

One way to make that happen would be to store our item collection’s LossyCodableList as a private property, and to then use a CodingKeys type to point to that property when encoding or decoding. We could then implement items as a computed property, for example like this:

extension Item {
    struct Collection: Codable {
        enum CodingKeys: String, CodingKey {
            case _items = "items"
        }

        var items: [Item] {
            get { _items.elements }
            set { _items.elements = newValue } 
        }
        
        private var _items: LossyCodableList<Item>
    }
}

Another option would be to give our Collection type a completely custom Decodable implementation, which would involve decoding each JSON array using LossyCodableList before assigning the resulting elements to our items property:

extension Item {
    struct Collection: Codable {
        enum CodingKeys: String, CodingKey {
            case items
        }

        var items: [Item]

        init(from decoder: Decoder) throws {
            let container = try decoder.container(keyedBy: CodingKeys.self)
            let collection = try container.decode(
                LossyCodableList<Item>.self,
                forKey: .items
            )
            
            items = collection.elements
        }
    }
}

Both of the above approaches are perfectly fine solutions, but let’s see if we could make things even nicer by using Swift’s property wrappers feature.

Both a type and a property wrapper

One really neat thing about the way that property wrappers are implemented in Swift is that they’re all just standard Swift types, which means that we can retrofit our LossyCodableList to also be able to act as a property wrapper.

All that we have to do to make that happen is to mark it with the @propertyWrapper attribute, and to implement the required wrappedValue property (which once again could be done as a computed property):

@propertyWrapper
struct LossyCodableList<Element> {
    var elements: [Element]

    var wrappedValue: [Element] {
    get { elements }
    set { elements = newValue }
}
}

With the above in place, we’ll now be able to mark any Array-based property with the @LossyCodableList attribute, and it’ll be lossily encoded and decoded — comparably transparently:

extension Item {
    struct Collection: Codable {
        @LossyCodableList var items: [Item]
    }
}

Of course, we’ll still be able to keep using LossyCodableList as a stand-alone type, just like we did before. All that we’ve done by turning it into a property wrapper is to also enable it to be used that way, which once again gives us a lot of added flexibility without any real cost.

Conclusion

At first glance, Codable might seem like an incredibly strict and somewhat limited API that either succeeds or fails, without any room for nuance or customization. However, once we get past the surface level, Codable is actually incredibly powerful, and can be customized in lots of different ways.

Silently ignoring invalid elements is definitely not always the right approach — very often we do want our coding processes to fail whenever any invalid data was encountered — but when that’s not the case, then either of the techniques used in this article can provide a great way to make our coding code more flexible and lossy without introducing a ton of additional complexity.

If you’ve got any questions, comments, or feedback, then feel free to reach out via either Twitter or email. If you want to, you can also support my work by sharing this article to help more people discover it.

Thanks for reading!