Lightning Read #1: Lazy Collections in Swift

I was planning to write “short” and simple articles about my findings from day-to-day Swift experiments for a while now… 🕵

Here goes the first lightning article! 🚀

Problem: Intermediate clutter that comes with map, flatMap and filter functions

Let’s say we have the following requirements in an imaginary project:

There are 6 frames for an animation in format animation_<index>.jpeg available to download.

Download animation frames using Request objects.
Discard every second frame for better performance.

In a real world, we would most probably have more frames but think of this as a simplified example.

Instead of using a for loop for these 6 frames, we will go functional and use map, flatMap and filter methods.

let frameIndexes: [Int] = Array(1...6)
let imageRequests: [Request] = frameIndexes
  // Optimize by filtering every second frame:
  .filter { $0 % 2 == 1 }
  // Transform frame number to image name:
  .map { "animation_\($0).jpeg" }
  // Transform image names to URLs:
  .flatMap { URL(string: "https://www.somehost.com/\($0)") }
  // Transform URLs to request objects:
  .map { Request(url: $0) }

This is much more readable than having a for-loop. But how about performance? Let’s dig into details by going over this code block line by line.

filter block runs 6 times for each frame index in the array.
A new array gets created with 3 filtered frame indexes.
map block runs 3 times for each item.
A new array gets created with 3 image names.
flatMap block runs 3 times for each item.
A new array gets created with 3 URLs. (Assuming URL creation will be successful on each image name.)
map block runs 3 times on each URL.
A new array gets created with 3 request objects.

Call order will look like this:

filter
filter
filter
filter
filter
filter
// A new array is created.
mapToName
mapToName
mapToName
// A new array is created.
flatMapToURL
flatMapToURL
flatMapToURL
// A new array is created.
mapToRequest
mapToRequest
mapToRequest
// A new array is created.

Although we only need the resulting array at the end, we created 3 more intermediate arrays, which has absolutely no use. This happened because filter, map and flatMap are eager functions that calculate immediately.

It’s not a problem for a small data set, but it can be problematic when dealing with a huge data set.

Solution: Using `LazyCollection`

Using a lazy collection, we can get rid of intermediate array creations in between. There is a lazy property defined on RandomAccessCollection which Array conforms to. lazy basically provides a lazy implementation of the same array, using LazyRandomAccessCollection<T> type, which only calculates values when in need.

Let’s implement the lazy approach now.

let frameIndexes: [Int] = Array(1...6)
let lazyImageRequests = frameIndexes
  .lazy
  .filter { $0 % 2 == 1 }
  .map { "animation_\($0).jpeg" }
  .flatMap { URL(string: "https://www.somehost.com/\($0)") }
  .map { Request(url: $0) }

Easy, right? We keep the same declarative way of transforming values from one to another, but achieved good performance just by adding .lazy after frameIndexes.

It will perform better because nothing will be executed unless we access an element. We have 2 options at this point:

We can use the lazy collection just like normal array and calculate each value as we access them.
We can go over the elements of lazy collection one by one and store them in a normal array.

We’ll go with the second option to inspect the call order.

var imageRequests: [Request] = []
for request in lazyImageRequests {
    imageRequests.append(request)
}
// We could also use:
// lazyImageRequests.forEach { imageRequests.append($0) }

After this for loop, call order would be:

// Item 0:
filter
// Item 1:
filter
mapToName
flatMapToURL
mapToRequest
// Item 2:
filter
// Item 3:
filter
mapToName
flatMapToURL
mapToRequest
// Item 4:
filter
// Item 5:
filter
mapToName
flatMapToURL
mapToRequest

Bottom Line

If we don’t use lazy;

We iterate over the original array 4 times.
We create 3 redundant intermediate arrays.

If we do use lazy;

We iterate only once.
We get each element calculated through the pipe as we access them.

Note on Caching

Beware that if we go over lazyImageRequests one more time, each item will be calculated again. So if you plan to access these items more than once, it’s recommended to cache them in a traditional array, just like we did in the example above.

Note on Array Conversion

We can also use the Array initializer to convert a lazy array to a normal array as below.

let imageRequests: [Request] = Array(lazyImageRequests)

I am not sure about the internals of lazy collections, but the code above doesn’t print the same call order as the for loop, for some reason. So keep in mind that using Array initializer for this purpose may not be as performant as expected.

If you’d like to try it out, implementation is available in LazyCollections repo.

Thanks for dropping by! And, as usual, help spread the word. ❤️👏

Image Credit: Photo by Max Bender on Unsplash

Problem: Intermediate clutter that comes with map, flatMap and filter functions

Solution: Using LazyCollection

Bottom Line

Note on Caching

Note on Array Conversion

Solution: Using `LazyCollection`