Testing Analytics Events

Testing Analytics Events

Since we make lots of important decisions by looking at the analytics data (if not, you should), I will not go into "why" having accurate analytics data is extremely important.

You probably also know that it is extremely painful to manually test if analytics events are being reported correctly. It's very time consuming to perform some actions in the app, wait for the analytics tool to receive these events and verify each. Now, multiply this with number of releases you'll have in the future... It's a nightmare. 😩

This time could be better spent elsewhere.

Any time you have this feeling, there's a good chance that things can be automated. Can we automate analytics testing? Why not.

Could we write black-box UI tests? With UI testing, we could actually record the UI actions leading to a specific analytics event. These UI actions can be repeated whenever we want to generate analytics events on the portal. But the verification part would still be manual on the analytics tool we're using. This would still require manual verification so it is probably not what we want.

Could we write unit tests? Simulate UI actions that would lead to an analytics event, check if the mock analytics layer received the event and verify the details... This could work! Let's try it out.

Talk is cheap. Show me the code.

Thanks Reddit 😂

Let's say we have a home screen that shows movie posters in a grid and want to track when the users tap on these posters. This event will have the following attributes:

  • Event Name: movie_tapped
  • Movie ID (e.g 67563)
  • Movie Slug (e.g 67563-daredevil)
  • Movie Index (e.g 6)

If we model this event, it would look something like the following:

struct AnalyticsEvent {
  let name: String
  let payload: [String: Any]
}
 
extension AnalyticsEvent {

  static func movieTapped(id: Int, slug: String, index: Int) -> AnalyticsEvent {
    return AnalyticsEvent(
      name: "movie_tapped",
      payload: ["movie_id": id, "movie_slug": slug, "movie_index": index]
    )
  }
}

OK, we modeled the event. Now, how do we send it? That kind of depends on the analytics tool we're using. Here, some important things to note:

  • Each analytics API can be slightly different, but at their core, they will all be able to process our simple AnalyticsEvent structure and have some kind of method to record screen views.
  • There is no need to constrain ourselves to a single analytics API as we might decide to use some other tool later on (or even use multiple analytics tools at the same time). So it's better to abstract away from any concrete implementation and access them through a protocol.
  • If we want to able to create a test environment, it needs to be isolated from external factors and be deterministic. If we use any analytics API directly, we won't be able to mock it for testing and therefore cannot verify any events.

This calls for an abstraction layer.

protocol AnalyticsProtocol {
  func screenView(_ screen: String)
  func send(_ event: AnalyticsEvent)
}

At the moment, we have all the tools to create an analytics event and send it to any analytics tool.

Onto the presentation layer... I am going to assume we're using MVP or MVVM here. We will have a view, which looks like this:

final class HomeViewController: UIViewController {

  let presenter: HomePresenter
  
  override func viewDidLoad() {
    super.viewDidLoad()
    presenter.start()
  }
  
  // ...
}

extension HomeViewController: UICollectionViewDelegate {

  func collectionView(_ collectionView: UICollectionView, didSelectItemAt indexPath: IndexPath) {
    // ...
    presenter.selectMovie(at: indexPath.item)
  }
}

We already know that the presenter needs to have a method to understand if a movie is tapped, send movie_tapped analytics event, and then show movie details. Let's model this.

final class HomeViewPresenter {

  private weak var view: HomeViewProtocol?
  private let movieService: MovieServiceProtocol
  private let router: HomeRouterProtocol
  private let analytics: AnalyticsProtocol
  
  init(view: HomeViewProtocol?,
       movieService: MovieServiceProtocol, 
       router: HomeRouterProtocol,
       analytics: AnalyticsProtocol) {
    self.view = view
    self.movieService = movieService
    self.router = router
    self.analytics = analytics
  }
  
  func start() {
    // fetch movies and notify view here...
  }
  
  func selectMovie(at index: Int) {
    let movie = movies[index]
    analytics.send(.movieTapped(id: movie.id, slug: movie.slug, index: index))
    router.showMovieDetails(movie)
  }
}

If we inject the correct implementation of our analytics tool into this presenter, all should work perfectly.

How about tests? If we inject a mock analytics implementation instead of a real one, we could actually capture these events and run assertions. Let's create this mock analytics module.

final class MockAnalytics: AnalyticsProtocol {

  private(set) var screenViews: [String] = []
  private(set) var events: [AnalyticsEvent] = []
  
  func screenView(_ screen: String) {
    screenViews.append(screen)
  }
  
  func send(_ event: AnalyticsEvent) {
    events.append(event)
  }
}

Now, we use this mock analytics module while testing the home screen analytics.

final class HomeViewTests {

  private var analytics: MockAnalytics!
  private var router: MockHomeRouter!
  private var presenter: HomeViewPresenter!
  
  override func setupWithError() throws {
    self.analytics = MockAnalytics()
    self.router = MockHomeRouter() // exercise for the reader
    self.presenter = HomeViewPresenter(
      view: nil,
      movieService: MockMovieService(file: "home-movies.json"), // exercise for the reader
      router: router,
      analytics: analytics
    )
  }
  
  // tests will go here...
}    

This is the boring setup part needed to create a controlled test environment. Now we can run assertions.

func test_analytics_movieTapped() throws {
  // Given the user is on home screen
  presenter.start()
  
  // When the user taps on movie at index 6
  presenter.selectMovie(at: 6)
  
  // Then `movie_tapped` analytics event should be sent
  //   | key         | value           |
  //   | movie_id    | 67563           |
  //   | movie_slug  | 67563-daredevil |
  //   | movie_index | 6               |
  let event = try XCTUnwrap(analytics.events.first)
  XCTAssertEqual(event.name, "movie_tapped")
  XCTAssertEqual(event.payload["movie_id"] as? Int, 67563)
  XCTAssertEqual(event.payload["movie_slug"] as? String, "67563-daredevil")
  XCTAssertEqual(event.payload["movie_index"] as? Int, 6)
}   

Those assertions look a little ugly though, right? It's easy to improve readability using some helper methods. We can create an extension on AnalyticsEvent and write custom assert functions.

extension AnalyticsEvent {

  func assertName(_ name: String, file: StaticString = #file, line: Int = #line) {
    XCTAssertEqual(self.name, name, file: file, line: line)
  }
  
  func assertPayload<T: Equatable>(key: String, equalsTo value: T, file: StaticString = #file, line: Int = #line) {
    XCTAssertEqual(self.payload[key] as? T, value, file: file, line: line)
  }
}

Then use them as follows:

let event = try XCTUnwrap(analytics.events.first)
event.assertName("movie_tapped")
event.assertPayload(key: "movie_id", equalsTo: 67563)
event.assertPayload(key: "movie_slug", equalsTo: "67563-daredevil")
event.assertPayload(key: "movie_index", equalsTo: 6)

We were able to deliver what we promised at the start. We now have automated tests for analytics, which are fast and repeatable. 🎉

Does this proof that everything will work perfectly? Not quite.

  • We still need to make sure presenter.selectMovie(at: ...) method is called by the view.
  • We still need to make sure the analytics SDK is happy with the payload. If the SDK doesn't support a type, which exists in the payload, it may fail in various ways.

But it is good enough! When this test is combined with another set of unit tests and a manual test (to see if it's visible on the portal), we can be pretty sure that it's all fine.

Thanks for reading so far! Hope this was helpful. 👨‍🔧

You can reach out to me on Twitter for any comments and questions. 🐦


Image Credit: Photo by Lukas Blazek on Unsplash