No AppStoreConnect Bug (iOS 13) Is Going to Stop Us

ASO Industry News

Matan Offer 16 February 2020

A bug that’s been sowing havoc among the ASO community and mobile marketing agencies for the past six months has been fixed. 

The bug caused some aspects of app analytics on App Store Connect to be distorted. Specifically, compromised Search Impressions made it quite difficult for ASO professionals to measure app performance and the effects of keyword and metadata changes.  

In this article we want to share how with a little out-of-the-box thinking we were able to find a way to deal with unreliable data and work alongside the bug. More than that, we consider this an opportunity to demonstrate what we believe to be good ASO practice. But more on this later. 

Early Warning Signs

It all started about six months ago, when Apple released the iOS 13 update. The first thing to catch our eye was a drop in search impressions for several of our app store customers all at once and for no apparent reason. We immediately checked the conversion rate for those apps, which we were unaffected. With a raised eyebrow we performed an audit of our other clients, and sure enough the same data warp was showing up for all of them. All apps were showing the same search impressions irregularity while the CVR for some of them was the same or growing.

Here’s what it looked like:

The sudden drop in search impressions across many apps raised our suspicions about a system error and sent us digging for answers.   

The chatter on the Apple Developer Forums revealed the scope of the problem. It turned out that many ASO developers were experiencing the same issue – a dramatic drop in search impressions. The reports also indicated that keyword positions remained stable and that App Units and Product Page Views metrics were not affected by the bug. This explained why the CVR was way past the 100% mark in some cases – a heartwarming data as much as it is false.  

App Store Connect Data Mismatch

While others were busy reporting the now obvious bug to Apple, we turned to contain the problem by changing tactics. 

Let’s back up here for a moment and recall some basic App Store concepts. As you know, the App Store has five traffic sources: Browse, Search, App Referrer, Web Referrer and Unavailable. Zooming in on two of them, Search Impressions refers to the number of users who have viewed an app via App Store Search, while Browse counts the number of users who saw the app while browsing the Top Charts or Categories sections.

To measure the impact of ASO work on said traffic, App Store Connect lays three reported metrics at our disposal: Impressions, App Units and Product Page Views. All three metrics are important for tracking discoverability and conversion. Impressions, especially, is a crucial discoverability measure for Search, since most users tend to skip viewing the Product Page and just download the app directly from Search. 

Here’s how Apple defines each of these metrics:

Search Impressions, then, are crucial for gauging organic performance and the success of keyword work. We use it, for example, to track keyword visibility and app rankings following a metadata update. If an app’s screenshots, title or other creative elements have changed, we can monitor the impact of that change using any one or preferably the combination of the three metrics. 

With the bug, we were essentially dealing with a data mismatch. The wrong metrics reporting the wrong traffic source. What it did was wrongly attribute Search Impressions to Browse. Meanwhile, Impressions from App Referrer, Web Referrer and App Units seemed to have been spared. This translated to a declining Search Impressions graph and an impossible CVR of <100% in some cases. This made it harder to track a major part of any ASO work. 

One Down, Two to Go

Since we’re the glass half full kind of company, we got to work with what we had left. Yes, we couldn’t rely on Search Impressions any more, but we still had two other metrics to go on. Back when talks of the bug were mere whispers, we checked App Units and Product Page Views and found they were still reporting the correct attributes. It wasn’t much, but it was all we needed to keep business as usual. 

In the end, Apple reported the problem fixed which means ASOers can now get back to monitoring keywords and metadata updates. Not to spoil the mood, but it’s important to note that we are yet to see data that seems 100% reliable, which means the fix is probably being gradually released across all apps and locales.

And now for the moral of the story. Bugs may come and bugs may go, but that doesn’t mean you can’t find a way to work around them. In this case, we relied on Apple Search Ads (ASA) data to get Impressions volume per word and draw the relevant conclusions for organic results. It wasn’t a perfect solution, but it was enough to give us a good idea of an app’s organic Store exposure, especially for those clients who had enough budget for a broad campaign.  

Matan Offer

Head of ASO. I am passionate about everything that's organic: Vegetables, crackers and app store searches. Performance optimization on App Store & Google Play are my playground. AB Testing and keyword researches are my toys.

Sign up to our newsletter

Email(Required)

Δ