Incremental lift quantifies the impact of actions (e.g., app installations, web conversions) users take after viewing an advertisement that are above and beyond what would have naturally occurred in the absence of an ad.
Why Measure Incrementality
Incrementality correlates with attribution, but the differences provide insight into the effectiveness of your marketing campaigns and whether an ad truly influenced conversions.
Overview
Our proposed methodology estimates the incremental events associated with your media campaign. We calculate incrementality by taking your attributed event count and subtracting out an estimated organic baseline event count.
Incremental Lift = Attributed Events − Modeled Organic Baseline
Our definition has the additional benefit of bounding the incremental lift count estimate between 0% and 100% of your attributed event count. This allows for easy comparison of ad effectiveness across campaigns and creatives.
Attributed Events
Attributed events are calculated using the same attribution rules you set up in Atlas. This creates alignment between your attribution rules and the estimated incrementality.
Modeled Organic Baseline
The modeled baseline event count is estimated using a machine learning model. By taking the organic event history of devices prior to exposure, we can model the counterfactual of what would have hypothetically happened if media was not served.
Specifically, we use a time series regression model to estimate the baseline organic rate. This allows us to incorporate cohort specific idiosyncrasies while also accounting for macro event trends.
The model estimate is then adjusted for attribution eligibility. This helps account for the device reexposure that drives the time series attribution count downward (as seen in the black time series line in the graph). In other words, attribution volume relative to an exposure today will decrease for two reasons. One, the media loses effectiveness as time from view passes. Two, reexposure to other media makes today’s exposure ineligible for attribution. Our model adjusts for this secondary source of bias.
Model Performance
Model evaluations cannot be directly tested since we are estimating a hypothetical that never occurs. However, we can see how the model performs by looking at devices exposed to a creative for product X while measuring the events of those same devices for product Y. This allows us to control for selective targeting while avoiding any direct impact of media on the likelihood of an event.
In this test scenario, the estimated organic baseline event count should equal the actual attribution event count. When we calculate this difference at the daily level for our model, we find the difference to be +/- 1.42%.
This slight bias is a tradeoff with gathering insights on a daily cadence. When you look at your media over multiple days, this difference disappears.