Conversion lift study

A conversion lift study measures an ad campaign's true incremental effect by randomly withholding ads from a holdout group of otherwise eligible users and comparing their conversion rate to that of users who were shown the ads. The difference between the two groups is the lift the campaign actually caused, as opposed to conversions that would have happened anyway.

This is distinct from last-touch or multi-touch attribution, which assigns credit for conversions that already happened among people who saw an ad, but cannot say how many of those people would have converted regardless. A campaign can show a strong attributed conversion count while producing little or no real lift, if it mostly reached people who were already going to buy, which is exactly the gap a lift study is designed to expose.

Lift studies are typically run directly by ad platforms using their own user pools, since only the platform can withhold ads from a true randomized holdout at the individual level, or approximated at the market level with a geo experiment when individual randomization isn't available. The main pitfalls are a holdout too small to detect a realistic lift given typical conversion rates, and a holdout that leaks, excluded users still seeing the ad through another channel or account, which biases the measured lift toward zero.