Geo experiment

A geo experiment assigns treatment, most often a marketing campaign, at the level of a geographic region, a city, a designated market area, a state, rather than to individual users, then compares outcomes in treated versus untreated regions. It is a practical alternative when individual-level randomization is impossible or unreliable, such as for TV, out-of-home, or broad digital campaigns that can't be targeted to specific people, or where measuring incrementality via ad-platform data is compromised by tracking limitations.

Regions are typically matched or randomized in pairs to balance pre-existing differences in size, demographics, and baseline sales, and the effect is estimated with methods built for this setting, commonly difference-in-differences or synthetic control, comparing each treated region's outcome against untreated or synthetic comparison regions over the same period.

Geo experiments are a core method behind conversion lift studies and broader incrementality testing for marketing spend, because they measure a channel's actual incremental effect on sales rather than relying on attribution models that can overstate a channel's contribution. Their main limitation is that markets are a scarce resource, there are only so many independent regions to randomize, which caps statistical power compared to a user-level test with millions of individual units.