Holdout test
Proof of what advertising really achieves
A holdout test measures the true impact of a campaign. A randomly selected control group does not see the ads. The difference in purchases or sign ups between both groups is the effect the campaign really caused.
How we use it
For us, holdout tests are the third level of measurement: after contribution and comparison comes proof. They show which part of the conversions would have happened anyway without the campaign.
Example
90 % of users see the retargeting campaign, 10 % form the control group. If the test group buys at 4 % and the control group at 3 %, the campaign caused one percentage point of additional purchases.
Related terms: Incrementality, attribution, cost per order. To the glossary
