What Cookie-Based Advertising Effectiveness Fails to Measure

Retargeted advertising is a popular form of digital advertising that algorithmically delivers ads to users who previously visited an advertiser’s website. We empirically investigate the challenge of estimating individual-level advertising response when the data stem from a field experiment randomized at the cookie level. Such experiments are common in the industry. We investigate how cookie-level randomization manifests itself at the individual level and the role it plays in measuring retargeted advertising effectiveness. We show that cookie-based analyses to assess retargeted advertising effectiveness are flawed. Cookie randomization breaks down because of the presence of individuals who have both treatment and control cookies—these individuals are more engaged and spend more. Because of the cookie propagation effect we uncover, individual assignment to the treatment and control groups is not random. Individual-level analyses, in conjunction with first-party data, paint a more complete picture of the impact of retargeted advertising. We detect an increase in offline sales that cookie-based analyses fail to capture. Although many marketers are apprehensive about the cookie-less world of the future, we show that cookie-based analyses can be misleading. Individual data, obtained with consent while protecting identity, are better and overcome many of these problems with cookie-based analyses.