IDEAS home Printed from https://ideas.repec.org/a/kap/qmktec/v12y2014i3p235-266.html
   My bibliography  Save this article

Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!

Author

Listed:
  • Randall Lewis
  • David Reiley

Abstract

A randomized experiment with 1.6 million customers measures positive causal effects of online advertising for a major retailer. The advertising profitably increases purchases by 5%. 93% of the increase occurs in brick-and-mortar stores; 78% of the increase derives from consumers who never click the ads. Our large sample reaches the statistical frontier for measuring economically relevant effects. We improve econometric efficiency by supplementing our experimental variation with non-experimental variation caused by consumer browsing behavior. Our experiment provides a specification check for observational difference-in-differences and cross-sectional estimators; the latter exhibits a large negative bias three times the estimated experimental effect. Copyright Springer Science+Business Media New York 2014

Suggested Citation

  • Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    • Handle: RePEc:kap:qmktec:v:12:y:2014:i:3:p:235-266
    DOI: 10.1007/s11129-014-9146-6
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11129-014-9146-6
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11129-014-9146-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stefano DellaVigna & Matthew Gentzkow, 2010. "Persuasion: Empirical Evidence," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 643-669, September.
    2. Randall Lewis & David Reiley, 2014. "Advertising Effectively Influences Older Users: How Field Experiments Can Improve Measurement and Targeting," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 44(2), pages 147-159, March.
    3. repec:feb:artefa:0087 is not listed on IDEAS
    4. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    5. Daniel A. Ackerberg, 2003. "Advertising, learning, and consumer choice in experience good markets: an empirical examination," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 44(3), pages 1007-1040, August.
    6. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    7. Duncan Simester & Yu (Jeffrey) Hu & Erik Brynjolfsson & Eric T. Anderson, 2009. "Dynamics Of Retail Advertising: Evidence From A Field Experiment," Economic Inquiry, Western Economic Association International, vol. 47(3), pages 482-499, July.
    8. Joseph O. Eastlack, Jr. & Ambar G. Rao, 1989. "Advertising Experiments at the Campbell Soup Company," Marketing Science, INFORMS, vol. 8(1), pages 57-71.
    9. Ackerberg, Daniel A, 2001. "Empirically Distinguishing Informative and Prestige Effects of Advertising," RAND Journal of Economics, The RAND Corporation, vol. 32(2), pages 316-333, Summer.
    10. Anindya Ghose & Sha Yang, 2009. "An Empirical Analysis of Search Engine Advertising: Sponsored Search in Electronic Markets," Management Science, INFORMS, vol. 55(10), pages 1605-1622, October.
    11. Leonard M. Lodish & Magid M. Abraham & Jeanne Livelsberger & Beth Lubetkin & Bruce Richardson & Mary Ellen Stevens, 1995. "A Summary of Fifty-Five In-Market Experimental Estimates of the Long-Term Effect of TV Advertising," Marketing Science, INFORMS, vol. 14(3_supplem), pages 133-140.
    12. Anindya Ghose & Sha Yang, 2007. "An Empirical Analysis of Search Engine Advertising: Sponsored Search and Cross-Selling in Electronic Markets," Working Papers 07-35, NET Institute, revised Sep 2007.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Navdeep Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    2. Navdeep S. Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    3. Avi Goldfarb, 2014. "What is Different About Online Advertising?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 44(2), pages 115-129, March.
    4. Chen He & Tobias J. Klein, 2023. "Advertising as a Reminder: Evidence from the Dutch State Lottery," Marketing Science, INFORMS, vol. 42(5), pages 892-909, September.
    5. Stefano DellaVigna & Matthew Gentzkow, 2010. "Persuasion: Empirical Evidence," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 643-669, September.
    6. Li, Xiaolin & Rao, Raghunath Singh & Narasimhan, Om & Gao, Xing, 2022. "Stay positive or go negative? Memory imperfections and messaging strategy," International Journal of Research in Marketing, Elsevier, vol. 39(4), pages 1127-1149.
    7. Randall Lewis & David Reiley, 2011. "Does Retail Advertising Work? Measuring the Effects of Advertising on Sales via a Controlled Experiment on Yahoo!," Working Paper series, University of East Anglia, Centre for Competition Policy (CCP) 2011-09, Centre for Competition Policy, University of East Anglia, Norwich, UK..
    8. Michael R. Baye & Babur De los Santos & Matthijs R. Wildenbeest, 2016. "Search Engine Optimization: What Drives Organic Traffic to Retail Sites?," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 25(1), pages 6-31, March.
    9. Duncan Simester & Yu (Jeffrey) Hu & Erik Brynjolfsson & Eric T. Anderson, 2009. "Dynamics Of Retail Advertising: Evidence From A Field Experiment," Economic Inquiry, Western Economic Association International, vol. 47(3), pages 482-499, July.
    10. Mariia I. Okuneva & Dmitriy B. Potapov, 2015. "The Effectiveness of Individual Targeting Through Smartphone Application in Retail: Evidence from Field Experiment," HSE Working papers WP BRP 47/MAN/2015, National Research University Higher School of Economics.
    11. Lauren H. Cohen & Umit G. Gurun, 2018. "Buying the Verdict," NBER Working Papers 24542, National Bureau of Economic Research, Inc.
    12. Lynne Pepall & Joseph Reiff, 2017. "Targeted Advertising and Cumulative Exposure Effects: The Impact of Banning Advertising to Children in Quebec," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 51(3), pages 235-256, November.
    13. Peitz, Martin & Reisinger, Markus, 2014. "The Economics of Internet Media," Working Papers 14-23, University of Mannheim, Department of Economics.
    14. Joseph Golden & John Joseph Horton, 2021. "The Effects of Search Advertising on Competitors: An Experiment Before a Merger," Management Science, INFORMS, vol. 67(1), pages 342-362, January.
    15. Sridhar Narayanan & Kirthi Kalyanam, 2015. "Position Effects in Search Advertising and their Moderators: A Regression Discontinuity Approach," Marketing Science, INFORMS, vol. 34(3), pages 388-407, May.
    16. Garrett A. Johnson & Randall A. Lewis & David H. Reiley, 2017. "When Less Is More: Data and Power in Advertising Experiments," Marketing Science, INFORMS, vol. 36(1), pages 43-53, January.
    17. Daniel Zantedeschi & Eleanor McDonnell Feit & Eric T. Bradlow, 2017. "Measuring Multichannel Advertising Response," Management Science, INFORMS, vol. 63(8), pages 2706-2728, August.
    18. Stephan Wachtel & Thomas Otter, 2013. "Successive Sample Selection and Its Relevance for Management Decisions," Marketing Science, INFORMS, vol. 32(1), pages 170-185, September.
    19. Jeonghye Choi & David R. Bell & Leonard M. Lodish, 2012. "Traditional and IS-Enabled Customer Acquisition on the Internet," Management Science, INFORMS, vol. 58(4), pages 754-769, April.
    20. Jacob LaRiviere & Mikolaj Czajkowski & Nick Hanley & Katherine Simpson, 2016. "What is the Causal Impact of Knowledge on Preferences in Stated Preference Studies?," Working Papers 2016-12, Faculty of Economic Sciences, University of Warsaw.

    More about this item

    Keywords

    Online advertising; Display advertising; Advertising effectiveness; Field experiment; Difference in differences; Codes: C93 - Field Experiments; M37 - Advertising; D12 - Consumer Economics: Empirical Analysis;
    All these keywords.

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • M37 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Marketing and Advertising - - - Advertising
    • D12 - Microeconomics - - Household Behavior - - - Consumer Economics: Empirical Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:qmktec:v:12:y:2014:i:3:p:235-266. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.