IDEAS home Printed from https://ideas.repec.org/p/uea/ueaccp/2011_09.html
   My bibliography  Save this paper

Does Retail Advertising Work? Measuring the Effects of Advertising on Sales via a Controlled Experiment on Yahoo!

Author

Listed:
  • Randall Lewis

    (Yahoo! Research)

  • David Reiley

    (Yahoo! Research)

Abstract

We measure the causal effects of online advertising on sales, using a randomized experiment performed in cooperation between Yahoo! and a major retailer. After identifying over one million customers matched in the databases of the retailer and Yahoo!, we randomly assign them to treatment and control groups. We analyze individual-level data on ad exposure and weekly purchases at this retailer, both online and in stores. We find statistically and economically significant impacts of the advertising on sales. The treatment effect persists for weeks after the end of an advertising campaign, and the total effect on revenues is estimated to be more than seven times the retailer's expenditure on advertising during the study. Additional results explore differences in the number of advertising impressions delivered to each individual, online and offline sales, and the effects of advertising on those who click the ads versus those who merely view them. Power calculations show that, due to the high variance of sales, our large number of observations brings us just to the frontier of being able to measure economically significant effects of advertising. We also demonstrate that without an experiment, using industry-standard methods based on endogenous crosssectional variation in advertising exposure, we would have obtained a wildly inaccurate estimate of advertising effectiveness.

Suggested Citation

  • Randall Lewis & David Reiley, 2011. "Does Retail Advertising Work? Measuring the Effects of Advertising on Sales via a Controlled Experiment on Yahoo!," Working Paper series, University of East Anglia, Centre for Competition Policy (CCP) 2011-09, Centre for Competition Policy, University of East Anglia, Norwich, UK..
    • Handle: RePEc:uea:ueaccp:2011_09
    as

    Download full text from publisher

    File URL: https://ueaeco.github.io/working-papers/papers/ccp/CCP-11-09.pdf
    File Function: main text
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    2. Daniel A. Ackerberg, 2003. "Advertising, learning, and consumer choice in experience good markets: an empirical examination," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 44(3), pages 1007-1040, August.
    3. Ackerberg, Daniel A, 2001. "Empirically Distinguishing Informative and Prestige Effects of Advertising," RAND Journal of Economics, The RAND Corporation, vol. 32(2), pages 316-333, Summer.
    4. repec:feb:artefa:0087 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brett R. Gordon & Wesley R. Hartmann, 2013. "Advertising Effects in Presidential Elections," Marketing Science, INFORMS, vol. 32(1), pages 19-35, June.
    2. L. Elisa Celis & Gregory Lewis & Markus Mobius & Hamid Nazerzadeh, 2014. "Buy-It-Now or Take-a-Chance: Price Discrimination Through Randomized Auctions," Management Science, INFORMS, vol. 60(12), pages 2927-2948, December.
    3. Sami Najafi-Asadolahi & Kristin Fridgeirsdottir, 2014. "Cost-per-Click Pricing for Display Advertising," Manufacturing & Service Operations Management, INFORMS, vol. 16(4), pages 482-497, October.
    4. Robert W. Fairlie & Samantha H. Grunberg, 2014. "Access To Technology And The Transfer Function Of Community Colleges: Evidence From A Field Experiment," Economic Inquiry, Western Economic Association International, vol. 52(3), pages 1040-1059, July.
    5. Vibhanshu Abhishek & Kartik Hosanagar, 2013. "Optimal Bidding in Multi-Item Multislot Sponsored Search Auctions," Operations Research, INFORMS, vol. 61(4), pages 855-873, August.
    6. Goh, Khim-Yong & Chu, Junhong & Wu, Jing, 2015. "Mobile Advertising: An Empirical Study of Temporal and Spatial Differences in Search Behavior and Advertising Response," Journal of Interactive Marketing, Elsevier, vol. 30(C), pages 34-45.
    7. Kireyev, Pavel & Pauwels, Koen & Gupta, Sunil, 2016. "Do display ads influence search? Attribution and dynamics in online advertising," International Journal of Research in Marketing, Elsevier, vol. 33(3), pages 475-490.
    8. Bergès, Fabian & Monier-Dilhan, Sylvette, 2012. "Mass Retailers’ Advertising Strategies Faced with Different Competitor Store Formats: Commodity Stores or Hard Discounts," TSE Working Papers 12-277, Toulouse School of Economics (TSE).
    9. Ajay Agrawal & John Horton & Nicola Lacetera & Elizabeth Lyons, 2015. "Digitization and the Contract Labor Market: A Research Agenda," NBER Chapters, in: Economic Analysis of the Digital Economy, pages 219-250, National Bureau of Economic Research, Inc.
    10. Anna G. Devlin & Wedad Elmaghraby & Rebecca W. Hamilton, 2018. "Why do suppliers choose wholesale price contracts? End-of-season payments disincentivize retailer marketing effort," Journal of the Academy of Marketing Science, Springer, vol. 46(2), pages 212-233, March.
    11. Fabian Bergès & Sylvette Monier-Dilhan, 2013. "Mass Retailers' Advertising Strategies Against Commodity Stores," Economics Bulletin, AccessEcon, vol. 33(4), pages 2968-2981.
    12. Benjamin Edelman, 2012. "Using Internet Data for Economic Research," Journal of Economic Perspectives, American Economic Association, vol. 26(2), pages 189-206, Spring.
    13. Mingyu Joo & Kenneth C. Wilbur & Bo Cowgill & Yi Zhu, 2014. "Television Advertising and Online Search," Management Science, INFORMS, vol. 60(1), pages 56-73, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    2. Chen He & Tobias J. Klein, 2023. "Advertising as a Reminder: Evidence from the Dutch State Lottery," Marketing Science, INFORMS, vol. 42(5), pages 892-909, September.
    3. Simon P. Anderson & Federico Ciliberto & Jura Liaukonyte & Régis Renault, 2016. "Push-me pull-you: comparative advertising in the OTC analgesics industry," RAND Journal of Economics, RAND Corporation, vol. 47(4), pages 1029-1056, November.
    4. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2020. "How much do consumers know about the quality of products? Evidence from the diaper market," The Japanese Economic Review, Springer, vol. 71(4), pages 541-569, October.
    5. Pradeep Chintagunta & Renna Jiang & Ginger Jin, 2009. "Information, learning, and drug diffusion: The case of Cox-2 inhibitors," Quantitative Marketing and Economics (QME), Springer, vol. 7(4), pages 399-443, December.
    6. Stefano DellaVigna & Matthew Gentzkow, 2010. "Persuasion: Empirical Evidence," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 643-669, September.
    7. Limin Fang, 2022. "The Effects of Online Review Platforms on Restaurant Revenue, Consumer Learning, and Welfare," Management Science, INFORMS, vol. 68(11), pages 8116-8143, November.
    8. Serafin Grundl & You Suk Kim, 2019. "Consumer mistakes and advertising: The case of mortgage refinancing," Quantitative Marketing and Economics (QME), Springer, vol. 17(2), pages 161-213, June.
    9. Jose M. Fernandez, 2013. "An Empirical Model Of Learning Under Ambiguity: The Case Of Clinical Trials," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 54(2), pages 549-573, May.
    10. Tran Thu & Moritaka Masahiro & Liu Ran & Fukuda Susumu, 2018. "Information effect on consumer adoption for a new beef brand in the Vietnamese market: prior knowledge, appealing the brand distinction, differentiation and similarity," Management & Marketing, Sciendo, vol. 13(3), pages 1014-1034, September.
    11. Naoki Aizawa & You Suk Kim, 2015. "Advertising and Risk Selection in Health Insurance Markets," Finance and Economics Discussion Series 2015-101, Board of Governors of the Federal Reserve System (U.S.).
    12. Matthew Shum, 2004. "Does Advertising Overcome Brand Loyalty? Evidence from the Breakfast‐Cereals Market," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 13(2), pages 241-272, June.
    13. Salma Karray, 2015. "Modeling brand advertising with heterogeneous consumer response: channel implications," Annals of Operations Research, Springer, vol. 233(1), pages 181-199, October.
    14. Hilger, James & Rafert, Greg & Villas-Boas, Sofia Berto, 2007. "Expert Opinion and the Demand for Experience Goods: An Experimental Approach," CUDARE Working Papers 6055, University of California, Berkeley, Department of Agricultural and Resource Economics.
    15. Beth A. Freeborn & Jason P. Hulbert, 2011. "Persuasive and Informative Advertising: A Classroom Experiment," The Journal of Economic Education, Taylor & Francis Journals, vol. 42(1), pages 51-59, January.
    16. Lauren H. Cohen & Umit G. Gurun, 2018. "Buying the Verdict," NBER Working Papers 24542, National Bureau of Economic Research, Inc.
    17. Kelli Marquardt & Conor Ryan, 2023. "The Role of Information in Pharmaceutical Advertising: Theory and Evidence," Working Paper Series WP 2023-40, Federal Reserve Bank of Chicago.
    18. He, Chen, 2018. "Essays on the role and effects of advertising," Other publications TiSEM 47a3272a-54f1-4a90-9714-c, Tilburg University, School of Economics and Management.
    19. Sofia B. Villas‐Boas & Kristin Kiesel & Joshua P. Berning & Hayley H. Chouinard & Jill J. McCluskey, 2020. "Consumer and Strategic Firm Response to Nutrition Shelf Labels," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(2), pages 458-479, March.
    20. Farina, Tatiana, 2011. "How Do Experience and Shopping Frequency Affect Consumers Brand Choice?," Insper Working Papers wpe_257, Insper Working Paper, Insper Instituto de Ensino e Pesquisa.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:uea:ueaccp:2011_09. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Juliette Hardmad (email available below). General contact details of provider: https://edirc.repec.org/data/esueauk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.