IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/19520.html
   My bibliography  Save this paper

Measuring the Effects of Advertising: The Digital Frontier

Author

Listed:
  • Randall Lewis
  • Justin M. Rao
  • David H. Reiley

Abstract

Online advertising offers unprecedented opportunities for measurement. A host of new metrics, clicks being the leading example, have become widespread in advertising science. New data and experimentation platforms open the door for firms and researchers to measure true causal effects of advertising on a variety of consumer behaviors, such as purchases. We dissect the new metrics and methods currently used by industry researchers, attacking the question, "How hard is it to reliably measure advertising effectiveness?" We outline the questions that we think can be answered by current data and methods, those that we believe will be in play within five years, and those that we believe could not be answered with arbitrarily large and detailed data. We pay close attention to the advances in computational advertising that are not only increasing the impact of advertising, but also usefully shifting the focus from "who to hit" to "what do I get."

Suggested Citation

  • Randall Lewis & Justin M. Rao & David H. Reiley, 2013. "Measuring the Effects of Advertising: The Digital Frontier," NBER Working Papers 19520, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:19520
    Note: IO
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w19520.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    2. Avi Goldfarb & Catherine Tucker, 2011. "Online Display Advertising: Targeting and Obtrusiveness," Marketing Science, INFORMS, vol. 30(3), pages 389-404, 05-06.
    3. Avi Goldfarb & Catherine E. Tucker, 2011. "Privacy Regulation and Online Advertising," Management Science, INFORMS, vol. 57(1), pages 57-71, January.
    4. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    5. Card, David, 1999. "The causal effect of education on earnings," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 30, pages 1801-1863, Elsevier.
    6. Jacob Mincer, 1962. "On-the-Job Training: Costs, Returns, and Some Implications," NBER Chapters, in: Investment in Human Beings, pages 50-79, National Bureau of Economic Research, Inc.
    7. O. Ashenfelter & D. Card (ed.), 1999. "Handbook of Labor Economics," Handbook of Labor Economics, Elsevier, edition 1, volume 3, number 3.
    8. Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    9. Tucker, Catherine E., 2012. "The economics of advertising and privacy," International Journal of Industrial Organization, Elsevier, vol. 30(3), pages 326-329.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Robert W. Fairlie & Samantha H. Grunberg, 2014. "Access To Technology And The Transfer Function Of Community Colleges: Evidence From A Field Experiment," Economic Inquiry, Western Economic Association International, vol. 52(3), pages 1040-1059, July.
    2. Luis Aguiar & Joel Waldfogel, 2018. "Platforms, Promotion, and Product Discovery: Evidence from Spotify Playlists," JRC Working Papers on Digital Economy 2018-04, Joint Research Centre.
    3. L. Elisa Celis & Gregory Lewis & Markus Mobius & Hamid Nazerzadeh, 2014. "Buy-It-Now or Take-a-Chance: Price Discrimination Through Randomized Auctions," Management Science, INFORMS, vol. 60(12), pages 2927-2948, December.
    4. Hana Choi & Carl F. Mela & Santiago R. Balseiro & Adam Leary, 2020. "Online Display Advertising Markets: A Literature Review and Future Directions," Information Systems Research, INFORMS, vol. 31(2), pages 556-575, June.
    5. Avi Goldfarb & Catherine Tucker, 2012. "Privacy and Innovation," Innovation Policy and the Economy, University of Chicago Press, vol. 12(1), pages 65-90.
    6. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    7. J. Levin & L. Einav, 2012. "Empirical Industrial Organization: A Progress Report," Voprosy Ekonomiki, NP Voprosy Ekonomiki, issue 1.
    8. Roktim Sarmah & Ayesha Khatun & Aayushi Singh, 2023. "Perception and Awareness of Youth Toward the Social Advertising Campaigns Being Run by Private Brands," International Journal of Asian Business and Information Management (IJABIM), IGI Global, vol. 14(1), pages 1-20, January.
    9. Avi Goldfarb & Catherine E. Tucker, 2015. "Standardization and the Effectiveness of Online Advertising," Management Science, INFORMS, vol. 61(11), pages 2707-2719, November.
    10. Bergès, Fabian & Monier-Dilhan, Sylvette, 2012. "Mass Retailers’ Advertising Strategies Faced with Different Competitor Store Formats: Commodity Stores or Hard Discounts," TSE Working Papers 12-277, Toulouse School of Economics (TSE).
    11. Susan Athey & Emilio Calvano & Joshua Gans, 2013. "The Impact of the Internet on Advertising Markets for News Media," NBER Working Papers 19419, National Bureau of Economic Research, Inc.
    12. Susan Athey & Emilio Calvano & Joshua S. Gans, 2014. "The Impact of Consumer Multi-homing on Advertising Markets and Media Competition," CSEF Working Papers 379, Centre for Studies in Economics and Finance (CSEF), University of Naples, Italy, revised 27 Apr 2016.
    13. Ajay Agrawal & John Horton & Nicola Lacetera & Elizabeth Lyons, 2015. "Digitization and the Contract Labor Market: A Research Agenda," NBER Chapters, in: Economic Analysis of the Digital Economy, pages 219-250, National Bureau of Economic Research, Inc.
    14. Adena, Maja & Hager, Anselm, 2020. "Does online fundraising increase charitable giving? A nation-wide field experiment on Facebook," Discussion Papers, Research Unit: Economics of Change SP II 2020-302, WZB Berlin Social Science Center.
    15. Lint Barrage & Eric Chyn & Justine Hastings, 2020. "Advertising and Environmental Stewardship: Evidence from the BP Oil Spill," American Economic Journal: Economic Policy, American Economic Association, vol. 12(1), pages 33-61, February.
    16. Jonathan Levin, 2011. "The Economics of Internet Markets," Discussion Papers 10-018, Stanford Institute for Economic Policy Research.
    17. Luis Aguiar & Joel Waldfogel, 2021. "Platforms, Power, and Promotion: Evidence from Spotify Playlists," Journal of Industrial Economics, Wiley Blackwell, vol. 69(3), pages 653-691, September.
    18. Fabian Bergès & Sylvette Monier-Dilhan, 2013. "Mass Retailers' Advertising Strategies Against Commodity Stores," Economics Bulletin, AccessEcon, vol. 33(4), pages 2968-2981.
    19. Krishen, Anjala S. & Dwivedi, Yogesh K. & Bindu, N. & Kumar, K. Satheesh, 2021. "A broad overview of interactive digital marketing: A bibliometric network analysis," Journal of Business Research, Elsevier, vol. 131(C), pages 183-195.
    20. Richard Friberg & Erik Grönqvist, 2012. "Do Expert Reviews Affect the Demand for Wine?," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 193-211, January.
    21. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2022. "Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement," Papers 2201.07055, arXiv.org, revised Oct 2022.
    22. Benjamin Edelman, 2012. "Using Internet Data for Economic Research," Journal of Economic Perspectives, American Economic Association, vol. 26(2), pages 189-206, Spring.
    23. Justin M. Rao & Andrey Simonov, 2019. "Firms’ reactions to public information on business practices: The case of search advertising," Quantitative Marketing and Economics (QME), Springer, vol. 17(2), pages 105-134, June.
    24. Anindya Ghose & Vilma Todri, 2015. "Towards a Digital Attribution Model: Measuring the Impact of Display Advertising on Online Consumer Behavior," Working Papers 15-15, NET Institute.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Randall Lewis & Dan Nguyen, 2015. "Display advertising’s competitive spillovers to consumer search," Quantitative Marketing and Economics (QME), Springer, vol. 13(2), pages 93-115, June.
    2. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    3. Avi Goldfarb, 2014. "What is Different About Online Advertising?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 44(2), pages 115-129, March.
    4. Miller, Klaus M. & Skiera, Bernd, 2024. "Economic consequences of online tracking restrictions: Evidence from cookies," International Journal of Research in Marketing, Elsevier, vol. 41(2), pages 241-264.
    5. Adriaan Zon & Roberto Antonietti, 2016. "Education and training in a model of endogenous growth with creative wear-and-tear," Economia Politica: Journal of Analytical and Institutional Economics, Springer;Fondazione Edison, vol. 33(1), pages 35-62, April.
    6. Hana Choi & Carl F. Mela & Santiago R. Balseiro & Adam Leary, 2020. "Online Display Advertising Markets: A Literature Review and Future Directions," Information Systems Research, INFORMS, vol. 31(2), pages 556-575, June.
    7. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    8. Habermalz, Steffen, 2003. "Job Matching and the Returns to Educational Signals," IZA Discussion Papers 726, Institute of Labor Economics (IZA).
    9. Joel Barajas & Ram Akella & Marius Holtan & Aaron Flores, 2016. "Experimental Designs and Estimation for Online Display Advertising Attribution in Marketplaces," Marketing Science, INFORMS, vol. 35(3), pages 465-483, May.
    10. Anna D’Annunzio & Antonio Russo, 2020. "Ad Networks and Consumer Tracking," Management Science, INFORMS, vol. 66(11), pages 5040-5058, November.
    11. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    12. Jacobs, Bas, 2007. "Real options and human capital investment," Labour Economics, Elsevier, vol. 14(6), pages 913-925, December.
    13. Francine D. Blau & Lawrence M. Kahn, 2013. "The Feasibility and Importance of Adding Measures of Actual Experience to Cross-Sectional Data Collection," Journal of Labor Economics, University of Chicago Press, vol. 31(S1), pages 17-58.
    14. Mariia I. Okuneva & Dmitriy B. Potapov, 2015. "The Effectiveness of Individual Targeting Through Smartphone Application in Retail: Evidence from Field Experiment," HSE Working papers WP BRP 47/MAN/2015, National Research University Higher School of Economics.
    15. Alexandre Chirat, 2022. "Consumer sovereignty in the digital society," EconomiX Working Papers 2022-25, University of Paris Nanterre, EconomiX.
    16. Jan Krämer & Daniel Schnurr & Michael Wohlfarth, 2019. "Winners, Losers, and Facebook: The Role of Social Logins in the Online Advertising Ecosystem," Management Science, INFORMS, vol. 65(4), pages 1678-1699, April.
    17. Peitz, Martin & Reisinger, Markus, 2014. "The Economics of Internet Media," Working Papers 14-23, University of Mannheim, Department of Economics.
    18. Anja Lambrecht & Avi Goldfarb & Alessandro Bonatti & Anindya Ghose & Daniel Goldstein & Randall Lewis & Anita Rao & Navdeep Sahni & Song Yao, 2014. "How do firms make money selling digital goods online?," Marketing Letters, Springer, vol. 25(3), pages 331-341, September.
    19. Garrett A. Johnson & Randall A. Lewis & David H. Reiley, 2017. "When Less Is More: Data and Power in Advertising Experiments," Marketing Science, INFORMS, vol. 36(1), pages 43-53, January.
    20. Yossi Perelman & Meir Yaish & Benjamin Bental, 2019. "The price of religiosity: Enticing young Haredi men into secular academic studies," Rationality and Society, , vol. 31(2), pages 129-151, May.

    More about this item

    JEL classification:

    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design
    • L22 - Industrial Organization - - Firm Objectives, Organization, and Behavior - - - Firm Organization and Market Structure
    • M37 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Marketing and Advertising - - - Advertising

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:19520. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.