IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1903.11198.html
   My bibliography  Save this paper

Parallel Experimentation and Competitive Interference on Online Advertising Platforms

Author

Listed:
  • Caio Waisman
  • Navdeep S. Sahni
  • Harikesh S. Nair
  • Xiliang Lin

Abstract

This paper studies the measurement of advertising effects on online platforms when parallel experimentation occurs, that is, when multiple advertisers experiment concurrently. It provides a framework that makes precise how parallel experimentation affects the experiment's value: while ignoring parallel experimentation yields an estimate of the average effect of advertising in-place, which has limited value in decision-making in an environment with variable advertising competition, accounting for parallel experimentation captures the actual uncertainty advertisers face due to competitive actions. It then implements an experimental design that enables the estimation of these effects on JD.com, a large e-commerce platform that is also a publisher of digital ads. Using traditional and kernel-based estimators, it shows that not accounting for competitive actions can result in the advertiser inaccurately estimating the advertising lift by a factor of two or higher, which can be consequential for decision-making.

Suggested Citation

  • Caio Waisman & Navdeep S. Sahni & Harikesh S. Nair & Xiliang Lin, 2019. "Parallel Experimentation and Competitive Interference on Online Advertising Platforms," Papers 1903.11198, arXiv.org, revised Feb 2024.
    • Handle: RePEc:arx:papers:1903.11198
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1903.11198
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Daron Acemoglu, 2010. "Theory, General Equilibrium, and Political Economy in Development Economics," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 17-32, Summer.
    2. Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    3. Bradley T. Shapiro, 2018. "Positive Spillovers and Free Riding in Advertising of Prescription Pharmaceuticals: The Case of Antidepressants," Journal of Political Economy, University of Chicago Press, vol. 126(1), pages 381-437.
    4. Navdeep S Sahni & Harikesh S Nair, 2020. "Does Advertising Serve as a Signal? Evidence from a Field Experiment in Mobile Search," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 87(3), pages 1529-1564.
    5. Navdeep S. Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    6. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    7. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    8. Sridhar Narayanan & Kirthi Kalyanam, 2015. "Position Effects in Search Advertising and their Moderators: A Regression Discontinuity Approach," Marketing Science, INFORMS, vol. 34(3), pages 388-407, May.
    9. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    10. Burke, Raymond R & Srull, Thomas K, 1988. "Competitive Interference and Consumer Memory for Advertising," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 15(1), pages 55-68, June.
    11. Navdeep S. Sahni, 2015. "Erratum to: Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 249-250, September.
    12. Avi Goldfarb & Catherine Tucker, 2011. "Search Engine Advertising: Channel Substitution When Pricing Ads to Context," Management Science, INFORMS, vol. 57(3), pages 458-470, March.
    13. Navdeep Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    14. Keller, Kevin Lane, 1987. "Memory Factors in Advertising: The Effect of Advertising Retrieval Cues on Brand Evaluations," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 14(3), pages 316-333, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    2. Geng, Tong & Lin, Xiliang & Nair, Harikesh S. & Hao, Jun & Xiang, Bin & Fan, Shurui, 2020. "Comparison Lift: Bandit-Based Experimentation System for Online Advertising," Research Papers 3904, Stanford University, Graduate School of Business.
    3. Caio Waisman & Brett R. Gordon, 2023. "Multi-cell experiments for marginal treatment effect estimation of digital ads," Papers 2302.13857, arXiv.org, revised Jan 2024.
    4. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Navdeep S. Sahni & Harikesh S. Nair, 2020. "Sponsorship Disclosure and Consumer Deception: Experimental Evidence from Native Advertising in Mobile Search," Marketing Science, INFORMS, vol. 39(1), pages 5-32, January.
    2. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    3. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    4. Kirthi Kalyanam & John McAteer & Jonathan Marek & James Hodges & Lifeng Lin, 2018. "Cross channel effects of search engine advertising on brick & mortar retail sales: Meta analysis of large scale field experiments on Google.com," Quantitative Marketing and Economics (QME), Springer, vol. 16(1), pages 1-42, March.
    5. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2023. "Predictive Incrementality by Experimentation (PIE) for Ad Measurement," Papers 2304.06828, arXiv.org.
    6. Hana Choi & Carl F. Mela & Santiago R. Balseiro & Adam Leary, 2020. "Online Display Advertising Markets: A Literature Review and Future Directions," Information Systems Research, INFORMS, vol. 31(2), pages 556-575, June.
    7. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    8. Stephan Seiler & Song Yao & Wenbo Wang, 2017. "Does Online Word of Mouth Increase Demand? (And How?) Evidence from a Natural Experiment," Marketing Science, INFORMS, vol. 36(6), pages 838-861, November.
    9. Johannes Hermle & Giorgio Martini, 2022. "Valid and Unobtrusive Measurement of Returns to Advertising through Asymmetric Budget Split," Papers 2207.00206, arXiv.org.
    10. Garrett A. Johnson & Randall A. Lewis & David H. Reiley, 2017. "When Less Is More: Data and Power in Advertising Experiments," Marketing Science, INFORMS, vol. 36(1), pages 43-53, January.
    11. Thomas W. Frick & Rodrigo Belo & Rahul Telang, 2023. "Incentive Misalignments in Programmatic Advertising: Evidence from a Randomized Field Experiment," Management Science, INFORMS, vol. 69(3), pages 1665-1686, March.
    12. Navdeep Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    13. Navdeep S. Sahni & Dan Zou & Pradeep K. Chintagunta, 2017. "Do Targeted Discount Offers Serve as Advertising? Evidence from 70 Field Experiments," Management Science, INFORMS, vol. 63(8), pages 2688-2705, August.
    14. Chen He & Tobias J. Klein, 2023. "Advertising as a Reminder: Evidence from the Dutch State Lottery," Marketing Science, INFORMS, vol. 42(5), pages 892-909, September.
    15. Ruichang Lu & Qiaowei Shen & Tenghui Wang & Xiaojun Zhang, 2022. "Frenemies: Corporate Advertising Under Common Ownership," Management Science, INFORMS, vol. 68(6), pages 4645-4669, June.
    16. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    17. Min Tian & Paul R. Hoban & Neeraj Arora, 2024. "What Cookie-Based Advertising Effectiveness Fails to Measure," Marketing Science, INFORMS, vol. 43(2), pages 407-418, March.
    18. Li, Xiaolin & Rao, Raghunath Singh & Narasimhan, Om & Gao, Xing, 2022. "Stay positive or go negative? Memory imperfections and messaging strategy," International Journal of Research in Marketing, Elsevier, vol. 39(4), pages 1127-1149.
    19. Navdeep S. Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    20. Li, Xiaolin & Singh Rao, Raghunath & Narasimhan, Om & Gao, Xing, 2022. "Stay positive or go negative? Memory imperfections and messaging strategy," LSE Research Online Documents on Economics 113556, London School of Economics and Political Science, LSE Library.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1903.11198. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.