IDEAS home Printed from https://ideas.repec.org/a/inm/ormksc/v43y2024i2p378-391.html
   My bibliography  Save this article

Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments

Author

Listed:
  • George Z. Gui

    (Columbia Business School, New York, New York 10027)

Abstract

Randomized controlled trials generate experimental variation that can credibly identify causal effects, but often suffer from limited scale, whereas observational data sets are large, but often violate desired identification assumptions. To improve estimation efficiency, I propose a method that leverages imperfect instruments—pretreatment covariates that satisfy the relevance condition, but may violate the exclusion restriction. I show that these imperfect instruments can be used to derive moment restrictions that, in combination with the experimental data, improve estimation efficiency. I outline estimators for implementing this strategy and show that my methods can reduce variance by up to 50%; therefore, only half of the experimental sample is required to attain the same statistical precision. I apply my method to a search-listing data set from Expedia that studies the causal effect of search rankings on clicks and show that the method can substantially improve the precision.

Suggested Citation

  • George Z. Gui, 2024. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Marketing Science, INFORMS, vol. 43(2), pages 378-391, March.
  • Handle: RePEc:inm:ormksc:v:43:y:2024:i:2:p:378-391
    DOI: 10.1287/mksc.2020.0435
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mksc.2020.0435
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mksc.2020.0435?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. M. Keith Chen & Judith A. Chevalier & Peter E. Rossi & Emily Oehlsen, 2019. "The Value of Flexible Work: Evidence from Uber Drivers," Journal of Political Economy, University of Chicago Press, vol. 127(6), pages 2735-2794.
    2. Raluca M. Ursu, 2018. "The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions," Marketing Science, INFORMS, vol. 37(4), pages 530-552, August.
    3. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    4. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    5. Guido W. Imbens & Tony Lancaster, 1994. "Combining Micro and Macro Data in Microeconometric Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 655-680.
    6. Susan Athey & Raj Chetty & Guido Imbens, 2020. "Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes," Papers 2006.09676, arXiv.org.
    7. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 263-306.
    8. Carlos Carrion & Zenan Wang & Harikesh Nair & Xianghong Luo & Yulin Lei & Xiliang Lin & Wenlong Chen & Qiyu Hu & Changping Peng & Yongjun Bao & Weipeng Yan, 2021. "Blending Advertising with Organic Content in E-Commerce: A Virtual Bids Optimization Approach," Papers 2105.13556, arXiv.org.
    9. Randall A. Lewis & Justin M. Rao, 2015. "The Unfavorable Economics of Measuring the Returns to Advertising," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(4), pages 1941-1973.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. George Z. Gui, 2020. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Papers 2010.05117, arXiv.org, revised Dec 2023.
    2. Adena, Maja & Hager, Anselm, 2024. "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook," Discussion Papers, Research Unit: Economics of Change SP II 2020-302r2, WZB Berlin Social Science Center, revised 2024.
    3. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2023. "Digital public health interventions at scale: The impact of social media advertising on beliefs and outcomes related to COVID vaccines," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(5), pages 2208110120-, January.
    4. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    5. Andre Veiga & Tommaso Valletti, 2020. "Attention, recall and purchase: Experimental evidence on online news and advertising," Working Papers 20-15, NET Institute.
    6. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    7. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    8. Adena, Maja & Hager, Anselm, 2020. "Does online fundraising increase charitable giving? A nation-wide field experiment on Facebook," Discussion Papers, Research Unit: Economics of Change SP II 2020-302, WZB Berlin Social Science Center.
    9. Ron Berman & Christophe Van den Bulte, 2022. "False Discovery in A/B Testing," Management Science, INFORMS, vol. 68(9), pages 6762-6782, September.
    10. Andrey Simonov & Shawndra Hill, 2021. "Competitive Advertising on Brand Search: Traffic Stealing and Click Quality," Marketing Science, INFORMS, vol. 40(5), pages 923-945, September.
    11. Berman, Ron & Heller, Yuval, 2020. "Naive Analytics Equilibrium," MPRA Paper 103824, University Library of Munich, Germany.
    12. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2022. "The Effectiveness of Digital Interventions on COVID-19 Attitudes and Beliefs," Papers 2206.10214, arXiv.org.
    13. Thomas W. Frick & Rodrigo Belo & Rahul Telang, 2023. "Incentive Misalignments in Programmatic Advertising: Evidence from a Randomized Field Experiment," Management Science, INFORMS, vol. 69(3), pages 1665-1686, March.
    14. Bradley T. Shapiro, 2020. "Advertising in Health Insurance Markets," Marketing Science, INFORMS, vol. 39(3), pages 587-611, May.
    15. Randall Lewis & Dan Nguyen, 2015. "Display advertising’s competitive spillovers to consumer search," Quantitative Marketing and Economics (QME), Springer, vol. 13(2), pages 93-115, June.
    16. Jacob LaRiviere & Mikolaj Czajkowski & Nick Hanley & Katherine Simpson, 2016. "What is the Causal Impact of Knowledge on Preferences in Stated Preference Studies?," Working Papers 2016-12, Faculty of Economic Sciences, University of Warsaw.
    17. Tobias Regner, 2021. "Crowdfunding a monthly income: an analysis of the membership platform Patreon," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 45(1), pages 133-142, March.
    18. Wei Zhou & Zidong Wang, 2020. "Competing for Search Traffic in Query Markets: Entry Strategy, Platform Design, and Entrepreneurship," Working Papers 20-12, NET Institute.
    19. Wesley R. Hartmann & Daniel Klapper, 2018. "Super Bowl Ads," Marketing Science, INFORMS, vol. 37(1), pages 78-96, January.
    20. Uddin, Main & Wang, Liang Choon & Smyth, Russell, 2021. "Do government-initiated energy comparison sites encourage consumer search and lower prices? Evidence from an online randomized controlled experiment in Australia," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 167-182.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormksc:v:43:y:2024:i:2:p:378-391. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.