IDEAS home Printed from https://ideas.repec.org/a/inm/ormksc/v43y2024i2p378-391.html
   My bibliography  Save this article

Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments

Author

Listed:
  • George Z. Gui

    (Columbia Business School, New York, New York 10027)

Abstract

Randomized controlled trials generate experimental variation that can credibly identify causal effects, but often suffer from limited scale, whereas observational data sets are large, but often violate desired identification assumptions. To improve estimation efficiency, I propose a method that leverages imperfect instruments—pretreatment covariates that satisfy the relevance condition, but may violate the exclusion restriction. I show that these imperfect instruments can be used to derive moment restrictions that, in combination with the experimental data, improve estimation efficiency. I outline estimators for implementing this strategy and show that my methods can reduce variance by up to 50%; therefore, only half of the experimental sample is required to attain the same statistical precision. I apply my method to a search-listing data set from Expedia that studies the causal effect of search rankings on clicks and show that the method can substantially improve the precision.

Suggested Citation

  • George Z. Gui, 2024. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Marketing Science, INFORMS, vol. 43(2), pages 378-391, March.
  • Handle: RePEc:inm:ormksc:v:43:y:2024:i:2:p:378-391
    DOI: 10.1287/mksc.2020.0435
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mksc.2020.0435
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mksc.2020.0435?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. M. Keith Chen & Judith A. Chevalier & Peter E. Rossi & Emily Oehlsen, 2019. "The Value of Flexible Work: Evidence from Uber Drivers," Journal of Political Economy, University of Chicago Press, vol. 127(6), pages 2735-2794.
    2. Guido W. Imbens & Tony Lancaster, 1994. "Combining Micro and Macro Data in Microeconometric Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 655-680.
    3. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    4. Susan Athey & Raj Chetty & Guido Imbens, 2020. "Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes," Papers 2006.09676, arXiv.org.
    5. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 263-306.
    6. Carrion, Carlos & Wang, Zenan & Nair, Harikesh & Luo, Xianghong & Lei, Yulin & Lin, Xiliang & Chen, Wenlong & Hu, Qiyu & Peng, Changping & Bao, Yongjun & Yan, Weipeng, 2021. "Blending Advertising with Organic Content in E-Commerce: A Virtual Bids Optimization Approach," Research Papers 3967, Stanford University, Graduate School of Business.
    7. Bertrand, Marianne & Karlan, Dean S. & Mullainathan, Sendhil & Shafir, Eldar & Zinman, Jonathan, 2005. "What's Psychology Worth? A Field Experiment in the Consumer Credit Market," Center Discussion Papers 28441, Yale University, Economic Growth Center.
    8. Raluca M. Ursu, 2018. "The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions," Marketing Science, INFORMS, vol. 37(4), pages 530-552, August.
    9. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    10. Randall A. Lewis & Justin M. Rao, 2015. "The Unfavorable Economics of Measuring the Returns to Advertising," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(4), pages 1941-1973.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. George Z. Gui, 2020. "Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments," Papers 2010.05117, arXiv.org, revised Dec 2023.
    2. Adena, Maja & Hager, Anselm, 2020. "Does online fundraising increase charitable giving? A nation-wide field experiment on Facebook," Discussion Papers, Research Unit: Economics of Change SP II 2020-302, WZB Berlin Social Science Center.
    3. Andre Veiga & Tommaso Valletti, 2020. "Attention, recall and purchase: Experimental evidence on online news and advertising," Working Papers 20-15, NET Institute.
    4. Andrey Simonov & Shawndra Hill, 2021. "Competitive Advertising on Brand Search: Traffic Stealing and Click Quality," Marketing Science, INFORMS, vol. 40(5), pages 923-945, September.
    5. Berman, Ron & Heller, Yuval, 2020. "Naive Analytics Equilibrium," MPRA Paper 103824, University Library of Munich, Germany.
    6. Mariia I. Okuneva & Dmitriy B. Potapov, 2015. "The Effectiveness of Individual Targeting Through Smartphone Application in Retail: Evidence from Field Experiment," HSE Working papers WP BRP 47/MAN/2015, National Research University Higher School of Economics.
    7. Navdeep S. Sahni & S. Christian Wheeler & Pradeep Chintagunta, 2018. "Personalization in Email Marketing: The Role of Noninformative Advertising Content," Marketing Science, INFORMS, vol. 37(2), pages 236-258, March.
    8. Wesley R. Hartmann & Daniel Klapper, 2018. "Super Bowl Ads," Marketing Science, INFORMS, vol. 37(1), pages 78-96, January.
    9. Wendy Netter Epstein & Christopher T. Robertson & David Yokum & Hansoo Ko & Kevin H. Wilson & Monica Ramos & Katherine Kettering & Margaret Houtz, 2022. "Can moral framing drive insurance enrollment in the United States?," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 19(4), pages 804-843, December.
    10. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2023. "Digital public health interventions at scale: The impact of social media advertising on beliefs and outcomes related to COVID vaccines," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(5), pages 2208110120-, January.
    11. Yi-Lin Tsai & Elisabeth Honka, 2021. "Informational and Noninformational Advertising Content," Marketing Science, INFORMS, vol. 40(6), pages 1030-1058, November.
    12. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    13. Susan Athey & Kristen Grabarz & Michael Luca & Nils Wernerfelt, 2022. "The Effectiveness of Digital Interventions on COVID-19 Attitudes and Beliefs," Papers 2206.10214, arXiv.org.
    14. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    15. Brett R Gordon & Kinshuk Jerath & Zsolt Katona & Sridhar Narayanan & Jiwoong Shin & Kenneth C Wilbur, 2019. "Inefficiencies in Digital Advertising Markets," Papers 1912.09012, arXiv.org, revised Feb 2020.
    16. Thomas W. Frick & Rodrigo Belo & Rahul Telang, 2023. "Incentive Misalignments in Programmatic Advertising: Evidence from a Randomized Field Experiment," Management Science, INFORMS, vol. 69(3), pages 1665-1686, March.
    17. Ron Berman & Christophe Van den Bulte, 2022. "False Discovery in A/B Testing," Management Science, INFORMS, vol. 68(9), pages 6762-6782, September.
    18. Susan Athey & Michael Luca, 2019. "Economists (and Economics) in Tech Companies," Journal of Economic Perspectives, American Economic Association, vol. 33(1), pages 209-230, Winter.
    19. Zinman, Jonathan, 2009. "Debit or credit?," Journal of Banking & Finance, Elsevier, vol. 33(2), pages 358-366, February.
    20. McHugh, Sandie & Ranyard, Rob & Lewis, Alan, 2011. "Understanding and knowledge of credit cost and duration: Effects on credit judgements and decisions," Journal of Economic Psychology, Elsevier, vol. 32(4), pages 609-620, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormksc:v:43:y:2024:i:2:p:378-391. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.