IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2302.13857.html
   My bibliography  Save this paper

Multi-cell experiments for marginal treatment effect estimation of digital ads

Author

Listed:
  • Caio Waisman
  • Brett R. Gordon

Abstract

Randomized experiments with treatment and control groups are an important tool to measure the impacts of interventions. However, in experimental settings with one-sided noncompliance, extant empirical approaches may not produce the estimands a decision-maker needs to solve their problem of interest. For example, these experimental designs are common in digital advertising settings, but typical methods do not yield effects that inform the intensive margin -- how many consumers should be reached or how much should be spent on a campaign. We propose a solution that combines a novel multi-cell experimental design with modern estimation techniques that enables decision-makers to recover enough information to solve problems with an intensive margin. Our design is straightforward to implement and does not require any additional budget to be carried out. We illustrate our approach through a series of simulations that are calibrated using an advertising experiment at Facebook, finding that our method outperforms standard techniques in generating better decisions.

Suggested Citation

  • Caio Waisman & Brett R. Gordon, 2023. "Multi-cell experiments for marginal treatment effect estimation of digital ads," Papers 2302.13857, arXiv.org, revised Jan 2024.
  • Handle: RePEc:arx:papers:2302.13857
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2302.13857
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Garrett A. Johnson & Randall A. Lewis & David H. Reiley, 2017. "When Less Is More: Data and Power in Advertising Experiments," Marketing Science, INFORMS, vol. 36(1), pages 43-53, January.
    2. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    3. repec:hal:pseose:halshs-00840901 is not listed on IDEAS
    4. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    5. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(2), pages 531-580.
    6. Eric French & Jae Song, 2014. "The Effect of Disability Insurance Receipt on Labor Supply," American Economic Journal: Economic Policy, American Economic Association, vol. 6(2), pages 291-337, May.
    7. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    8. Bruno Crépon & Florencia Devoto & Esther Duflo & William Parienté, 2015. "Estimating the Impact of Microcredit on Those Who Take It Up: Evidence from a Randomized Experiment in Morocco," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 123-150, January.
    9. repec:adr:anecst:y:2008:i:91-92:p:12 is not listed on IDEAS
    10. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    11. Peter Z. Schochet & John Burghardt & Sheena McConnell, 2008. "Does Job Corps Work? Impact Findings from the National Job Corps Study," American Economic Review, American Economic Association, vol. 98(5), pages 1864-1886, December.
    12. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2022. "Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement," Papers 2201.07055, arXiv.org, revised Oct 2022.
    13. Duncan M. Holthausen, Jr. & Gert Assmus, 1982. "Advertising Budget Allocation under Uncertainty," Management Science, INFORMS, vol. 28(5), pages 487-499, May.
    14. Christian N. Brinch & Magne Mogstad & Matthew Wiswall, 2017. "Beyond LATE with a Discrete Instrument," Journal of Political Economy, University of Chicago Press, vol. 125(4), pages 985-1039.
    15. Olsen, Randall J, 1980. "A Least Squares Correction for Selectivity Bias," Econometrica, Econometric Society, vol. 48(7), pages 1815-1820, November.
    16. Magne Mogstad & Andres Santos & Alexander Torgovitsky, 2018. "Using Instrumental Variables for Inference About Policy Relevant Treatment Parameters," Econometrica, Econometric Society, vol. 86(5), pages 1589-1619, September.
    17. repec:mpr:mprres:6097 is not listed on IDEAS
    18. Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    19. Caio Waisman & Navdeep S. Sahni & Harikesh S. Nair & Xiliang Lin, 2019. "Parallel Experimentation and Competitive Interference on Online Advertising Platforms," Papers 1903.11198, arXiv.org, revised Feb 2024.
    20. Raj Chetty & Nathaniel Hendren & Lawrence F. Katz, 2016. "The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment," American Economic Review, American Economic Association, vol. 106(4), pages 855-902, April.
    21. Robert Moffitt, 2008. "Estimating Marginal Treatment Effects in Heterogeneous Populations," Annals of Economics and Statistics, GENES, issue 91-92, pages 239-261.
    22. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    23. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    2. Matthias Westphal & Daniel A Kamhöfer & Hendrik Schmitz, 2022. "Marginal College Wage Premiums Under Selection Into Employment," The Economic Journal, Royal Economic Society, vol. 132(646), pages 2231-2272.
    3. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    4. Patrick Kline & Christopher R. Walters, 2019. "On Heckits, LATE, and Numerical Equivalence," Econometrica, Econometric Society, vol. 87(2), pages 677-696, March.
    5. Robert A. Moffitt & Matthew V. Zahn, 2019. "The Marginal Labor Supply Disincentives of Welfare: Evidence from Administrative Barriers to Participation," NBER Working Papers 26028, National Bureau of Economic Research, Inc.
    6. Pereda-Fernández, Santiago, 2023. "Identification and estimation of triangular models with a binary treatment," Journal of Econometrics, Elsevier, vol. 234(2), pages 585-623.
    7. Gong, Jie & Lu, Yi & Xie, Huihua, 2020. "The average and distributional effects of teenage adversity on long-term health," Journal of Health Economics, Elsevier, vol. 71(C).
    8. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    9. Bartalotti, Otávio & Kédagni, Désiré & Possebom, Vitor, 2023. "Identifying marginal treatment effects in the presence of sample selection," Journal of Econometrics, Elsevier, vol. 234(2), pages 565-584.
    10. Amanda E Kowalski, 2023. "Behaviour within a Clinical Trial and Implications for Mammography Guidelines," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 432-462.
    11. Domenico Depalo, 2020. "Explaining the causal effect of adherence to medication on cholesterol through the marginal patient," Health Economics, John Wiley & Sons, Ltd., vol. 29(S1), pages 110-126, October.
    12. Marx, Philip, 2024. "Sharp bounds in the latent index selection model," Journal of Econometrics, Elsevier, vol. 238(2).
    13. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    14. Elisa Gerten & Michael Beckmann & Elisa Gerten & Matthias Kräkel, 2022. "Information and Communication Technology, Hierarchy, and Job Design," ECONtribute Discussion Papers Series 189, University of Bonn and University of Cologne, Germany.
    15. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    16. Vishal Kamat & Samuel Norris & Matthew Pecenco, 2023. "Identification in Multiple Treatment Models under Discrete Variation," Papers 2307.06174, arXiv.org.
    17. Pedro Carneiro & James J. Heckman & Edward J. Vytlacil, 2011. "Estimating Marginal Returns to Education," American Economic Review, American Economic Association, vol. 101(6), pages 2754-2781, October.
    18. Tafti, Elena Ashtari, 2023. "Technology, Skills, and Performance: The Case of Robots in Surgery," CINCH Working Paper Series (since 2020) 78746, Duisburg-Essen University Library, DuEPublico.
    19. Yu-Chang Chen & Haitian Xie, 2022. "Personalized Subsidy Rules," Papers 2202.13545, arXiv.org, revised Mar 2022.
    20. Tymon S{l}oczy'nski, 2018. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," Papers 1810.01576, arXiv.org, revised May 2020.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2302.13857. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.