IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1803.09159.html
   My bibliography  Save this paper

Efficient Discovery of Heterogeneous Quantile Treatment Effects in Randomized Experiments via Anomalous Pattern Detection

Author

Listed:
  • Edward McFowland III
  • Sriram Somanchi
  • Daniel B. Neill

Abstract

In the recent literature on estimating heterogeneous treatment effects, each proposed method makes its own set of restrictive assumptions about the intervention's effects and which subpopulations to explicitly estimate. Moreover, the majority of the literature provides no mechanism to identify which subpopulations are the most affected--beyond manual inspection--and provides little guarantee on the correctness of the identified subpopulations. Therefore, we propose Treatment Effect Subset Scan (TESS), a new method for discovering which subpopulation in a randomized experiment is most significantly affected by a treatment. We frame this challenge as a pattern detection problem where we efficiently maximize a nonparametric scan statistic (a measure of the conditional quantile treatment effect) over subpopulations. Furthermore, we identify the subpopulation which experiences the largest distributional change as a result of the intervention, while making minimal assumptions about the intervention's effects or the underlying data generating process. In addition to the algorithm, we demonstrate that under the sharp null hypothesis of no treatment effect, the asymptotic Type I and II error can be controlled, and provide sufficient conditions for detection consistency--i.e., exact identification of the affected subpopulation. Finally, we validate the efficacy of the method by discovering heterogeneous treatment effects in simulations and in real-world data from a well-known program evaluation study.

Suggested Citation

  • Edward McFowland III & Sriram Somanchi & Daniel B. Neill, 2018. "Efficient Discovery of Heterogeneous Quantile Treatment Effects in Randomized Experiments via Anomalous Pattern Detection," Papers 1803.09159, arXiv.org, revised May 2023.
  • Handle: RePEc:arx:papers:1803.09159
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1803.09159
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    2. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 497-532.
    3. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    4. Grimmer, Justin & Messing, Solomon & Westwood, Sean J., 2017. "Estimating Heterogeneous Treatment Effects and the Effects of Heterogeneous Treatments with Ensemble Methods," Political Analysis, Cambridge University Press, vol. 25(4), pages 413-434, October.
    5. Daniel B. Neill, 2012. "Fast subset scan for spatial pattern detection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 74(2), pages 337-360, March.
    6. Blundell,Richard & Newey,Whitney K. & Persson,Torsten (ed.), 2006. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521871525, January.
    7. Blundell,Richard & Newey,Whitney K. & Persson,Torsten (ed.), 2006. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521692083, January.
    8. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    9. David W. Nickerson & Todd Rogers, 2014. "Political Campaigns and Big Data," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 51-74, Spring.
    10. Jill U. Adams, 2015. "Genetics: Big hopes for big data," Nature, Nature, vol. 527(7578), pages 108-109, November.
    11. Lu Tian & Ash A. Alizadeh & Andrew J. Gentles & Robert Tibshirani, 2014. "A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1517-1532, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    2. Mochen Yang & Edward McFowland & Gordon Burtch & Gediminas Adomavicius, 2022. "Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem," INFORMS Joural on Data Science, INFORMS, vol. 1(2), pages 138-155, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    2. Martin Huber, 2012. "Identification of Average Treatment Effects in Social Experiments Under Alternative Forms of Attrition," Journal of Educational and Behavioral Statistics, , vol. 37(3), pages 443-474, June.
    3. Engel, Christoph, 2020. "Estimating heterogeneous reactions to experimental treatments," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 124-147.
    4. Michael C. Knaus & Michael Lechner & Anthony Strittmatter, 2022. "Heterogeneous Employment Effects of Job Search Programs: A Machine Learning Approach," Journal of Human Resources, University of Wisconsin Press, vol. 57(2), pages 597-636.
    5. Feng, Sanying & Kong, Kaidi & Kong, Yinfei & Li, Gaorong & Wang, Zhaoliang, 2022. "Statistical inference of heterogeneous treatment effect based on single-index model," Computational Statistics & Data Analysis, Elsevier, vol. 175(C).
    6. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    7. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    8. Michel, Christian, 2017. "Market regulation of voluntary add-on contracts," International Journal of Industrial Organization, Elsevier, vol. 54(C), pages 239-268.
    9. Shonosuke Sugasawa & Hisashi Noma, 2021. "Efficient screening of predictive biomarkers for individual treatment selection," Biometrics, The International Biometric Society, vol. 77(1), pages 249-257, March.
    10. Federico Ciliberto & Elie Tamer, 2009. "Market Structure and Multiple Equilibria in Airline Markets," Econometrica, Econometric Society, vol. 77(6), pages 1791-1828, November.
    11. Carmona, Guilherme & Fajardo, José, 2009. "Existence of equilibrium in common agency games with adverse selection," Games and Economic Behavior, Elsevier, vol. 66(2), pages 749-760, July.
    12. León, Gianmarco, 2017. "Turnout, political preferences and information: Experimental evidence from Peru," Journal of Development Economics, Elsevier, vol. 127(C), pages 56-71.
    13. Dimitris Georgarakos & Giacomo Pasini, 2011. "Trust, Sociability, and Stock Market Participation," Review of Finance, European Finance Association, vol. 15(4), pages 693-725.
    14. Luís Cabral, 2018. "We’re Number 1: Price Wars for Market Share Leadership," Management Science, INFORMS, vol. 64(5), pages 2013-2030, May.
    15. Ming Li & Dipjyoti Majumdar, 2010. "A Psychologically Based Model of Voter Turnout," Journal of Public Economic Theory, Association for Public Economic Theory, vol. 12(5), pages 979-1002, October.
    16. Per Krusell & Anthony Smith & Joachim Hubmer, 2015. "The historical evolution of the wealth distribution: A quantitative-theoretic investigation," 2015 Meeting Papers 1406, Society for Economic Dynamics.
    17. Jentzsch, Nicola & Sapi, Geza & Suleymanova, Irina, 2013. "Targeted pricing and customer data sharing among rivals," International Journal of Industrial Organization, Elsevier, vol. 31(2), pages 131-144.
    18. Davis, John B., 2010. "Neuroeconomics: Constructing identity," Journal of Economic Behavior & Organization, Elsevier, vol. 76(3), pages 574-583, December.
    19. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1803.09159. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.