IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2404.13356.html
   My bibliography  Save this paper

How do applied researchers use the Causal Forest? A methodological review of a method

Author

Listed:
  • Patrick Rehill

Abstract

This methodological review examines the use of the causal forest method by applied researchers across 133 peer-reviewed papers. It shows that the emerging best practice relies heavily on the approach and tools created by the original authors of the causal forest such as their grf package and the approaches given by them in examples. Generally researchers use the causal forest on a relatively low-dimensional dataset relying on observed controls or in some cases experiments to identify effects. There are several common ways to then communicate results -- by mapping out the univariate distribution of individual-level treatment effect estimates, displaying variable importance results for the forest and graphing the distribution of treatment effects across covariates that are important either for theoretical reasons or because they have high variable importance. Some deviations from this common practice are interesting and deserve further development and use. Others are unnecessary or even harmful. The paper concludes by reflecting on the emerging best practice for causal forest use and paths for future research.

Suggested Citation

  • Patrick Rehill, 2024. "How do applied researchers use the Causal Forest? A methodological review of a method," Papers 2404.13356, arXiv.org, revised Dec 2024.
  • Handle: RePEc:arx:papers:2404.13356
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2404.13356
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Marta Serra-Garcia & Nora Szech, 2023. "Incentives and Defaults Can Increase COVID-19 Vaccine Intentions and Test Demand," Management Science, INFORMS, vol. 69(2), pages 1037-1049, February.
    2. Guihua Wang, 2022. "The Effect of Medicaid Expansion on Wait Time in the Emergency Department," Management Science, INFORMS, vol. 68(9), pages 6648-6665, September.
    3. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    4. J. Michelle Brock & Ralph De Haas, 2023. "Discriminatory Lending: Evidence from Bankers in the Lab," American Economic Journal: Applied Economics, American Economic Association, vol. 15(2), pages 31-68, April.
    5. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    6. Cockx, Bart & Lechner, Michael & Bollens, Joost, 2023. "Priority to unemployed immigrants? A causal machine learning evaluation of training in Belgium," Labour Economics, Elsevier, vol. 80(C).
    7. Victor Chernozhukov & Mert Demirer & Esther Duflo & Iv'an Fern'andez-Val, 2017. "Fisher-Schultz Lecture: Generic Machine Learning Inference on Heterogenous Treatment Effects in Randomized Experiments, with an Application to Immunization in India," Papers 1712.04802, arXiv.org, revised Oct 2023.
    8. Dylong, Patrick & Uebelmesser, Silke, 2024. "Biased beliefs about immigration and economic concerns: Evidence from representative experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 217(C), pages 453-482.
    9. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    10. Dana Turjeman & Fred M. Feinberg, 2024. "When the Data Are Out: Measuring Behavioral Changes Following a Data Breach," Marketing Science, INFORMS, vol. 43(2), pages 440-461, March.
    11. Yujie Xu & Vivian Loftness & Edson Severnini, 2021. "Using Machine Learning to Predict Retrofit Effects for a Commercial Building Portfolio," Energies, MDPI, vol. 14(14), pages 1-24, July.
    12. Xinkun Nie & Stefan Wager, 2017. "Quasi-Oracle Estimation of Heterogeneous Treatment Effects," Papers 1712.04912, arXiv.org, revised Aug 2020.
    13. Obbey A. Elamin, 2023. "Overeducation wage penalty for university graduates: evidence from the MENA region using machine learning techniques," Middle East Development Journal, Taylor & Francis Journals, vol. 15(1), pages 151-188, January.
    14. Qihang Xue & Huimin Wang & Caiquan Bai, 2023. "Local green finance policies and corporate ESG performance," International Review of Finance, International Review of Finance Ltd., vol. 23(4), pages 721-749, December.
    15. Chengfeng Yu & Jiyu Yu & Da Gao, 2024. "Smart Cities and Greener Futures: Evidence from a Quasi-Natural Experiment in China’s Smart City Construction," Sustainability, MDPI, vol. 16(2), pages 1-28, January.
    16. Huber, Martin & Meier, Jonas & Wallimann, Hannes, 2022. "Business analytics meets artificial intelligence: Assessing the demand effects of discounts on Swiss train tickets," Transportation Research Part B: Methodological, Elsevier, vol. 163(C), pages 22-39.
    17. Mesplé-Somps, Sandrine & Nilsson, Björn, 2023. "Role models, aspirations and desire to migrate," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 819-839.
    18. Amann, Erwin & Rzepka, Sylvi, 2023. "The effect of goal-setting prompts in a blended learning environment—evidence from a field experiment," Economics of Education Review, Elsevier, vol. 92(C).
    19. Hodula, Martin & Melecký, Martin & Pfeifer, Lukáš & Szabo, Milan, 2023. "Cooling the mortgage loan market: The effect of borrower-based limits on new mortgage lending," Journal of International Money and Finance, Elsevier, vol. 132(C).
    20. Jonathan M.V. Davis & Sara B. Heller, 2020. "Rethinking the Benefits of Youth Employment Programs: The Heterogeneous Effects of Summer Jobs," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 664-677, October.
    21. Patrick Rehill & Nicholas Biddle, 2023. "Transparency challenges in policy evaluation with causal machine learning -- improving usability and accountability," Papers 2310.13240, arXiv.org, revised Mar 2024.
    22. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    23. Vira Semenova & Victor Chernozhukov, 2021. "Debiased machine learning of conditional average treatment effects and other causal functions," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 264-289.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Blättler, Kevin & Wallimann, Hannes & von Arx, Widar, 2024. "Free public transport to the destination: A causal analysis of tourists’ travel mode choice," Transportation Research Part A: Policy and Practice, Elsevier, vol. 187(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Goller, Daniel & Harrer, Tamara & Lechner, Michael & Wolff, Joachim, 2021. "Active labour market policies for the long-term unemployed: New evidence from causal machine learning," Economics Working Paper Series 2108, University of St. Gallen, School of Economics and Political Science.
    2. Patrick Rehill & Nicholas Biddle, 2023. "Transparency challenges in policy evaluation with causal machine learning -- improving usability and accountability," Papers 2310.13240, arXiv.org, revised Mar 2024.
    3. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    4. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    5. Daniel Goller, 2023. "Analysing a built-in advantage in asymmetric darts contests using causal machine learning," Annals of Operations Research, Springer, vol. 325(1), pages 649-679, June.
    6. Michael Lechner, 2023. "Causal Machine Learning and its use for public policy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 159(1), pages 1-15, December.
    7. Hugo Bodory & Martin Huber & Michael Lechner, 2024. "The Finite Sample Performance of Instrumental Variable-Based Estimators of the Local Average Treatment Effect When Controlling for Covariates," Computational Economics, Springer;Society for Computational Economics, vol. 64(4), pages 2053-2078, October.
    8. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    9. Michael C Knaus, 2022. "Double machine learning-based programme evaluation under unconfoundedness [Econometric methods for program evaluation]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 602-627.
    10. Cockx, Bart & Lechner, Michael & Bollens, Joost, 2023. "Priority to unemployed immigrants? A causal machine learning evaluation of training in Belgium," Labour Economics, Elsevier, vol. 80(C).
    11. Daniel Boller & Michael Lechner & Gabriel Okasa, 2021. "The Effect of Sport in Online Dating: Evidence from Causal Machine Learning," Papers 2104.04601, arXiv.org.
    12. Patrick Rehill & Nicholas Biddle, 2024. "Heterogeneous treatment effect estimation with high-dimensional data in public policy evaluation -- an application to the conditioning of cash transfers in Morocco using causal machine learning," Papers 2401.07075, arXiv.org, revised Mar 2024.
    13. Henrika Langen & Martin Huber, 2022. "How causal machine learning can leverage marketing strategies: Assessing and improving the performance of a coupon campaign," Papers 2204.10820, arXiv.org, revised Jun 2022.
    14. Rahul Singh & Liyuan Xu & Arthur Gretton, 2020. "Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves," Papers 2010.04855, arXiv.org, revised Oct 2022.
    15. Martin Huber & Jannis Kueck, 2022. "Testing the identification of causal effects in observational data," Papers 2203.15890, arXiv.org, revised Jun 2023.
    16. Phillip Heiler & Michael C. Knaus, 2021. "Effect or Treatment Heterogeneity? Policy Evaluation with Aggregated and Disaggregated Treatments," Papers 2110.01427, arXiv.org, revised Aug 2023.
    17. Zhaonan Qu & Ruoxuan Xiong & Jizhou Liu & Guido Imbens, 2021. "Semiparametric Estimation of Treatment Effects in Observational Studies with Heterogeneous Partial Interference," Papers 2107.12420, arXiv.org, revised Jun 2024.
    18. Alejandro Sanchez-Becerra, 2023. "Robust inference for the treatment effect variance in experiments using machine learning," Papers 2306.03363, arXiv.org.
    19. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    20. Verhagen, Mark D., 2023. "Using machine learning to monitor the equity of large-scale policy interventions: The Dutch decentralisation of the Social Domain," SocArXiv qzm7y, Center for Open Science.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2404.13356. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.