IDEAS home Printed from https://ideas.repec.org/a/kap/jgeosy/v26y2024i4d10.1007_s10109-023-00413-0.html
   My bibliography  Save this article

A structured comparison of causal machine learning methods to assess heterogeneous treatment effects in spatial data

Author

Listed:
  • Kevin Credit

    (Maynooth University)

  • Matthew Lehnert

    (Satelytics)

Abstract

The development of the “causal” forest by Wager and Athey (J Am Stat Assoc 113(523): 1228–1242, 2018) represents a significant advance in the area of explanatory/causal machine learning. However, this approach has not yet been widely applied to geographically referenced data, which present some unique issues: the random split of the test and training sets in the typical causal forest design fractures the spatial fabric of geographic data. To help solve this issue, we use a simulated dataset with known properties for average treatment effects and conditional average treatment effects to compare the performance of CF models across different definitions of the test/train split. We also develop a new “spatial” T-learner that can be implemented using predictive methods like random forest to provide estimates of heterogeneous treatment effects across all units. Our results show that all of the machine learning models outperform traditional ordinary least squares regression at identifying the true average treatment effect, but are not significantly different from one another. We then apply the preferred causal forest model in the context of analysing the treatment effect of the construction of the Valley Metro light rail (tram) system on on-road CO2 emissions per capita at the block group level in Maricopa County, Arizona, and find that the neighbourhoods most likely to benefit from treatment are those with higher pre-treatment proportions of transit and pedestrian commuting and lower proportions of auto commuting.

Suggested Citation

  • Kevin Credit & Matthew Lehnert, 2024. "A structured comparison of causal machine learning methods to assess heterogeneous treatment effects in spatial data," Journal of Geographical Systems, Springer, vol. 26(4), pages 483-510, October.
  • Handle: RePEc:kap:jgeosy:v:26:y:2024:i:4:d:10.1007_s10109-023-00413-0
    DOI: 10.1007/s10109-023-00413-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10109-023-00413-0
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s10109-023-00413-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Glaeser, Edward L. & Kahn, Matthew E., 2010. "The greenness of cities: Carbon dioxide emissions and urban development," Journal of Urban Economics, Elsevier, vol. 67(3), pages 404-418, May.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. Hoffman, Ian & Mast, Evan, 2019. "Heterogeneity in the effect of federal spending on local crime: Evidence from causal forests," Regional Science and Urban Economics, Elsevier, vol. 78(C).
    4. Reid Ewing & Shima Hamidi, 2014. "Longitudinal Analysis of Transit's Land Use Multiplier in Portland (OR)," Journal of the American Planning Association, Taylor & Francis Journals, vol. 80(2), pages 123-137, April.
    5. Kosfeld, Reinhold & Mitze, Timo & Rode, Johannes & Wälde, Klaus, 2021. "The Covid-19 containment effects of public health measures - A spatial difference-in-differences approach," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 126127, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    6. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    7. Anthony Strittmatter, 2018. "What Is the Value Added by Using Causal Machine Learning Methods in a Welfare Experiment Evaluation?," Papers 1812.06533, arXiv.org, revised Dec 2021.
    8. Jonathan M.V. Davis & Sara B. Heller, 2020. "Rethinking the Benefits of Youth Employment Programs: The Heterogeneous Effects of Summer Jobs," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 664-677, October.
    9. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    10. Reinhold Kosfeld & Timo Mitze & Johannes Rode & Klaus Wälde, 2021. "The Covid‐19 containment effects of public health measures: A spatial difference‐in‐differences approach," Journal of Regional Science, Wiley Blackwell, vol. 61(4), pages 799-825, September.
    11. Kevin Credit, 2018. "Transit-oriented economic development: The impact of light rail on new business starts in the Phoenix, AZ Region, USA," Urban Studies, Urban Studies Journal Limited, vol. 55(13), pages 2838-2862, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    2. Hiroyuki Egami & Md. Shafiur Rahman & Tsuyoshi Yamamoto & Chihiro Egami & Takahisa Wakabayashi, 2024. "Causal effect of video gaming on mental well-being in Japan 2020–2022," Nature Human Behaviour, Nature, vol. 8(10), pages 1943-1956, October.
    3. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    4. Shonosuke Sugasawa & Hisashi Noma, 2021. "Efficient screening of predictive biomarkers for individual treatment selection," Biometrics, The International Biometric Society, vol. 77(1), pages 249-257, March.
    5. Samuel de Haas & Georg Goetz & Sven Heim, 2021. "Measuring the effects of COVID-19-related night curfews: Empirical evidence from Germany," MAGKS Papers on Economics 202118, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    6. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    7. Black, Dan A. & Grogger, Jeffrey & Kirchmaier, Tom & Sanders, Koen, 2023. "Criminal charges, risk assessment and violent recidivism in cases of domestic abuse," LSE Research Online Documents on Economics 121374, London School of Economics and Political Science, LSE Library.
    8. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    9. Hodula, Martin & Melecký, Martin & Pfeifer, Lukáš & Szabo, Milan, 2023. "Cooling the mortgage loan market: The effect of borrower-based limits on new mortgage lending," Journal of International Money and Finance, Elsevier, vol. 132(C).
    10. Jiafeng Gu, 2021. "Effects of Patent Policy on Outputs and Commercialization of Academic Patents in China: A Spatial Difference-in-Differences Analysis," Sustainability, MDPI, vol. 13(23), pages 1-17, December.
    11. Kwan Ok Lee & Hyojung Lee, 2022. "Public responses to COVID‐19 case disclosure and their spatial implications," Journal of Regional Science, Wiley Blackwell, vol. 62(3), pages 732-756, June.
    12. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.
    13. Silke Hamann & Annekatrin Niebuhr & Duncan Roth & Georg Sieglen, 2023. "How does the Covid‐19 pandemic affect regional labor markets and why do large cities suffer most?," Journal of Regional Science, Wiley Blackwell, vol. 63(5), pages 1228-1250, November.
    14. Cooper, Daniel & Garga, Vaishali & Luengo-Prado, María José & Tang, Jenny, 2023. "The mitigating effect of masks on the spread of Covid-19," Economics & Human Biology, Elsevier, vol. 48(C).
    15. Yiyan Huang & Cheuk Hang Leung & Siyi Wang & Yijun Li & Qi Wu, 2024. "Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators," Papers 2402.18392, arXiv.org, revised Oct 2024.
    16. Manuel Denzer & Philipp Grunau, 2024. "The impacts of working from home on individual health and well-being," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 25(5), pages 743-762, July.
    17. Phillip Heiler & Michael C. Knaus, 2021. "Effect or Treatment Heterogeneity? Policy Evaluation with Aggregated and Disaggregated Treatments," Papers 2110.01427, arXiv.org, revised Aug 2023.
    18. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    19. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.
    20. Goller, Daniel & Harrer, Tamara & Lechner, Michael & Wolff, Joachim, 2021. "Active labour market policies for the long-term unemployed: New evidence from causal machine learning," Economics Working Paper Series 2108, University of St. Gallen, School of Economics and Political Science.

    More about this item

    Keywords

    Causal forest; Heterogeneous treatment effects; Machine learning; Causal inference; Spatial; CO2 emissions; Transit;
    All these keywords.

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jgeosy:v:26:y:2024:i:4:d:10.1007_s10109-023-00413-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.