IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v40y2024i2p564-580.html
   My bibliography  Save this article

Counterfactual reconciliation: Incorporating aggregation constraints for more accurate causal effect estimates

Author

Listed:
  • Cengiz, Doruk
  • Tekgüç, Hasan

Abstract

We extend the scope of the forecast reconciliation literature and use its tools in the context of causal inference. Researchers are interested in both the average treatment effect on the treated and treatment effect heterogeneity. We show that ex post correction of the counterfactual estimates using the aggregation constraints that stem from the hierarchical or grouped structure of the data is likely to yield more accurate estimates. Building on the geometric interpretation of forecast reconciliation, we provide additional insights into the exact factors determining the size of the accuracy improvement due to the reconciliation. We experiment with U.S. GDP and employment data. We find that the reconciled treatment effect estimates tend to be closer to the truth than the original (base) counterfactual estimates even in cases where the aggregation constraints are non-linear. Consistent with our theoretical expectations, improvement is greater when machine learning methods are used.

Suggested Citation

  • Cengiz, Doruk & Tekgüç, Hasan, 2024. "Counterfactual reconciliation: Incorporating aggregation constraints for more accurate causal effect estimates," International Journal of Forecasting, Elsevier, vol. 40(2), pages 564-580.
  • Handle: RePEc:eee:intfor:v:40:y:2024:i:2:p:564-580
    DOI: 10.1016/j.ijforecast.2022.08.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207022001200
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2022.08.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    2. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    3. Cl'ement de Chaisemartin & Xavier D'Haultf{oe}uille, 2020. "Difference-in-Differences Estimators of Intertemporal Treatment Effects," Papers 2007.04267, arXiv.org, revised Dec 2024.
    4. Jeon, Jooyoung & Panagiotelis, Anastasios & Petropoulos, Fotios, 2019. "Probabilistic forecast reconciliation with applications to wind power and electric load," European Journal of Operational Research, Elsevier, vol. 279(2), pages 364-379.
    5. Nikolay Doudchenko & Guido W. Imbens, 2016. "Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis," NBER Working Papers 22791, National Bureau of Economic Research, Inc.
    6. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2021. "Matrix Completion Methods for Causal Panel Data Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1716-1730, October.
    7. Daron Acemoglu & Suresh Naidu & Pascual Restrepo & James A. Robinson, 2019. "Democracy Does Cause Growth," Journal of Political Economy, University of Chicago Press, vol. 127(1), pages 47-100.
    8. Xu, Yiqing, 2017. "Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models," Political Analysis, Cambridge University Press, vol. 25(1), pages 57-76, January.
    9. Shanika L. Wickramasuriya & George Athanasopoulos & Rob J. Hyndman, 2019. "Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 804-819, April.
    10. Laurent Gobillon & Thierry Magnac, 2016. "Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 535-551, July.
    11. Panagiotelis, Anastasios & Athanasopoulos, George & Gamakumara, Puwasala & Hyndman, Rob J., 2021. "Forecast reconciliation: A geometric view with new insights on bias correction," International Journal of Forecasting, Elsevier, vol. 37(1), pages 343-359.
    12. Clément de Chaisemartin & Xavier D’Haultfœuille, 2023. "Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: a survey," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 1-30.
    13. Athanasopoulos, George & Ahmed, Roman A. & Hyndman, Rob J., 2009. "Hierarchical forecasts for Australian domestic tourism," International Journal of Forecasting, Elsevier, vol. 25(1), pages 146-166.
    14. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    15. Hollyman, Ross & Petropoulos, Fotios & Tipping, Michael E., 2021. "Understanding forecast reconciliation," European Journal of Operational Research, Elsevier, vol. 294(1), pages 149-160.
    16. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    17. N. Gregory Mankiw & David Romer & David N. Weil, 1992. "A Contribution to the Empirics of Economic Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(2), pages 407-437.
    18. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    19. Hyndman, Rob J. & Ahmed, Roman A. & Athanasopoulos, George & Shang, Han Lin, 2011. "Optimal combination forecasts for hierarchical time series," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2579-2589, September.
    20. Souhaib Ben Taieb & James W. Taylor & Rob J. Hyndman, 2021. "Hierarchical Probabilistic Forecasting of Electricity Demand With Smart Meter Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(533), pages 27-43, March.
    21. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    22. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cengiz, Doruk & Tekgüç, Hasan, 2022. "Counterfactual Reconciliation: Incorporating Aggregation Constraints For More Accurate Causal Effect Estimates," MPRA Paper 114478, University Library of Munich, Germany.
    2. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    3. Sviták, Jan & Tichem, Jan & Haasbeek, Stefan, 2021. "Price effects of search advertising restrictions," International Journal of Industrial Organization, Elsevier, vol. 77(C).
    4. Athanasopoulos, George & Hyndman, Rob J. & Kourentzes, Nikolaos & Panagiotelis, Anastasios, 2024. "Forecast reconciliation: A review," International Journal of Forecasting, Elsevier, vol. 40(2), pages 430-456.
    5. Di Fonzo, Tommaso & Girolimetto, Daniele, 2024. "Forecast combination-based forecast reconciliation: Insights and extensions," International Journal of Forecasting, Elsevier, vol. 40(2), pages 490-514.
    6. Samuel Verevis & Murat Üngör, 2021. "What has New Zealand gained from The FTA with China?: Two counterfactual analyses†," Scottish Journal of Political Economy, Scottish Economic Society, vol. 68(1), pages 20-50, February.
    7. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    8. Irene Botosaru & Bruno Ferman, 2019. "On the role of covariates in the synthetic control method," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 117-130.
    9. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    10. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    11. Giulio Grossi & Marco Mariani & Alessandra Mattei & Patrizia Lattarulo & Ozge Oner, 2020. "Direct and spillover effects of a new tramway line on the commercial vitality of peripheral streets. A synthetic-control approach," Papers 2004.05027, arXiv.org, revised Nov 2023.
    12. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    13. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2021. "Matrix Completion Methods for Causal Panel Data Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1716-1730, October.
    14. Zhang, Bohan & Panagiotelis, Anastasios & Kang, Yanfei, 2024. "Discrete forecast reconciliation," European Journal of Operational Research, Elsevier, vol. 318(1), pages 143-153.
    15. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    16. Di Fonzo, Tommaso & Girolimetto, Daniele, 2023. "Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives," International Journal of Forecasting, Elsevier, vol. 39(1), pages 39-57.
    17. Girolimetto, Daniele & Athanasopoulos, George & Di Fonzo, Tommaso & Hyndman, Rob J., 2024. "Cross-temporal probabilistic forecast reconciliation: Methodological and practical issues," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1134-1151.
    18. Møller, Jan Kloppenborg & Nystrup, Peter & Madsen, Henrik, 2024. "Likelihood-based inference in temporal hierarchies," International Journal of Forecasting, Elsevier, vol. 40(2), pages 515-531.
    19. Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org, revised Aug 2022.
    20. Li, Han & Hyndman, Rob J., 2021. "Assessing mortality inequality in the U.S.: What can be said about the future?," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 152-162.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:40:y:2024:i:2:p:564-580. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.