IDEAS home Printed from https://ideas.repec.org/p/spa/wpaper/2024wpecon16.html
   My bibliography  Save this paper

On “Imputation of Counterfactual Outcomes when the Errors are Predictable'': Discussions on Misspecification and Suggestions of Sensitivity Analyses

Author

Listed:
  • Luis A. F. Alvarez
  • Bruno Ferman

Abstract

Gonçalves and Ng (2024) propose an interesting and simple way to improve counterfactual imputation methods when errors are predictable. For unconditional analyses, this approach yields smaller mean-squared error and tighter prediction intervals in large samples, even if the dependence of the errors is misspecified. For conditional analyses, this approach corrects the bias of standard methods, and provides valid asymptotic inference, if the dependence of the errors is correctly specified. In this comment, we first discuss how the assumptions imposed on the errors depend on the model and estimator adopted. This enables researchers to assess the validity of the assumptions imposed on the structure of the errors, and the relevant information set for conditional analyses. We then propose a simple sensitivity analysis in order to quantify the amount of misspecification on the dependence structure of the errors required for the conclusions of conditional analyses to be changed.

Suggested Citation

  • Luis A. F. Alvarez & Bruno Ferman, 2024. "On “Imputation of Counterfactual Outcomes when the Errors are Predictable'': Discussions on Misspecification and Suggestions of Sensitivity Analyses," Working Papers, Department of Economics 2024_16, University of São Paulo (FEA-USP).
  • Handle: RePEc:spa:wpaper:2024wpecon16
    as

    Download full text from publisher

    File URL: http://www.repec.eae.fea.usp.br/documentos/Alvarez_Ferman_16WP.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    2. Bruno Ferman, 2021. "On the Properties of the Synthetic Control Estimator with Many Periods and Many Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1764-1772, October.
    3. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    4. Luis Alvarez & Bruno Ferman, 2020. "Inference in Difference-in-Differences with Few Treated Units and Spatial Correlation," Papers 2006.16997, arXiv.org, revised Apr 2023.
    5. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alberto Abadie & Jaume Vives-i-Bastida, 2022. "Synthetic Controls in Action," Papers 2203.06279, arXiv.org.
    2. Fry, Joseph, 2024. "A method of moments approach to asymptotically unbiased Synthetic Controls," Journal of Econometrics, Elsevier, vol. 244(1).
    3. Luis Alvarez & Bruno Ferman, 2023. "Extensions for Inference in Difference-in-Differences with Few Treated Clusters," Papers 2302.03131, arXiv.org.
    4. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    5. Stefano, Roberta di & Mellace, Giovanni, 2020. "The inclusive synthetic control method," Discussion Papers on Economics 14/2020, University of Southern Denmark, Department of Economics.
    6. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    7. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    8. Ignacio Martinez & Jaume Vives-i-Bastida, 2022. "Bayesian and Frequentist Inference for Synthetic Controls," Papers 2206.01779, arXiv.org, revised Jul 2024.
    9. Ferman, Bruno, 2021. "Matching estimators with few treated and many control observations," Journal of Econometrics, Elsevier, vol. 225(2), pages 295-307.
    10. Alberto Abadie & Jinglong Zhao, 2021. "Synthetic Controls for Experimental Design," Papers 2108.02196, arXiv.org, revised Sep 2024.
    11. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    12. Guido W. Imbens & Davide Viviano, 2023. "Identification and Inference for Synthetic Controls with Confounding," Papers 2312.00955, arXiv.org.
    13. Lu Zhang & Xiaomeng Zhang & Xinyu Zhang, 2024. "Asymptotic Properties of the Distributional Synthetic Controls," Papers 2405.00953, arXiv.org, revised Aug 2024.
    14. Zongwu Cai & Ying Fang & Ming Lin & Zixuan Wu, 2023. "A Quasi Synthetic Control Method for Nonlinear Models With High-Dimensional Covariates," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202305, University of Kansas, Department of Economics, revised Aug 2023.
    15. Joseph Fry, 2023. "A Method of Moments Approach to Asymptotically Unbiased Synthetic Controls," Papers 2312.01209, arXiv.org, revised Mar 2024.
    16. Nuno Garoupa & Rok Spruk, 2024. "Populist Constitutional Backsliding and Judicial Independence: Evidence from Turkiye," Papers 2410.02439, arXiv.org.
    17. Li, Xingyu & Shen, Yan & Zhou, Qiankun, 2024. "Confidence intervals of treatment effects in panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 240(1).
    18. Robert Messerle & Jonas Schreyögg, 2024. "Country-level effects of diagnosis-related groups: evidence from Germany’s comprehensive reform of hospital payments," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 25(6), pages 1013-1030, August.
    19. Cummins Joseph & Miller Douglas L. & Smith Brock & Simon David, 2024. "Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator," Journal of Econometric Methods, De Gruyter, vol. 13(1), pages 67-95, January.
    20. Eli Ben‐Michael & Avi Feller & Jesse Rothstein, 2022. "Synthetic controls with staggered adoption," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 351-381, April.

    More about this item

    Keywords

    treatment effect; synthetic control; sensitivity analysis;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spa:wpaper:2024wpecon16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Pedro Garcia Duarte The email address of this maintainer does not seem to be valid anymore. Please ask Pedro Garcia Duarte to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/deuspbr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.