IDEAS home Printed from https://ideas.repec.org/a/spr/empeco/v60y2021i1d10.1007_s00181-020-01978-1.html
   My bibliography  Save this article

Can a time-varying structure provide a more robust panel construction of counterfactuals-straitjacket or straitjackets?

Author

Listed:
  • Shui Ki Wan

    (Hong Kong Baptist University)

  • Cheng Hsiao

    (University of Southern California
    WISE, Xiamen University)

  • Qiankun Zhou

    (Louisiana State University)

Abstract

Measuring treatment effects are a complicated task as the outcomes of receiving and not receiving the treatment cannot be observed simultaneously. Thus, the issue of obtaining accurate measurement is an issue of predicting the counterfactuals accurately. In this study, we explore the suitability of using time-varying parameter models in a panel to generate robust measure of counterfactuals, hence robust measure of treatment effects. We suggest some within-sample tests for constant parameter versus time-varying parameter models and diagnostic tools based on time-varying parameter framework as a flexible alternative to predict missing data. Monte Carlos and two empirical studies are examined in this framework. The results appear to show that if the focus is on minimizing the mean square error of the predicted treatment effects, a “straitjacket” approach relying on the best selected model from the pre-treatment data remains the best option in view of the missing post-treatment information on counterfactuals. On the other hand, the confidence band based on the time-varying parameter model provides a more robust inference to hedge against possible changes of the relations between the treated units and the controls in the post-treatment period.

Suggested Citation

  • Shui Ki Wan & Cheng Hsiao & Qiankun Zhou, 2021. "Can a time-varying structure provide a more robust panel construction of counterfactuals-straitjacket or straitjackets?," Empirical Economics, Springer, vol. 60(1), pages 113-129, January.
    • Handle: RePEc:spr:empeco:v:60:y:2021:i:1:d:10.1007_s00181-020-01978-1
    DOI: 10.1007/s00181-020-01978-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00181-020-01978-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00181-020-01978-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Cheng Hsiao & H. Steve Ching & Shui Ki Wan, 2012. "A Panel Data Approach For Program Evaluation: Measuring The Benefits Of Political And Economic Integration Of Hong Kong With Mainland China," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(5), pages 705-740, August.
    2. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 71, Elsevier.
    3. Bai, Jushan, 1997. "Estimating Multiple Breaks One at a Time," Econometric Theory, Cambridge University Press, vol. 13(3), pages 315-352, June.
    4. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    5. Xu, Yiqing, 2017. "Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models," Political Analysis, Cambridge University Press, vol. 25(1), pages 57-76, January.
    6. Melissa Dell & Benjamin F. Jones & Benjamin A. Olken, 2014. "What Do We Learn from the Weather? The New Climate-Economy Literature," Journal of Economic Literature, American Economic Association, vol. 52(3), pages 740-798, September.
    7. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    8. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    9. Cheng Hsiao & Qiankun Zhou, 2019. "Panel parametric, semiparametric, and nonparametric construction of counterfactuals," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(4), pages 463-481, June.
    10. Hsiao, Cheng & Wan, Shui Ki, 2014. "Is there an optimal forecast combination?," Journal of Econometrics, Elsevier, vol. 178(P2), pages 294-309.
    11. Li, Kathleen T. & Bell, David R., 2017. "Estimation of average treatment effects with panel data: Asymptotic theory and implementation," Journal of Econometrics, Elsevier, vol. 197(1), pages 65-75.
    12. Wan, Shui-Ki & Xie, Yimeng & Hsiao, Cheng, 2018. "Panel data approach vs synthetic control method," Economics Letters, Elsevier, vol. 164(C), pages 121-123.
    13. Cooley, Thomas F & Prescott, Edward C, 1976. "Estimation in the Presence of Stochastic Parameter Variation," Econometrica, Econometric Society, vol. 44(1), pages 167-184, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Echevarría, Cruz A. & Hasancebi, Serhat & García-Enríquez, Javier, 2022. "Economic Effects of Macao’s Integration with Mainland China: A Causal Inference Study," Journal of Economic Integration, Center for Economic Integration, Sejong University, vol. 37(2), pages 179-215.
    2. Shi, Zhentao & Huang, Jingyi, 2023. "Forward-selected panel data approach for program evaluation," Journal of Econometrics, Elsevier, vol. 234(2), pages 512-535.
    3. Carvalho, Carlos & Masini, Ricardo & Medeiros, Marcelo C., 2018. "ArCo: An artificial counterfactual approach for high-dimensional panel time-series data," Journal of Econometrics, Elsevier, vol. 207(2), pages 352-380.
    4. Callaway, Brantly & Karami, Sonia, 2023. "Treatment effects in interactive fixed effects models with a small number of time periods," Journal of Econometrics, Elsevier, vol. 233(1), pages 184-208.
    5. Hongjun Li & Zheng Li & Cheng Hsiao, 2023. "Assessing the impacts of pandemic and the increase in minimum down payment rate on Shanghai housing prices," Empirical Economics, Springer, vol. 64(6), pages 2661-2682, June.
    6. Viviano, Davide & Bradic, Jelena, 2023. "Synthetic Learner: Model-free inference on treatments over time," Journal of Econometrics, Elsevier, vol. 234(2), pages 691-713.
    7. Li, Xingyu & Shen, Yan & Zhou, Qiankun, 2024. "Confidence intervals of treatment effects in panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 240(1).
    8. Xiong, Ruoxuan & Pelger, Markus, 2023. "Large dimensional latent factor modeling with missing observations and applications to causal inference," Journal of Econometrics, Elsevier, vol. 233(1), pages 271-301.
    9. Gharehgozli, Orkideh, 2021. "An empirical comparison between a regression framework and the Synthetic Control Method," The Quarterly Review of Economics and Finance, Elsevier, vol. 81(C), pages 70-81.
    10. Fu, Yifan & Zhong, Shiquan & Ling, Shuai & He, Zhengbing, 2024. "Closing the loophole of vehicle ownership restriction: The impact of non-local vehicle restriction on new vehicle registrations and air pollution," Transportation Research Part A: Policy and Practice, Elsevier, vol. 185(C).
    11. Pengyue Wu & Wenjing Xu & Jing Ma, 2024. "Policy Evolution and Effect Evaluation of Zhejiang Manufacturing Industry Based on Text Data," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(1), pages 2895-2932, March.
    12. Xiao Ke & Cheng Hsiao, 2022. "Economic impact of the most drastic lockdown during COVID‐19 pandemic—The experience of Hubei, China," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 187-209, January.
    13. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    14. Luya Wang & Jeffrey S. Racine & Qiaoyu Wang, 2024. "Bootstrap Inference on a Factor Model Based Average Treatment Effects Estimator," Department of Economics Working Papers 2024-03, McMaster University.
    15. Yi‐Ting Chen, 2020. "A distributional synthetic control method for policy evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 505-525, August.
    16. Wei Tian, 2023. "Individual Causal Inference Using Panel Data With Multiple Outcomes," Papers 2306.01969, arXiv.org.
    17. Domenico Depalo, 2020. "Explaining the causal effect of adherence to medication on cholesterol through the marginal patient," Health Economics, John Wiley & Sons, Ltd., vol. 29(S1), pages 110-126, October.
    18. Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org, revised Aug 2022.
    19. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    20. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:empeco:v:60:y:2021:i:1:d:10.1007_s00181-020-01978-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.