IDEAS home Printed from https://ideas.repec.org/p/cdp/texdis/td662.html
   My bibliography  Save this paper

Evaluating difference-in-differences models under different treatment assignment mechanism and in the presence of spillover effects

Author

Listed:
  • Guilherme Araújo Lima

    (UFMG)

  • Igor Viveiros Melo Souza

    (UFMG)

  • Mauro Sayar Ferreira

    (UFMG)

Abstract

We conduct Monte Carlo experiments to evaluate the performance of different Difference-in-Differences estimators under treatment assignment mechanisms affected by shocks suffered by treated units and also in contexts where the treatment effect spills over to units in the control group. In particular, we compare the estimators proposed by Callaway and Sant'Anna (2021), Borusyak et al. (2021), and Sun and Abraham (2021), as well as the two-way fixed effects (TWFE) estimator. The results demonstrate that the treatment assignment mechanisms we design, and the presence of spillover effects can severely compromise the performance of the considered estimators, leading to bias and, even more importantly, inconsistency. Therefore, cautious for interpreting the results should be taken in applications where the environment studied resembles those we consider. The development of more robust estimators is a necessity and a prosperous research venue.

Suggested Citation

  • Guilherme Araújo Lima & Igor Viveiros Melo Souza & Mauro Sayar Ferreira, 2023. "Evaluating difference-in-differences models under different treatment assignment mechanism and in the presence of spillover effects," Textos para Discussão Cedeplar-UFMG 662, Cedeplar, Universidade Federal de Minas Gerais.
    • Handle: RePEc:cdp:texdis:td662
    as

    Download full text from publisher

    File URL: https://www.cedeplar.ufmg.br/pesquisas/td/TD%20662.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    2. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    3. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    4. Michelle Marcus & Pedro H. C. Sant’Anna, 2021. "The Role of Parallel Trends in Event Study Settings: An Application to Environmental Economics," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 8(2), pages 235-275.
    5. Clément de Chaisemartin & Xavier D’Haultfœuille, 2023. "Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: a survey," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 1-30.
    6. Lechner, Michael, 2011. "The Estimation of Causal Effects by Difference-in-Difference Methods," Foundations and Trends(R) in Econometrics, now publishers, vol. 4(3), pages 165-224, November.
    7. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    8. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    9. John Gardner, 2022. "Two-stage differences in differences," Papers 2207.05943, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dalia Ghanem & Pedro H. C. Sant'Anna & Kaspar Wüthrich, 2022. "Selection and Parallel Trends," CESifo Working Paper Series 9910, CESifo.
    2. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    3. Gregory Faletto, 2023. "Fused Extended Two-Way Fixed Effects for Difference-in-Differences with Staggered Adoptions," Papers 2312.05985, arXiv.org, revised Apr 2024.
    4. Alex Hollingsworth & Krzysztof Karbownik & Melissa A. Thomasson & Anthony Wray, 2022. "The Gift of a Lifetime: The Hospital, Modern Medicine, and Mortality," NBER Working Papers 30663, National Bureau of Economic Research, Inc.
    5. Blemings, Benjamin & Zhang, Peilu & Neill, Clinton L., 2023. "Where is the value? The impacts of sow gestation crate laws on pork supply and consumer value perceptions," Food Policy, Elsevier, vol. 117(C).
    6. Mayberry, Anthony A., 2023. "Demilitarization and economic growth: Empirical evidence in support of a peace dividend," Journal of Comparative Economics, Elsevier, vol. 51(3), pages 960-988.
    7. HOSHI, Kisho, 2023. "New Medical Schools, Access to Doctors and Health Outcomes : Evidence from Japan," CEI Working Paper Series 2023-05, Center for Economic Institutions, Institute of Economic Research, Hitotsubashi University.
    8. Kurt Schmidheiny & Sebastian Siegloch, 2023. "On event studies and distributed‐lags in two‐way fixed effects models: Identification, equivalence, and generalization," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 695-713, August.
    9. Bhattacharjee, Arnab & Aravena, Claudia & Castillo, Natalia & Ehrlich, Marco & Taou, Nadia & Wagner, Thomas, 2022. "Agroforestry Programs in the Colombian Amazon: Selection, Treatment and Exposure Effects on Deforestation," National Institute of Economic and Social Research (NIESR) Discussion Papers 537, National Institute of Economic and Social Research.
    10. Simon Freyaldenhoven & Christian Hansen & Jorge Pérez Pérez & Jesse M. Shapiro, 2021. "Visualization, Identification, and Estimation in the Linear Panel Event-Study Design," NBER Working Papers 29170, National Bureau of Economic Research, Inc.
    11. Clément de Chaisemartin & Xavier D’Haultfœuille, 2023. "Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: a survey," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 1-30.
    12. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    13. Hufschmidt, Patrick, 2023. "Green parties and building permissions: Evidence from Bavarian municipalities," Ruhr Economic Papers 1052, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    14. Chareyron, Sylvain & Goffette-Nagot, Florence & Letrouit, Lucie, 2022. "Externalities from urban renewal: Evidence from a French program," Regional Science and Urban Economics, Elsevier, vol. 95(C).
    15. de Chaisemartin, Clément & D’Haultfœuille, Xavier, 2023. "Two-way fixed effects and differences-in-differences estimators with several treatments," Journal of Econometrics, Elsevier, vol. 236(2).
    16. Rösner, Anja & Haucap, Justus & Heimeshoff, Ulrich, 2020. "The impact of consumer protection in the digital age: Evidence from the European Union," International Journal of Industrial Organization, Elsevier, vol. 73(C).
    17. Arindrajit Dube & Daniele Girardi & Òscar Jordà & Alan M. Taylor, 2023. "A Local Projections Approach to Difference-in-Differences Event Studies," Working Paper Series 2023-12, Federal Reserve Bank of San Francisco.
    18. Rotunno, Lorenzo & Roy, Sanchari & Sakakibara, Anri & Vezina, Pierre-Louis, 2023. "Trade Policy and Jobs in Vietnam: The Unintended Consequences of Trump’s Trade War," SocArXiv 9rdne, Center for Open Science.
    19. Jeffrey M Wooldridge, 2023. "Simple approaches to nonlinear difference-in-differences with panel data," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 31-66.
    20. Nieto, Adrián, 2022. "Can subsidies to permanent employment change fertility decisions?," Labour Economics, Elsevier, vol. 78(C).

    More about this item

    Keywords

    Difference-in-Differences; Causal Inference; Treatment assignment mechanisms; Spillover effects.;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cdp:texdis:td662. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Gustavo Britto (email available below). General contact details of provider: https://edirc.repec.org/data/pufmgbr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.