IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/20342.html
   My bibliography  Save this paper

Broken or Fixed Effects?

Author

Listed:
  • Charles E. Gibbons
  • Juan Carlos Suárez Serrato
  • Michael B. Urbancic

Abstract

We replicate eight influential papers to provide empirical evidence that, in the presence of heterogeneous treatment effects, OLS with fixed effects (FE) is generally not a consistent estimator of the average treatment effect (ATE). We propose two alternative estimators that recover the ATE in the presence of group-specific heterogeneity. We document that heterogeneous treatment effects are common and the ATE is often statistically and economically different from the FE estimate. In all but one of our replications, there is statistically significant treatment effect heterogeneity and, in six, the ATEs are either economically or statistically different from the FE estimates.

Suggested Citation

  • Charles E. Gibbons & Juan Carlos Suárez Serrato & Michael B. Urbancic, 2014. "Broken or Fixed Effects?," NBER Working Papers 20342, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:20342
    Note: LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w20342.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Marianne P. Bitler & Jonah B. Gelbach & Hilary W. Hoynes, 2006. "What Mean Impacts Miss: Distributional Effects of Welfare Reform Experiments," American Economic Review, American Economic Association, vol. 96(4), pages 988-1012, September.
    2. John Van Reenen & Rupert Harrison & Rachel Griffith, 2006. "How Special Is the Special Relationship? Using the Impact of U.S. R&D Spillovers on U.K. Firms as a Test of Technology Sourcing," American Economic Review, American Economic Association, vol. 96(5), pages 1859-1875, December.
    3. Kling, Jeffrey R, 2001. "Interpreting Instrumental Variables Estimates of the Returns to Schooling," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 358-364, July.
    4. Kelly Bedard & Olivier Deschênes, 2006. "The Long-Term Impact of Military Service on Health: Evidence from World War II and Korean War Veterans," American Economic Review, American Economic Association, vol. 96(1), pages 176-194, March.
    5. Kline Patrick & Santos Andres, 2012. "A Score Based Approach to Wild Bootstrap Inference," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 23-41, August.
    6. Papke, Leslie E., 1994. "Tax policy and urban development : Evidence from the Indiana enterprise zone program," Journal of Public Economics, Elsevier, vol. 54(1), pages 37-49, May.
    7. Jeffrey M. Wooldridge, 2005. "Fixed-Effects and Related Estimators for Correlated Random-Coefficient and Treatment-Effect Panel Data Models," The Review of Economics and Statistics, MIT Press, vol. 87(2), pages 385-390, May.
    8. Chamberlain, Gary, 1982. "Multivariate regression models for panel data," Journal of Econometrics, Elsevier, vol. 18(1), pages 5-46, January.
    9. Bryan S. Graham & James L. Powell, 2012. "Identification and Estimation of Average Partial Effects in “Irregular” Correlated Random Coefficient Panel Data Models," Econometrica, Econometric Society, vol. 80(5), pages 2105-2152, September.
    10. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    11. Leora Friedberg, 1998. "Did Unilateral Divorce Raise Divorce Rates? Evidence from Panel Data," NBER Working Papers 6398, National Bureau of Economic Research, Inc.
    12. Gary Solon & Steven J. Haider & Jeffrey M. Wooldridge, 2015. "What Are We Weighting For?," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 301-316.
    13. Abhijit Banerjee & Lakshmi Iyer, 2005. "History, Institutions, and Economic Performance: The Legacy of Colonial Land Tenure Systems in India," American Economic Review, American Economic Association, vol. 95(4), pages 1190-1213, September.
    14. Philip Oreopoulos, 2006. "Estimating Average and Local Average Treatment Effects of Education when Compulsory Schooling Laws Really Matter," American Economic Review, American Economic Association, vol. 96(1), pages 152-175, March.
    15. Murphy, Kevin M & Topel, Robert H, 2002. "Estimation and Inference in Two-Step Econometric Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 88-97, January.
    16. Lance Lochner & Enrico Moretti, 2004. "The Effect of Education on Crime: Evidence from Prison Inmates, Arrests, and Self-Reports," American Economic Review, American Economic Association, vol. 94(1), pages 155-189, March.
    17. Dean S. Karlan & Jonathan Zinman, 2008. "Credit Elasticities in Less-Developed Economies: Implications for Microfinance," American Economic Review, American Economic Association, vol. 98(3), pages 1040-1068, June.
    18. Wooldridge, Jeffrey M., 1997. "On two stage least squares estimation of the average treatment effect in a random coefficient model," Economics Letters, Elsevier, vol. 56(2), pages 129-133, October.
    19. Marianne P. Bitler & Jonah B. Gelbach & Hilary W. Hoynes, 2017. "Can Variation in Subgroups' Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment," The Review of Economics and Statistics, MIT Press, vol. 99(4), pages 683-697, July.
    20. Ghanem, Dalia, 2017. "Testing identifying assumptions in nonseparable panel data models," Journal of Econometrics, Elsevier, vol. 197(2), pages 202-217.
    21. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, April.
    22. Chamberlain, Gary, 1992. "Efficiency Bounds for Semiparametric Regression," Econometrica, Econometric Society, vol. 60(3), pages 567-596, May.
    23. Angrist, Joshua D. & Krueger, Alan B., 1999. "Empirical strategies in labor economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 23, pages 1277-1366, Elsevier.
    24. Lance Lochner & Enrico Moretti, 2011. "Estimating and Testing Non-Linear Models Using Instrumental Variables," University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers 20112, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
    25. David Card & Carlos Dobkin & Nicole Maestas, 2008. "The Impact of Nearly Universal Insurance Coverage on Health Care Utilization: Evidence from Medicare," American Economic Review, American Economic Association, vol. 98(5), pages 2242-2258, December.
    26. Victor Chernozhukov & Iván Fernández‐Val & Jinyong Hahn & Whitney Newey, 2013. "Average and Quantile Effects in Nonseparable Panel Models," Econometrica, Econometric Society, vol. 81(2), pages 535-580, March.
    27. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jeanine Miklós-Thal & Hannes Ullrich, 2016. "Career Prospects and Effort Incentives: Evidence from Professional Soccer," Management Science, INFORMS, vol. 62(6), pages 1645-1667, June.
    2. Kurt Schmidheiny & Sebastian Siegloch, 2023. "On event studies and distributed‐lags in two‐way fixed effects models: Identification, equivalence, and generalization," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 695-713, August.
    3. Daniel Garrett & Andrey Ordin & James W Roberts & Juan Carlos Suárez Serrato, 2023. "Tax Advantages and Imperfect Competition in Auctions for Municipal Bonds," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(2), pages 815-851.
    4. Suárez Serrato, Juan Carlos & Zidar, Owen, 2018. "The structure of state corporate taxation and its impact on state tax revenues and economic activity," Journal of Public Economics, Elsevier, vol. 167(C), pages 158-176.
    5. Pedro Portugal, 2020. "The sources of wage variability in Portugal: a binge reading survey," Economic Bulletin and Financial Stability Report Articles and Banco de Portugal Economic Studies, Banco de Portugal, Economics and Research Department.
    6. Navdeep S. Sahni & Dan Zou & Pradeep K. Chintagunta, 2017. "Do Targeted Discount Offers Serve as Advertising? Evidence from 70 Field Experiments," Management Science, INFORMS, vol. 63(8), pages 2688-2705, August.
    7. Leandro D’Aurizio & Domenico Depalo, 2016. "An Evaluation of the Policies on Repayment of Government’s Trade Debt in Italy," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 2(2), pages 167-196, July.
    8. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    9. Słoczyński, Tymon, 2012. "New Evidence on Linear Regression and Treatment Effect Heterogeneity," MPRA Paper 39524, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dalia Ghanem & Pedro H. C. Sant'Anna & Kaspar Wüthrich, 2022. "Selection and Parallel Trends," CESifo Working Paper Series 9910, CESifo.
    2. Słoczyński, Tymon, 2012. "New Evidence on Linear Regression and Treatment Effect Heterogeneity," MPRA Paper 39524, University Library of Munich, Germany.
    3. Michael Bates & Seolah Kim, 2024. "Estimating the price elasticity of gasoline demand in correlated random coefficient models with endogeneity," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(4), pages 679-696, June.
    4. Valentin Verdier, 2020. "Average treatment effects for stayers with correlated random coefficient models of panel data," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(7), pages 917-939, November.
    5. Sylvie Charlot & Riccardo Crescenzi & Antonio Musolesi, 2014. "Augmented and Unconstrained: revisiting the Regional Knowledge Production Function," SEEDS Working Papers 2414, SEEDS, Sustainability Environmental Economics and Dynamics Studies, revised Aug 2014.
    6. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    7. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    8. Tymon S{l}oczy'nski, 2018. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," Papers 1810.01576, arXiv.org, revised May 2020.
    9. Musolesi Antonio & Mazzanti Massimiliano, 2014. "Nonlinearity, heterogeneity and unobserved effects in the carbon dioxide emissions-economic development relation for advanced countries," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(5), pages 521-541, December.
    10. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    11. Irene Botosaru & Chris Muris, 2017. "Binarization for panel models with fixed effects," CeMMAP working papers 31/17, Institute for Fiscal Studies.
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Graham, Bryan S. & Hahn, Jinyong & Poirier, Alexandre & Powell, James L., 2018. "A quantile correlated random coefficients panel data model," Journal of Econometrics, Elsevier, vol. 206(2), pages 305-335.
    14. Bryan S. Graham & Jinyong Hahn & Alexandre Poirier & James L. Powell, 2015. "Quantile Regression with Panel Data," NBER Working Papers 21034, National Bureau of Economic Research, Inc.
    15. Cl'ement de Chaisemartin & Xavier D'Haultf{oe}uille, 2021. "Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A Survey," Papers 2112.04565, arXiv.org, revised Jun 2022.
    16. Michael Bates & Seolah Kim, 2019. "Per-Cluster Instrumental Variables Estimation: Uncovering the Price Elasticity of the Demand for Gasoline," Working Papers 202003, University of California at Riverside, Department of Economics.
    17. A. Belloni & V. Chernozhukov & I. Fernández‐Val & C. Hansen, 2017. "Program Evaluation and Causal Inference With High‐Dimensional Data," Econometrica, Econometric Society, vol. 85, pages 233-298, January.
    18. Card, David, 2001. "Estimating the Return to Schooling: Progress on Some Persistent Econometric Problems," Econometrica, Econometric Society, vol. 69(5), pages 1127-1160, September.
    19. Xiao, Zhijie & Xu, Lan, 2019. "What do mean impacts miss? Distributional effects of corporate diversification," Journal of Econometrics, Elsevier, vol. 213(1), pages 92-120.
    20. Louise Laage, 2020. "A Correlated Random Coefficient Panel Model with Time-Varying Endogeneity," Papers 2003.09367, arXiv.org, revised Nov 2022.

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:20342. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.