IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v210y2019i2p379-404.html
   My bibliography  Save this article

Specification tests for the propensity score

Author

Listed:
  • Sant’Anna, Pedro H.C.
  • Song, Xiaojun

Abstract

This paper proposes new nonparametric diagnostic tools to assess the asymptotic validity of different treatment effects estimators that rely on the correct specification of the propensity score. We derive a particular restriction relating the propensity score distribution of treated and control groups, and develop specification tests based upon it. The resulting tests do not suffer from the “curse of dimensionality” when the vector of covariates is high-dimensional, are fully data-driven, do not require tuning parameters such as bandwidths, and are able to detect a broad class of local alternatives converging to the null at the parametric rate n−1∕2, with n the sample size. We show that the use of an orthogonal projection on the tangent space of nuisance parameters facilitates the simulation of critical values by means of a multiplier bootstrap procedure, and can lead to power gains. The finite sample performance of the tests is examined by means of a Monte Carlo experiment and an empirical application. Open-source software is available for implementing the proposed tests.

Suggested Citation

  • Sant’Anna, Pedro H.C. & Song, Xiaojun, 2019. "Specification tests for the propensity score," Journal of Econometrics, Elsevier, vol. 210(2), pages 379-404.
    • Handle: RePEc:eee:econom:v:210:y:2019:i:2:p:379-404
    DOI: 10.1016/j.jeconom.2019.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407619300272
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2019.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Escanciano, J. Carlos, 2006. "A Consistent Diagnostic Test For Regression Models Using Projections," Econometric Theory, Cambridge University Press, vol. 22(6), pages 1030-1051, December.
    3. Manuel A. Domínguez & Ignacio N. Lobato, 2004. "Consistent Estimation of Models Defined by Conditional Moment Restrictions," Econometrica, Econometric Society, vol. 72(5), pages 1601-1615, September.
    4. Escanciano, J.C. & Goh, S.C., 2014. "Specification analysis of linear quantile models," Journal of Econometrics, Elsevier, vol. 178(P3), pages 495-507.
    5. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    6. Wenceslao González-Manteiga & Rosa Crujeiras, 2013. "Rejoinder on: An updated review of Goodness-of-Fit tests for regression models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 22(3), pages 442-447, September.
    7. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    8. Rose, Andrew K., 2004. "Do WTO members have more liberal trade policy?," Journal of International Economics, Elsevier, vol. 63(2), pages 209-235, July.
    9. Wenceslao González-Manteiga & Rosa Crujeiras, 2013. "An updated review of Goodness-of-Fit tests for regression models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 22(3), pages 361-411, September.
    10. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    11. Escanciano, J. Carlos, 2009. "On The Lack Of Power Of Omnibus Specification Tests," Econometric Theory, Cambridge University Press, vol. 25(1), pages 162-194, February.
    12. Wooldridge, Jeffrey M., 2007. "Inverse probability weighted estimation for general missing data problems," Journal of Econometrics, Elsevier, vol. 141(2), pages 1281-1301, December.
    13. Herman J. Bierens & Werner Ploberger, 1997. "Asymptotic Theory of Integrated Conditional Moment Tests," Econometrica, Econometric Society, vol. 65(5), pages 1129-1152, September.
    14. Markus Frlich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
    15. Victor Chernozhukov & Juan Carlos Escanciano & Hidehiko Ichimura & Whitney K. Newey & James M. Robins, 2022. "Locally Robust Semiparametric Estimation," Econometrica, Econometric Society, vol. 90(4), pages 1501-1535, July.
    16. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
    17. Alberto Abadie & Guido W. Imbens, 2016. "Matching on the Estimated Propensity Score," Econometrica, Econometric Society, vol. 84, pages 781-807, March.
    18. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    19. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    20. Fortin, Nicole & Lemieux, Thomas & Firpo, Sergio, 2011. "Decomposition Methods in Economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 1, pages 1-102, Elsevier.
    21. Escanciano, J. Carlos, 2006. "Goodness-of-Fit Tests for Linear and Nonlinear Time Series Models," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 531-541, June.
    22. A. Belloni & V. Chernozhukov & I. Fernández‐Val & C. Hansen, 2017. "Program Evaluation and Causal Inference With High‐Dimensional Data," Econometrica, Econometric Society, vol. 85, pages 233-298, January.
    23. Leeb, Hannes & Pötscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(1), pages 21-59, February.
    24. Shakeeb Khan & Elie Tamer, 2010. "Irregular Identification, Support Conditions, and Inverse Weight Estimation," Econometrica, Econometric Society, vol. 78(6), pages 2021-2042, November.
    25. Jeffrey A. Frankel & Andrew K. Rose, 2005. "Is Trade Good or Bad for the Environment? Sorting Out the Causality," The Review of Economics and Statistics, MIT Press, vol. 87(1), pages 85-91, February.
    26. Fan, Yanqin & Li, Qi, 1996. "Consistent Model Specification Tests: Omitted Variables and Semiparametric Functional Forms," Econometrica, Econometric Society, vol. 64(4), pages 865-890, July.
    27. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    28. Donald, Stephen G. & Hsu, Yu-Chin, 2014. "Estimation and inference for distribution functions and quantile functions in treatment effect models," Journal of Econometrics, Elsevier, vol. 178(P3), pages 383-397.
    29. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    30. Shaikh, Azeem M. & Simonsen, Marianne & Vytlacil, Edward J. & Yildiz, Nese, 2009. "A specification test for the propensity score using its distribution conditional on participation," Journal of Econometrics, Elsevier, vol. 151(1), pages 33-46, July.
    31. Seung‐Hyun Hong, 2013. "Measuring The Effect Of Napster On Recorded Music Sales: Difference‐In‐Differences Estimates Under Compositional Changes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 297-324, March.
    32. Millimet, Daniel L. & Tchernis, Rusty, 2009. "On the Specification of Propensity Scores, With Applications to the Analysis of Trade Policies," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(3), pages 397-415.
    33. Bierens, Herman J, 1990. "A Consistent Conditional Moment Test of Functional Form," Econometrica, Econometric Society, vol. 58(6), pages 1443-1458, November.
    34. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    35. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    36. John Xu Zheng, 1996. "A consistent test of functional form via nonparametric estimation techniques," Journal of Econometrics, Elsevier, vol. 75(2), pages 263-289, December.
    37. Winfried Stute & Li‐Xing Zhu, 2002. "Model Checks for Generalized Linear Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 29(3), pages 535-545, September.
    38. Słoczyński, Tymon & Wooldridge, Jeffrey M., 2018. "A General Double Robustness Result For Estimating Average Treatment Effects," Econometric Theory, Cambridge University Press, vol. 34(1), pages 112-133, February.
    39. Lee, Sokbae & Seo, Myung Hwan & Shin, Youngki, 2011. "Testing for Threshold Effects in Regression Models," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 220-231.
    40. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70, Elsevier.
    41. Stinchcombe, Maxwell B. & White, Halbert, 1998. "Consistent Specification Testing With Nuisance Parameters Present Only Under The Alternative," Econometric Theory, Cambridge University Press, vol. 14(3), pages 295-325, June.
    42. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    43. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(2), pages 261-294.
    44. Li, Qi & Racine, Jeffrey S. & Wooldridge, Jeffrey M., 2009. "Efficient Estimation of Average Treatment Effects with Mixed Categorical and Continuous Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(2), pages 206-223.
    45. Michael G. Akritas & Ingrid Van Keilegom, 2001. "Non‐parametric Estimation of the Residual Distribution," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 28(3), pages 549-567, September.
    46. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    47. Wang-Sheng Lee, 2013. "Propensity score matching and variations on the balancing test," Empirical Economics, Springer, vol. 44(1), pages 47-80, February.
    48. Li, Q. & Wang, Suojin, 1998. "A simple consistent bootstrap test for a parametric regression function," Journal of Econometrics, Elsevier, vol. 87(1), pages 145-165, August.
    49. Delgado, Miguel A. & Stute, Winfried, 2008. "Distribution-free specification tests of conditional models," Journal of Econometrics, Elsevier, vol. 143(1), pages 37-55, March.
    50. Bierens, Herman J., 1982. "Consistent model specification tests," Journal of Econometrics, Elsevier, vol. 20(1), pages 105-134, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    2. Pereda-Fernández, Santiago, 2023. "Identification and estimation of triangular models with a binary treatment," Journal of Econometrics, Elsevier, vol. 234(2), pages 585-623.
    3. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    4. Liu, Sheng & Wang, Yukai, 2023. "Green innovation effect of pilot zones for green finance reform: Evidence of quasi natural experiment," Technological Forecasting and Social Change, Elsevier, vol. 186(PA).
    5. Hu, Zhi-An & Li, Jinghong & Nie, Zhuo, 2023. "Long Live friendship? The long-term impact of Soviet aid on Sino-Russian trade," Journal of Development Economics, Elsevier, vol. 164(C).
    6. Hans Fricke & Markus Frölich & Martin Huber & Michael Lechner, 2020. "Endogeneity and non‐response bias in treatment evaluation – nonparametric identification of causal effects by instruments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 481-504, August.
    7. Pedro H. C. Sant'Anna & Xiaojun Song, 2020. "Specification tests for generalized propensity scores using double projections," Papers 2003.13803, arXiv.org, revised Apr 2023.
    8. Frazier, David T. & Oka, Tatsushi & Zhu, Dan, 2019. "Indirect inference with a non-smooth criterion function," Journal of Econometrics, Elsevier, vol. 212(2), pages 623-645.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    2. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    3. Advani, Arun & Sloczynski, Tymon, 2013. "Mostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies," IZA Discussion Papers 7874, Institute of Labor Economics (IZA).
    4. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    5. Arun Advani & Toru Kitagawa & Tymon Słoczyński, 2019. "Mostly harmless simulations? Using Monte Carlo studies for estimator selection," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(6), pages 893-910, September.
    6. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    7. Lee, Ying-Ying, 2018. "Efficient propensity score regression estimators of multivalued treatment effects for the treated," Journal of Econometrics, Elsevier, vol. 204(2), pages 207-222.
    8. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2010. "How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score," IZA Discussion Papers 5268, Institute of Labor Economics (IZA).
    9. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    10. Flores, Carlos A. & Mitnik, Oscar A., 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," IZA Discussion Papers 4451, Institute of Labor Economics (IZA).
    11. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    12. Frölich, Markus & Huber, Martin & Wiesenfarth, Manuel, 2017. "The finite sample performance of semi- and non-parametric estimators for treatment effects and policy evaluation," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 91-102.
    13. Steven Lehrer & Gregory Kordas, 2013. "Matching using semiparametric propensity scores," Empirical Economics, Springer, vol. 44(1), pages 13-45, February.
    14. Brantly Callaway & Pedro H. C. Sant'Anna, 2018. "Difference-in-Differences with Multiple Time Periods and an Application on the Minimum Wage and Employment," DETU Working Papers 1804, Department of Economics, Temple University.
    15. Hugo Bodory & Lorenzo Camponovo & Martin Huber & Michael Lechner, 2020. "The Finite Sample Performance of Inference Methods for Propensity Score Matching and Weighting Estimators," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(1), pages 183-200, January.
    16. Shengfang Tang & Zongwu Cai & Ying Fang & Ming Lin, 2019. "Testing Unconfoundedness Assumption Using Auxiliary Variables," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 201905, University of Kansas, Department of Economics, revised Mar 2019.
    17. Michael C. Knaus, 2021. "A double machine learning approach to estimate the effects of musical practice on student’s skills," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 282-300, January.
    18. Wei Huang & Oliver Linton & Zheng Zhang, 2022. "A Unified Framework for Specification Tests of Continuous Treatment Effect Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(4), pages 1817-1830, October.
    19. Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
    20. Farrell, Max H., 2015. "Robust inference on average treatment effects with possibly more covariates than observations," Journal of Econometrics, Elsevier, vol. 189(1), pages 1-23.

    More about this item

    Keywords

    Empirical processes; Integrated moments; Multiplier bootstrap; Projection; Treatment effects;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • C35 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:210:y:2019:i:2:p:379-404. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.