IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0343.html
   My bibliography  Save this paper

Do Instrumental Variables Belong in Propensity Scores?

Author

Listed:
  • Jay Bhattacharya
  • William B. Vogt

Abstract

Propensity score matching is a popular way to make causal inferences about a binary treatment in observational data. The validity of these methods depends on which variables are used to predict the propensity score. We ask: "Absent strong ignorability, what would be the effect of including an instrumental variable in the predictor set of a propensity score matching estimator?" In the case of linear adjustment, using an instrumental variable as a predictor variable for the propensity score yields greater inconsistency than the naive estimator. This additional inconsistency is increasing in the predictive power of the instrument. In the case of stratification, with a strong instrument, propensity score matching yields greater inconsistency than the naive estimator. Since the propensity score matching estimator with the instrument in the predictor set is both more biased and more variable than the naive estimator, it is conceivable that the confidence intervals for the matching estimator would have greater coverage rates. In a Monte Carlo simulation, we show that this need not be the case. Our results are further illustrated with two empirical examples: one, the Tennessee STAR experiment, with a strong instrument and the other, the Connors' (1996) Swan-Ganz catheterization dataset, with a weak instrument.

Suggested Citation

  • Jay Bhattacharya & William B. Vogt, 2007. "Do Instrumental Variables Belong in Propensity Scores?," NBER Technical Working Papers 0343, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberte:0343
    Note: TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0343.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    2. Heckman, James J & Honore, Bo E, 1990. "The Empirical Content of the Roy Model," Econometrica, Econometric Society, vol. 58(5), pages 1121-1149, September.
    3. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    4. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    5. Bhattacharya, Jay & Shaikh, Azeem M. & Vytlacil, Edward, 2012. "Treatment effect bounds: An application to Swan–Ganz catheterization," Journal of Econometrics, Elsevier, vol. 168(2), pages 223-243.
    6. James Heckman, 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 441-462.
    7. Joshua D. Angrist & Guido W. Imbens & D.B. Rubin, 1993. "Identification of Causal Effects Using Instrumental Variables," NBER Technical Working Papers 0136, National Bureau of Economic Research, Inc.
    8. Christopher Taber & Hidehiko Ichimura, 2001. "Propensity-Score Matching with Instrumental Variables," American Economic Review, American Economic Association, vol. 91(2), pages 119-124, May.
    9. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chabé-Ferret, Sylvain, 2015. "Analysis of the bias of Matching and Difference-in-Difference under alternative earnings and selection processes," Journal of Econometrics, Elsevier, vol. 185(1), pages 110-123.
    2. Ay, Jean-Sauveur & Le Gallo, Julie, 2021. "The Signaling Values of Nested Wine Names," Working Papers 321851, American Association of Wine Economists.
    3. Deter, Max & van Hoorn, André, 2023. "Selection, socialization, and risk preferences in the finance industry: Longitudinal evidence for German finance professionals," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    4. Rebecca Riley & Hilary Metcalf & John Forth, 2013. "The business case for equal opportunities," Industrial Relations Journal, Wiley Blackwell, vol. 44(3), pages 216-239, May.
    5. Wyss Richard & Lunt Mark & Brookhart M. Alan & Glynn Robert J. & Stürmer Til, 2014. "Reducing Bias Amplification in the Presence of Unmeasured Confounding through Out-of-Sample Estimation Strategies for the Disease Risk Score," Journal of Causal Inference, De Gruyter, vol. 2(2), pages 131-146, September.
    6. Steiner Peter M. & Kim Yongnam, 2016. "The Mechanics of Omitted Variable Bias: Bias Amplification and Cancellation of Offsetting Biases," Journal of Causal Inference, De Gruyter, vol. 4(2), pages 1-22, September.
    7. Bisakha Sen & Stephen Mennemeyer & Lisa C. Gary, 2009. "The Relationship Between Neighborhood Quality and Obesity Among Children," NBER Working Papers 14985, National Bureau of Economic Research, Inc.
    8. Rebecca Riley & Hilary Metcalf & John Forth, 2013. "The business case for equal opportunities," Industrial Relations Journal, Wiley Blackwell, vol. 44(3), pages 216-239, May.
    9. Chabé-Ferret, Sylvain, 2012. "Matching vs Differencing when Estimating Treatment Effects with Panel Data: the Example of the Effect of Job Training Programs on Earnings," TSE Working Papers 12-356, Toulouse School of Economics (TSE).
    10. Steven Lawry & Cyrus Samii & Ruth Hall & Aaron Leopold & Donna Hornby & Farai Mtero, 2014. "The Impact of Land Property Rights Interventions on Investment and Agricultural Productivity in Developing Countries: a Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 10(1), pages 1-104.
    11. Pearl Judea, 2013. "Linear Models: A Useful “Microscope” for Causal Analysis," Journal of Causal Inference, De Gruyter, vol. 1(1), pages 155-170, June.
    12. Bryan Keller, 2020. "Variable Selection for Causal Effect Estimation: Nonparametric Conditional Independence Testing With Random Forests," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 119-142, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    2. James J. Heckman, 2008. "The Principles Underlying Evaluation Estimators with an Application to Matching," Annals of Economics and Statistics, GENES, issue 91-92, pages 9-73.
    3. Aakvik, Arild & Heckman, James J. & Vytlacil, Edward J., 2005. "Estimating treatment effects for discrete outcomes when responses to treatment vary: an application to Norwegian vocational rehabilitation programs," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 15-51.
    4. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    5. Cockx, Bart & Bardoulat, Isabelle, 1999. "Vocational Training: Does it speed up the Transition Rate out of Unemployment ?," LIDAM Discussion Papers IRES 1999032, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    6. Heckman, James J., 2010. "The Assumptions Underlying Evaluation Estimators," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 30(2), December.
    7. Anirban Basu & James J. Heckman & Salvador Navarro‐Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self‐selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157, November.
    8. Aakvik, Arild & Salvanes, Kjell G. & Vaage, Kjell, 2003. "Measuring Heterogeneity in the Returns to Education in Norway Using Educational Reforms," IZA Discussion Papers 815, Institute of Labor Economics (IZA).
    9. P. Lovaglio & S. Verzillo, 2016. "Heterogeneous economic returns to higher education: evidence from Italy," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(2), pages 791-822, March.
    10. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    11. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    12. Lina Zhang & David T. Frazier & D. S. Poskitt & Xueyan Zhao, 2020. "Decomposing Identification Gains and Evaluating Instrument Identification Power for Partially Identified Average Treatment Effects," Papers 2009.02642, arXiv.org, revised Sep 2022.
    13. Stefan Boes, 2009. "Partial Identification of Discrete Counterfactual Distributions with Sequential Update of Information," SOI - Working Papers 0918, Socioeconomic Institute - University of Zurich.
    14. Thomas Brodaty & Bruno Crépon & Denis Fougère, 2007. "Les méthodes micro-économétriques d'évaluation et leurs applications aux politiques actives de l'emploi," Economie & Prévision, La Documentation Française, vol. 0(1), pages 93-118.
    15. Baum-Snow, Nathaniel & Ferreira, Fernando, 2015. "Causal Inference in Urban and Regional Economics," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 3-68, Elsevier.
    16. Arthur Lewbel, 2019. "The Identification Zoo: Meanings of Identification in Econometrics," Journal of Economic Literature, American Economic Association, vol. 57(4), pages 835-903, December.
    17. Heckman, James J. & Pinto, Rodrigo, 2022. "Causality and Econometrics," IZA Discussion Papers 15081, Institute of Labor Economics (IZA).
    18. Heckman, James J. & Vytlacil, Edward J., 2000. "The relationship between treatment parameters within a latent variable framework," Economics Letters, Elsevier, vol. 66(1), pages 33-39, January.
    19. Guilhem Bascle, 2008. "Controlling for endogeneity with instrumental variables in strategic management research," Post-Print hal-00576795, HAL.
    20. Chen, Heng & Fan, Yanqin & Wu, Jisong, 2014. "A flexible parametric approach for estimating switching regime models and treatment effect parameters," Journal of Econometrics, Elsevier, vol. 181(2), pages 77-91.

    More about this item

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • I1 - Health, Education, and Welfare - - Health
    • I2 - Health, Education, and Welfare - - Education

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0343. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.