IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0185.html
   My bibliography  Save this paper

Instrumental Variables: A Cautionary Tale

Author

Listed:
  • James J. Heckman

Abstract

This paper considers the use of instrumental variables to estimate the mean effect of treatment on the treated. It reviews previous work on this topic by Heckman and Robb (1985, 1986) and demonstrates that (a) unless the effect of treatment is the same for everyone (conditional on observables), or (b) treatment effects are variable across persons but the person-specific component of the variability not forecastable by observables does not determine participation in the program, widely-used instrumental variable methods produce inconsistent estimators of the parameter of interest. Neither assumption is very palatable. The first assumes a homogeneity that is implausible. The second assumes either very rich data available to the econometrician or that the persons being studied either do not have better information than the econometrician or that they do not use it. Instrumental variable methods do not provide a general solution to the evaluation problem.

Suggested Citation

  • James J. Heckman, 1995. "Instrumental Variables: A Cautionary Tale," NBER Technical Working Papers 0185, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberte:0185
    Note: LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0185.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nancy Clements & James Heckman & Jeffrey Smith, 1994. "Making the Most Out Of Social Experiments: Reducing the Intrinsic Uncertainty in Evidence from Randomized Trials with an Application to the JTPA Exp," NBER Technical Working Papers 0149, National Bureau of Economic Research, Inc.
    2. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    3. Willis, Robert J & Rosen, Sherwin, 1979. "Education and Self-Selection," Journal of Political Economy, University of Chicago Press, vol. 87(5), pages 7-36, October.
    4. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    5. Heckman, James J, 1978. "Dummy Endogenous Variables in a Simultaneous Equation System," Econometrica, Econometric Society, vol. 46(4), pages 931-959, July.
    6. Heckman, James J & Honore, Bo E, 1990. "The Empirical Content of the Roy Model," Econometrica, Econometric Society, vol. 58(5), pages 1121-1149, September.
    7. James J. Heckman & Jeffrey A. Smith, 1998. "Evaluating the Welfare State," NBER Working Papers 6542, National Bureau of Economic Research, Inc.
    8. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    9. Joshua D. Angrist & Guido W. Imbens & D.B. Rubin, 1993. "Identification of Causal Effects Using Instrumental Variables," NBER Technical Working Papers 0136, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cecilia Elena Rouse, 1997. "Private School Vouchers and Student Achievement: An Evaluation of the Milwaukee Parental Choice Program," NBER Working Papers 5964, National Bureau of Economic Research, Inc.
    2. Sebastian Calonico & Hugo Ñopo, 2007. "Returns to Private Education in Peru," Research Department Publications 4516, Inter-American Development Bank, Research Department.
    3. Lanot, Gauthier & Walker, Ian, 1998. "The union/non-union wage differential: An application of semi-parametric methods," Journal of Econometrics, Elsevier, vol. 84(2), pages 327-349, June.
    4. Aki Kangasharju, 2007. "Do Wage Subsidies Increase Employment in Subsidized Firms?," Economica, London School of Economics and Political Science, vol. 74(293), pages 51-67, February.
    5. Julie Hotchkiss, 2002. "Endogeneity of tenure in the determination of quit behaviour of young workers," Applied Economics Letters, Taylor & Francis Journals, vol. 9(4), pages 231-233.
    6. Saul Lach, 2002. "Do R&D Subsidies Stimulate or Displace Private R&D? Evidence from Israel," Journal of Industrial Economics, Wiley Blackwell, vol. 50(4), pages 369-390, December.
    7. Freund, D.A. & Kniesner, T.J. & LoSasso, A.T., 1996. "How Managed Care Affects Medicaid Utilization : A Synthetic Difference-in-Difference Zero-Inflated Count Model," Discussion Paper 1996-40, Tilburg University, Center for Economic Research.
    8. Lorraine Dearden, 1999. "Qualifications and earnings in Britain: how reliable are conventional OLS estimates of the returns to education?," IFS Working Papers W99/07, Institute for Fiscal Studies.
    9. Lorraine Dearden, 1998. "Ability, families, education and earnings in Britain," IFS Working Papers W98/14, Institute for Fiscal Studies.
    10. Yanovitzky, Itzhak & Zanutto, Elaine & Hornik, Robert, 2005. "Estimating causal effects of public health education campaigns using propensity score methodology," Evaluation and Program Planning, Elsevier, vol. 28(2), pages 209-220, May.
    11. Sebastian Calonico & Hugo Ñopo, 2007. "Retornos a la Educación Privada en Perú," Research Department Publications 4517, Inter-American Development Bank, Research Department.
    12. Miguel Angel Malo & Fernando Muñoz-Bullón, 2006. "Employment promotion measures and the quality of the job match for persons with disabilities," Hacienda Pública Española / Review of Public Economics, IEF, vol. 179(4), pages 79-111, September.
    13. Isaboke, H. N. & Zhang, Q. & Nyarindo, W. N., 2016. "The effect of weather index based micro-insurance on food security status of smallholders," Agricultural and Resource Economics: International Scientific E-Journal, Agricultural and Resource Economics: International Scientific E-Journal, vol. 2(3), September.
    14. Enrico Santarelli & Hien Thu Tran, 2016. "Diversification strategies and firm performance in Vietnam," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 24(1), pages 31-68, January.
    15. Shlomo Yitzhaki & Edna Schechtman, 2004. "The Gini Instrumental Variable, or the “double instrumental variable” estimator," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(3), pages 287-313.
    16. Malo, Miguel A., 2005. "Job matching quality effects of employment promotion measures for people with disabilities," DEE - Working Papers. Business Economics. WB wb055315, Universidad Carlos III de Madrid. Departamento de Economía de la Empresa.
    17. Francisco Gallego, 2002. "Competencia y Resultados Educativos: Teoría y Evidencia para Chile," Latin American Journal of Economics-formerly Cuadernos de Economía, Instituto de Economía. Pontificia Universidad Católica de Chile., vol. 39(118), pages 309-352.
    18. Preety Srivastava, 2010. "Does Bingeing Affect Earnings?," The Economic Record, The Economic Society of Australia, vol. 86(275), pages 578-595, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    2. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    3. Javier A. Birchenall & Thomas G. Koch, 2015. ""Gallantry in Action": Evidence of Advantageous Selection in a Voluntary Army," Journal of Law and Economics, University of Chicago Press, vol. 58(1), pages 111-138.
    4. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    5. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    6. van der Klaauw, Bas, 2014. "From micro data to causality: Forty years of empirical labor economics," Labour Economics, Elsevier, vol. 30(C), pages 88-97.
    7. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    8. Joshua D. Angrist, 1991. "Instrumental Variables Estimation of Average Treatment Effects in Econometrics and Epidemiology," NBER Technical Working Papers 0115, National Bureau of Economic Research, Inc.
    9. John Engberg & Dennis Epple & Jason Imbrogno & Holger Sieg & Ron Zimmer, 2009. "Estimation of Causal Effects in Experiments with Multiple Sources of Noncompliance," NBER Working Papers 14842, National Bureau of Economic Research, Inc.
    10. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: An application in breast cancer patients," Health, Econometrics and Data Group (HEDG) Working Papers 07/07, HEDG, c/o Department of Economics, University of York.
    11. Anirban Basu & James J. Heckman & Salvador Navarro‐Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self‐selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157, November.
    12. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    13. Angrist, Joshua D. & Chen, Stacey H. & Frandsen, Brigham R., 2010. "Did Vietnam veterans get sicker in the 1990s? The complicated effects of military service on self-reported health," Journal of Public Economics, Elsevier, vol. 94(11-12), pages 824-837, December.
    14. Heckman, James, 2001. "Accounting for Heterogeneity, Diversity and General Equilibrium in Evaluating Social Programmes," Economic Journal, Royal Economic Society, vol. 111(475), pages 654-699, November.
    15. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    16. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    17. James J. Heckman, 2008. "Econometric Causality," International Statistical Review, International Statistical Institute, vol. 76(1), pages 1-27, April.
    18. David S. Lee & Thomas Lemieux, 2009. "Regression Discontinuity Designs In Economics," Working Papers 1118, Princeton University, Department of Economics, Industrial Relations Section..
    19. Aakvik, Arild & Salvanes, Kjell G. & Vaage, Kjell, 2003. "Measuring Heterogeneity in the Returns to Education in Norway Using Educational Reforms," IZA Discussion Papers 815, Institute of Labor Economics (IZA).
    20. Anna Piil Damm, 2009. "Ethnic Enclaves and Immigrant Labor Market Outcomes: Quasi-Experimental Evidence," Journal of Labor Economics, University of Chicago Press, vol. 27(2), pages 281-314, April.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0185. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.