IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v243y2024i1s0304407624000642.html
   My bibliography  Save this article

Policy evaluation with multiple instrumental variables

Author

Listed:
  • Mogstad, Magne
  • Torgovitsky, Alexander
  • Walters, Christopher R.

Abstract

Marginal treatment effect methods are widely used for causal inference and policy evaluation with instrumental variables. However, they fundamentally rely on the well-known monotonicity (threshold-crossing) condition on treatment choice behavior. This condition cannot hold with multiple instruments unless treatment choice is effectively homogeneous. We develop a new marginal treatment effect framework under a weaker, partial monotonicity condition. The partial monotonicity condition is implied by standard choice theory and allows for rich unobserved heterogeneity even in the presence of multiple instruments. The new framework can be viewed as having multiple different choice models for the same observed treatment variable, all of which must be consistent with the data and with each other. Using this framework, we develop a methodology for partial identification of clearly stated, policy-relevant target parameters while allowing for a wide variety of nonparametric shape restrictions and parametric functional form assumptions. We show how the methodology can be used to combine multiple instruments together to yield more informative empirical conclusions than one would obtain by using each instrument separately. The methodology provides a blueprint for extracting and aggregating information from multiple controlled or natural experiments while still allowing for rich unobserved heterogeneity in both treatment effects and choice behavior.

Suggested Citation

  • Mogstad, Magne & Torgovitsky, Alexander & Walters, Christopher R., 2024. "Policy evaluation with multiple instrumental variables," Journal of Econometrics, Elsevier, vol. 243(1).
  • Handle: RePEc:eee:econom:v:243:y:2024:i:1:s0304407624000642
    DOI: 10.1016/j.jeconom.2024.105718
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407624000642
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jeconom.2024.105718?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    2. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2008. "Instrumental Variables in Models with Multiple Outcomes: The General Unordered Case," Annals of Economics and Statistics, GENES, issue 91-92, pages 151-174.
    3. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    4. Joseph J. Doyle Jr., 2007. "Child Protection and Child Outcomes: Measuring the Effects of Foster Care," American Economic Review, American Economic Association, vol. 97(5), pages 1583-1610, December.
    5. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    6. Magne Mogstad & Andres Santos & Alexander Torgovitsky, 2017. "Using Instrumental Variables for Inference about Policy Relevant Treatment Effects," NBER Working Papers 23568, National Bureau of Economic Research, Inc.
    7. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
    8. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August.
    9. David Arnold & Will Dobbie & Crystal S Yang, 2018. "Racial Bias in Bail Decisions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1885-1932.
    10. Gautier, Eric & Hoderlein, Stefan, 2011. "A triangular treatment effect model with random coefficients in the selection equation," TSE Working Papers 15-598, Toulouse School of Economics (TSE), revised 25 Aug 2015.
    11. David Autor & Andreas Kostøl & Magne Mogstad & Bradley Setzler, 2019. "Disability Benefits, Consumption Insurance, and Household Labor Supply," American Economic Review, American Economic Association, vol. 109(7), pages 2613-2654, July.
    12. Pedro Carneiro & James J. Heckman & Edward Vytlacil, 2010. "Evaluating Marginal Policy Changes and the Average Effect of Treatment for Individuals at the Margin," Econometrica, Econometric Society, vol. 78(1), pages 377-394, January.
    13. Martin Nybom, 2017. "The Distribution of Lifetime Earnings Returns to College," Journal of Labor Economics, University of Chicago Press, vol. 35(4), pages 903-952.
    14. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492, National Bureau of Economic Research, Inc.
    15. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    16. Thomas J. Kane & Cecilia Rouse, 1993. "Labor Market Returns to Two- And Four-Year College: Is A Credit a Credit And Do Degrees Matter?," Working Papers 690, Princeton University, Department of Economics, Industrial Relations Section..
    17. Flavio Cunha & James J. Heckman & Salvador Navarro, 2007. "The Identification And Economic Content Of Ordered Choice Models With Stochastic Thresholds," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 48(4), pages 1273-1309, November.
    18. Thomas Cornelissen & Christian Dustmann & Anna Raute & Uta Schönberg, 2018. "Who Benefits from Universal Child Care? Estimating Marginal Returns to Early Child Care Attendance," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2356-2409.
    19. Patrick Kline & Christopher R. Walters, 2016. "Evaluating Public Programs with Close Substitutes: The Case of HeadStart," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1795-1848.
    20. Magne Mogstad & Alexander Torgovitsky & Christopher R. Walters, 2021. "The Causal Interpretation of Two-Stage Least Squares with Multiple Instrumental Variables," American Economic Review, American Economic Association, vol. 111(11), pages 3663-3698, November.
    21. Jack Mountjoy, 2022. "Community Colleges and Upward Mobility," American Economic Review, American Economic Association, vol. 112(8), pages 2580-2630, August.
    22. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    23. Eric French & Jae Song, 2014. "The Effect of Disability Insurance Receipt on Labor Supply," American Economic Journal: Economic Policy, American Economic Association, vol. 6(2), pages 291-337, May.
    24. Christian N. Brinch & Magne Mogstad & Matthew Wiswall, 2017. "Beyond LATE with a Discrete Instrument," Journal of Political Economy, University of Chicago Press, vol. 125(4), pages 985-1039.
    25. Pedro Carneiro & James J. Heckman & Edward J. Vytlacil, 2011. "Estimating Marginal Returns to Education," American Economic Review, American Economic Association, vol. 101(6), pages 2754-2781, October.
    26. Domenico Depalo, 2020. "Explaining the causal effect of adherence to medication on cholesterol through the marginal patient," Health Economics, John Wiley & Sons, Ltd., vol. 29(S1), pages 110-126, October.
    27. repec:fth:prinin:311 is not listed on IDEAS
    28. Amanda E Kowalski, 2023. "Behaviour within a Clinical Trial and Implications for Mammography Guidelines," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 432-462.
    29. repec:adr:anecst:y:2008:i:91-92:p:12 is not listed on IDEAS
    30. Christopher R. Walters, 2018. "The Demand for Effective Charter Schools," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2179-2223.
    31. Thomas J. Kane & Cecilia E. Rouse, 1993. "Labor Market Returns to Two- and Four-Year Colleges: Is a Credit a Credit and Do Degrees Matter?," NBER Working Papers 4268, National Bureau of Economic Research, Inc.
    32. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    33. Alexander Torgovitsky, 2019. "Nonparametric Inference on State Dependence in Unemployment," Econometrica, Econometric Society, vol. 87(5), pages 1475-1505, September.
    34. Flavio Cunha & James J. Heckman & Salvador Navarro, 2007. "The Identification & Economic Content of Ordered Choice Models with Stochastic Thresholds," Working Papers 200726, Geary Institute, University College Dublin.
    35. James Heckman & Justin L. Tobias & Edward Vytlacil, 2001. "Four Parameters of Interest in the Evaluation of Social Programs," Southern Economic Journal, John Wiley & Sons, vol. 68(2), pages 210-223, October.
    36. Toru Kitagawa, 2015. "A Test for Instrument Validity," Econometrica, Econometric Society, vol. 83(5), pages 2043-2063, September.
    37. Carneiro, Pedro & Lee, Sokbae, 2009. "Estimating distributions of potential outcomes using local instrumental variables with an application to changes in college enrollment and wage inequality," Journal of Econometrics, Elsevier, vol. 149(2), pages 191-208, April.
    38. Pedro Carneiro & Karsten T. Hansen & James J. Heckman, 2003. "2001 Lawrence R. Klein Lecture Estimating Distributions of Treatment Effects with an Application to the Returns to Schooling and Measurement of the Effects of Uncertainty on College Choice," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 44(2), pages 361-422, May.
    39. James Heckman & Justin L. Tobias & Edward Vytlacil, 2001. "Four Parameters of Interest in the Evaluation of Social Programs," Southern Economic Journal, John Wiley & Sons, vol. 68(2), pages 210-223, October.
    40. Heckman, James J, 1974. "Shadow Prices, Market Wages, and Labor Supply," Econometrica, Econometric Society, vol. 42(4), pages 679-694, July.
    41. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    42. Magne Mogstad & Andres Santos & Alexander Torgovitsky, 2018. "Using Instrumental Variables for Inference About Policy Relevant Treatment Parameters," Econometrica, Econometric Society, vol. 86(5), pages 1589-1619, September.
    43. Imbens, Guido W., 2014. "Instrumental Variables: An Econometrician's Perspective," IZA Discussion Papers 8048, Institute of Labor Economics (IZA).
    44. Edward Vytlacil & James J. Heckman, 2001. "Policy-Relevant Treatment Effects," American Economic Review, American Economic Association, vol. 91(2), pages 107-111, May.
    45. James J. Heckman & Rodrigo Pinto, 2018. "Unordered Monotonicity," Econometrica, Econometric Society, vol. 86(1), pages 1-35, January.
    46. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    47. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    48. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70, Elsevier.
    49. Robert Moffitt, 2008. "Estimating Marginal Treatment Effects in Heterogeneous Populations," Annals of Economics and Statistics, GENES, issue 91-92, pages 239-261.
    50. Thomas J. Kane & Cecilia E. Rouse, 1993. "Labor Market Returns to Two- And Four-Year College: Is a Credit a Credit And Do Degrees Matter?," Working Papers 690, Princeton University, Department of Economics, Industrial Relations Section..
    51. Guido W. Imbens & Donald B. Rubin, 1997. "Estimating Outcome Distributions for Compliers in Instrumental Variables Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 555-574.
    52. Felfe, Christina & Lalive, Rafael, 2018. "Does early child care affect children's development?," Journal of Public Economics, Elsevier, vol. 159(C), pages 33-53.
    53. repec:adr:anecst:y:2008:i:91-92:p:08 is not listed on IDEAS
    54. Heckman, James J, 1990. "Varieties of Selection Bias," American Economic Review, American Economic Association, vol. 80(2), pages 313-318, May.
    55. James Heckman, 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 441-462.
    56. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    57. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    58. James J. Heckman & Jeffrey A. Smith, 1998. "Evaluating the Welfare State," NBER Working Papers 6542, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hoshino Tadao & Yanagi Takahide, 2022. "Estimating marginal treatment effects under unobserved group heterogeneity," Journal of Causal Inference, De Gruyter, vol. 10(1), pages 197-216, January.
    2. Fusejima, Koki, 2024. "Identification of multi-valued treatment effects with unobserved heterogeneity," Journal of Econometrics, Elsevier, vol. 238(1).
    3. Tommasi, Denni & Zhang, Lina, 2024. "Bounding program benefits when participation is misreported," Journal of Econometrics, Elsevier, vol. 238(1).
    4. Leonard Goff, 2024. "When does IV identification not restrict outcomes?," Papers 2406.02835, arXiv.org, revised Sep 2024.
    5. Ravi B. Sojitra & Vasilis Syrgkanis, 2024. "Dynamic Local Average Treatment Effects," Papers 2405.01463, arXiv.org, revised May 2024.
    6. Bhuller, Manudeep & Sigstad, Henrik, 2024. "2SLS with multiple treatments," Journal of Econometrics, Elsevier, vol. 242(1).
    7. Shengli Dai & Weimin Zhang & Linshan Lan, 2022. "Quantitative Evaluation of China’s Ecological Protection Compensation Policy Based on PMC Index Model," IJERPH, MDPI, vol. 19(16), pages 1-24, August.
    8. Stephen Coussens & Jann Spiess, 2021. "Improving Inference from Simple Instruments through Compliance Estimation," Papers 2108.03726, arXiv.org.
    9. Yu-Chang Chen & Haitian Xie, 2022. "Personalized Subsidy Rules," Papers 2202.13545, arXiv.org, revised Mar 2022.
    10. Vishal Kamat & Samuel Norris & Matthew Pecenco, 2023. "Identification in Multiple Treatment Models under Discrete Variation," Papers 2307.06174, arXiv.org.
    11. van ’t Hoff, Nadja & Lewbel, Arthur & Mellace, Giovanni, 2023. "Limited Monotonicity and the Combined Compliers LATE," Discussion Papers on Economics 2/2023, University of Southern Denmark, Department of Economics.
    12. Mustafa Özer & Jan Fidrmuc, 2024. "Does Immigration Affect the Natives’ Mental Health? Causal Evidence from Forced Syrian Migration to Turkey," CESifo Working Paper Series 11399, CESifo.
    13. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    2. Domenico Depalo, 2020. "Explaining the causal effect of adherence to medication on cholesterol through the marginal patient," Health Economics, John Wiley & Sons, Ltd., vol. 29(S1), pages 110-126, October.
    3. Sokbae Lee & Bernard Salanié, 2018. "Identifying Effects of Multivalued Treatments," Econometrica, Econometric Society, vol. 86(6), pages 1939-1963, November.
    4. Pereda-Fernández, Santiago, 2023. "Identification and estimation of triangular models with a binary treatment," Journal of Econometrics, Elsevier, vol. 234(2), pages 585-623.
    5. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    6. Matthias Westphal & Daniel A Kamhöfer & Hendrik Schmitz, 2022. "Marginal College Wage Premiums Under Selection Into Employment," The Economic Journal, Royal Economic Society, vol. 132(646), pages 2231-2272.
    7. Bartalotti, Otávio & Kédagni, Désiré & Possebom, Vitor, 2023. "Identifying marginal treatment effects in the presence of sample selection," Journal of Econometrics, Elsevier, vol. 234(2), pages 565-584.
    8. Patrick Kline & Christopher R. Walters, 2019. "On Heckits, LATE, and Numerical Equivalence," Econometrica, Econometric Society, vol. 87(2), pages 677-696, March.
    9. Robert A. Moffitt & Matthew V. Zahn, 2019. "The Marginal Labor Supply Disincentives of Welfare: Evidence from Administrative Barriers to Participation," NBER Working Papers 26028, National Bureau of Economic Research, Inc.
    10. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    11. Heckman, James J. & Humphries, John Eric & Veramendi, Gregory, 2016. "Dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 191(2), pages 276-292.
    12. Gathmann, Christina & Vonnahme, Christina & Busse, Anna & Kim, Jongoh, 2021. "Marginal returns to citizenship and educational performance," Ruhr Economic Papers 920, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    13. Bhuller, Manudeep & Sigstad, Henrik, 2024. "2SLS with multiple treatments," Journal of Econometrics, Elsevier, vol. 242(1).
    14. Sloczynski, Tymon, 2021. "When Should We (Not) Interpret Linear IV Estimands as LATE?," IZA Discussion Papers 14349, Institute of Labor Economics (IZA).
    15. Thomas Carr & Toru Kitagawa, 2021. "Testing Instrument Validity with Covariates," Papers 2112.08092, arXiv.org, revised Sep 2023.
    16. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    17. Pedro Carneiro & James J. Heckman & Edward J. Vytlacil, 2011. "Estimating Marginal Returns to Education," American Economic Review, American Economic Association, vol. 101(6), pages 2754-2781, October.
    18. Magne Mogstad & Andres Santos & Alexander Torgovitsky, 2018. "Using Instrumental Variables for Inference About Policy Relevant Treatment Parameters," Econometrica, Econometric Society, vol. 86(5), pages 1589-1619, September.
    19. James J. Heckman, 2008. "Econometric Causality," International Statistical Review, International Statistical Institute, vol. 76(1), pages 1-27, April.
    20. Pedro Carneiro & Michael Lokshin & Nithin Umapathi, 2017. "Average and Marginal Returns to Upper Secondary Schooling in Indonesia," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 16-36, January.

    More about this item

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:243:y:2024:i:1:s0304407624000642. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.