IDEAS home Printed from https://ideas.repec.org/p/usg/econwp/201404.html
   My bibliography  Save this paper

Treatment evaluation with multiple outcome periods under endogeneity and attrition

Author

Listed:
  • Frölich, Markus
  • Huber, Martin

Abstract

This paper develops a nonparametric methodology for treatment evaluation with multiple outcome periods under treatment endogeneity and missing outcomes. We use instrumental variables, pre-treatment characteristics, and short-term (or intermediate) outcomes to identify the average treatment effect on the outcomes of compliers (the subpopulation whose treatment reacts on the instrument) in multiple periods based on inverse probability weighting. Treatment selection and attrition may depend on both observed characteristics and the unobservable compliance type, which is possibly related to unobserved factors. We also provide a simulation study and apply our methods to the evaluation of a policy intervention targeting college achievement, where we find that controlling for attrition considerably affects the effect estimates.

Suggested Citation

  • Frölich, Markus & Huber, Martin, 2014. "Treatment evaluation with multiple outcome periods under endogeneity and attrition," Economics Working Paper Series 1404, University of St. Gallen, School of Economics and Political Science.
    • Handle: RePEc:usg:econwp:2014:04
    as

    Download full text from publisher

    File URL: http://ux-tauri.unisg.ch/RePEc/usg/econwp/EWP-1404.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Paolo Frumento & Fabrizia Mealli & Barbara Pacini & Donald B. Rubin, 2012. "Evaluating the Effect of Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing Outcome Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 450-466, June.
    3. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492, National Bureau of Economic Research, Inc.
    4. Murphy S.A. & van der Laan M.J. & Robins J.M., 2001. "Marginal Mean Models for Dynamic Regimes," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1410-1423, December.
    5. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    6. Michael Lechner & Ruth Miquel & Conny Wunsch, 2011. "Long‐Run Effects Of Public Sector Sponsored Training In West Germany," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 742-784, August.
    7. Weili Ding & Steven F. Lehrer, 2010. "Estimating Treatment Effects from Contaminated Multiperiod Education Experiments: The Dynamic Impacts of Class Size Reductions," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 31-42, February.
    8. A. Mattei & F. Mealli, 2007. "Application of the Principal Stratification Approach to the Faenza Randomized Experiment on Breast Self-Examination," Biometrics, The International Biometric Society, vol. 63(2), pages 437-446, June.
    9. John Fitzgerald & Peter Gottschalk & Robert Moffitt, 1998. "An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics," Journal of Human Resources, University of Wisconsin Press, vol. 33(2), pages 251-299.
    10. Markus Frölich & Blaise Melly, 2013. "Unconditional Quantile Treatment Effects Under Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(3), pages 346-357, July.
    11. Markus Frlich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
    12. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    13. JAMES G. MacKINNON, 2006. "Bootstrap Methods in Econometrics," The Economic Record, The Economic Society of Australia, vol. 82(s1), pages 2-18, September.
    14. Racine, Jeff & Li, Qi, 2004. "Nonparametric estimation of regression functions with both categorical and continuous data," Journal of Econometrics, Elsevier, vol. 119(1), pages 99-130, March.
    15. Yau L.H.Y. & Little R.J., 2001. "Inference for the Complier-Average Causal Effect From Longitudinal Data Subject to Noncompliance and Missing Data, With Application to a Job Training Assessment for the Unemployed," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1232-1244, December.
    16. Philip Oreopoulos & Daniel Lang & Joshua Angrist, 2009. "Incentives and Services for College Achievement: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 136-163, January.
    17. Junni L. Zhang & Donald B. Rubin, 2003. "Estimation of Causal Effects via Principal Stratification When Some Outcomes are Truncated by “Deathâ€," Journal of Educational and Behavioral Statistics, , vol. 28(4), pages 353-368, December.
    18. Frolich, Markus, 2007. "Nonparametric IV estimation of local average treatment effects with covariates," Journal of Econometrics, Elsevier, vol. 139(1), pages 35-75, July.
    19. Judith Lok & Richard Gill & Aad Van Der Vaart & James Robins, 2004. "Estimating the causal effect of a time‐varying treatment on time‐to‐event using structural nested failure time models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 58(3), pages 271-295, August.
    20. Ekaterini Kyriazidou, 1997. "Estimation of a Panel Data Sample Selection Model," Econometrica, Econometric Society, vol. 65(6), pages 1335-1364, November.
    21. Lu Wang & Andrea Rotnitzky & Xihong Lin & Randall E. Millikan & Peter F. Thall, 2012. "Evaluation of Viable Dynamic Treatment Regimes in a Sequentially Randomized Trial of Advanced Prostate Cancer," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 493-508, June.
    22. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    23. Abadie, Alberto, 2003. "Semiparametric instrumental variable estimation of treatment response models," Journal of Econometrics, Elsevier, vol. 113(2), pages 231-263, April.
    24. Frangakis, Constantine E. & Brookmeyer, Ronald S. & Varadhan, Ravi & Safaeian, Mahboobeh & Vlahov, David & Strathdee, Steffanie A., 2004. "Methodology for Evaluating a Partially Controlled Longitudinal Treatment Using Principal Stratification, With Application to a Needle Exchange Program," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 239-249, January.
    25. Joshua Angrist & Eric Bettinger & Michael Kremer, 2006. "Long-Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia," American Economic Review, American Economic Association, vol. 96(3), pages 847-862, June.
    26. Bollinger, Christopher R & David, Martin H, 2001. "Estimation with Response Error and Nonresponse: Food-Stamp Participation in the SIPP," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(2), pages 129-141, April.
    27. Semykina, Anastasia & Wooldridge, Jeffrey M., 2010. "Estimating panel data models in the presence of endogeneity and selection," Journal of Econometrics, Elsevier, vol. 157(2), pages 375-380, August.
    28. Mitali Das & Whitney K. Newey & Francis Vella, 2003. "Nonparametric Estimation of Sample Selection Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 70(1), pages 33-58.
    29. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    30. Keisuke Hirano & Guido W. Imbens & Geert Ridder & Donald B. Rubin, 2001. "Combining Panel Data Sets with Attrition and Refreshment Samples," Econometrica, Econometric Society, vol. 69(6), pages 1645-1659, November.
    31. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    32. JM Abowd & Bruno Crépon & Francis Kramarz, 1997. "Moment Estimation with Attrition," Working Papers 97-35, Center for Research in Economics and Statistics.
    33. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    34. Ekaterini Kyriazidou, 2001. "Estimation of Dynamic Panel Data Sample Selection Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 68(3), pages 543-572.
    35. Barnard J. & Frangakis C.E. & Hill J.L. & Rubin D.B., 2003. "Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 299-323, January.
    36. Tan, Zhiqiang, 2006. "Regression and Weighting Methods for Causal Inference Using Instrumental Variables," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1607-1618, December.
    37. Markus Frölich, 2008. "Parametric and Nonparametric Regression in the Presence of Endogenous Control Variables," International Statistical Review, International Statistical Institute, vol. 76(2), pages 214-227, August.
    38. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    39. Hausman, Jerry A & Wise, David A, 1979. "Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment," Econometrica, Econometric Society, vol. 47(2), pages 455-473, March.
    40. Shepherd, Bryan E. & Redman, Mary W. & Ankerst, Donna P., 2008. "Does Finasteride Affect the Severity of Prostate Cancer? A Causal Sensitivity Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1392-1404.
    41. Charles F. Manski, 1989. "Anatomy of the Selection Problem," Journal of Human Resources, University of Wisconsin Press, vol. 24(3), pages 343-360.
    42. Vincent Tinto, 1997. "Classrooms as Communities," The Journal of Higher Education, Taylor & Francis Journals, vol. 68(6), pages 599-623, November.
    43. Martin Huber, 2012. "Identification of Average Treatment Effects in Social Experiments Under Alternative Forms of Attrition," Journal of Educational and Behavioral Statistics, , vol. 37(3), pages 443-474, June.
    44. Zhang, Junni L. & Rubin, Donald B. & Mealli, Fabrizia, 2009. "Likelihood-Based Analysis of Causal Effects of Job-Training Programs Using Principal Stratification," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 166-176.
    45. Martin Huber & Giovanni Mellace, 2015. "Testing Instrument Validity for LATE Identification Based on Inequality Moment Constraints," The Review of Economics and Statistics, MIT Press, vol. 97(2), pages 398-411, May.
    46. Yahong Peng & Roderick J. A. Little & Trivellore E. Raghunathan, 2004. "An Extended General Location Model for Causal Inferences from Data Subject to Noncompliance and Missing Values," Biometrics, The International Biometric Society, vol. 60(3), pages 598-607, September.
    47. Imai, Kosuke, 2008. "Sharp bounds on the causal effects in randomized experiments with "truncation-by-death"," Statistics & Probability Letters, Elsevier, vol. 78(2), pages 144-149, February.
    48. Abowd J.M. & Crepon B. & Kramarz F., 2001. "Moment Estimation With Attrition: An Application to Economic Models," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1223-1231, December.
    49. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kaitlin Anderson & Gema Zamarro & Jennifer Steele & Trey Miller, 2021. "Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations," Evaluation Review, , vol. 45(1-2), pages 70-104, February.
    2. Markus Frölich & Martin Huber, 2017. "Direct and indirect treatment effects–causal chains and mediation analysis with instrumental variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1645-1666, November.
    3. Hans Fricke & Markus Frölich & Martin Huber & Michael Lechner, 2020. "Endogeneity and non‐response bias in treatment evaluation – nonparametric identification of causal effects by instruments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 481-504, August.
    4. Vitor Possebom, 2019. "Sharp Bounds for the Marginal Treatment Effect with Sample Selection," Papers 1904.08522, arXiv.org.
    5. Martin Huber, 2021. "On the Plausibility of the Latent Ignorability Assumption," Econometrics, MDPI, vol. 9(4), pages 1-6, December.
    6. Akanksha Negi, 2020. "Doubly weighted M-estimation for nonrandom assignment and missing outcomes," Papers 2011.11485, arXiv.org.
    7. Martin Huber & Anna Solovyeva, 2020. "Direct and Indirect Effects under Sample Selection and Outcome Attrition," Econometrics, MDPI, vol. 8(4), pages 1-25, December.
    8. Salm, Martin & Siflinger, Bettina & Xie, Mingjia, 2021. "The Effect of Retirement on Mental Health: Indirect Treatment Effects and Causal Mediation," Other publications TiSEM e28efa7f-8219-437c-a26d-2, Tilburg University, School of Economics and Management.
    9. Gayani Rathnayake & Akanksha Negi & Otavio Bartalotti & Xueyan Zhao, 2024. "Difference-in-Differences with Sample Selection," Papers 2411.09221, arXiv.org.
    10. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    11. Possebom, Vitor, 2018. "Sharp bounds on the MTE with sample selection," MPRA Paper 89785, University Library of Munich, Germany.
    12. Christophe Bell'ego & David Benatia & Vincent Dortet-Bernardet, 2023. "The Chained Difference-in-Differences," Papers 2301.01085, arXiv.org, revised May 2024.
    13. Kevin L. Cope, 2023. "Measuring law's normative force," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 20(4), pages 1005-1044, December.
    14. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    15. Avdeenko, Alexandra & Frölich, Markus, 2020. "Research standards in empirical development economics: What’s well begun, is half done," World Development, Elsevier, vol. 127(C).
    16. repec:hhs:ifauwp:2025_012 is not listed on IDEAS

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hans Fricke & Markus Frölich & Martin Huber & Michael Lechner, 2020. "Endogeneity and non‐response bias in treatment evaluation – nonparametric identification of causal effects by instruments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 481-504, August.
    2. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    3. Martin Huber & Anna Solovyeva, 2020. "Direct and Indirect Effects under Sample Selection and Outcome Attrition," Econometrics, MDPI, vol. 8(4), pages 1-25, December.
    4. Martin Huber, 2012. "Identification of Average Treatment Effects in Social Experiments Under Alternative Forms of Attrition," Journal of Educational and Behavioral Statistics, , vol. 37(3), pages 443-474, June.
    5. Martin Huber, 2014. "Treatment Evaluation in the Presence of Sample Selection," Econometric Reviews, Taylor & Francis Journals, vol. 33(8), pages 869-905, November.
    6. Martin Huber, 2010. "Identification of average treatment effects in social experiments under different forms of attrition," University of St. Gallen Department of Economics working paper series 2010 2010-22, Department of Economics, University of St. Gallen.
    7. Simon Calmar Andersen & Louise Beuchert & Phillip Heiler & Helena Skyt Nielsen, 2023. "A Guide to Impact Evaluation under Sample Selection and Missing Data: Teacher's Aides and Adolescent Mental Health," Papers 2308.04963, arXiv.org.
    8. Martin Huber & Michael Lechner & Andreas Steinmayr, 2015. "Radius matching on the propensity score with bias adjustment: tuning parameters and finite sample behaviour," Empirical Economics, Springer, vol. 49(1), pages 1-31, August.
    9. Bodory, Hugo & Huber, Martin, 2018. "The causalweight package for causal inference in R," FSES Working Papers 493, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    10. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    11. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    12. Hugo Bodory & Martin Huber & Michael Lechner, 2024. "The Finite Sample Performance of Instrumental Variable-Based Estimators of the Local Average Treatment Effect When Controlling for Covariates," Computational Economics, Springer;Society for Computational Economics, vol. 64(4), pages 2053-2078, October.
    13. Michela Bia & Martin Huber & Lukáš Lafférs, 2024. "Double Machine Learning for Sample Selection Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(3), pages 958-969, July.
    14. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    15. Lechner, Michael, 2013. "Treatment effects and panel data," Economics Working Paper Series 1314, University of St. Gallen, School of Economics and Political Science.
    16. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    17. Marco Caliendo & Stefan Tübbicke, 2020. "New evidence on long-term effects of start-up subsidies: matching estimates and their robustness," Empirical Economics, Springer, vol. 59(4), pages 1605-1631, October.
    18. Markus Frölich & Martin Huber, 2017. "Direct and indirect treatment effects–causal chains and mediation analysis with instrumental variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1645-1666, November.
    19. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    20. Martin Huber & Giovanni Mellace, 2015. "Sharp Bounds on Causal Effects under Sample Selection," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(1), pages 129-151, February.

    More about this item

    Keywords

    Treatment effect; attrition; endogeneity; panel data; weighting;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C24 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Truncated and Censored Models; Switching Regression Models; Threshold Regression Models
    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:usg:econwp:2014:04. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/vwasgch.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.