Horseshoes, Hand Grenades, and Treatment Effects? Reassessing Bias in Nonexperimental Estimators
Author
Abstract
Suggested Citation
Download full text from publisher
References listed on IDEAS
- LaLonde, Robert J, 1986.
"Evaluating the Econometric Evaluations of Training Programs with Experimental Data,"
American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
- Robert J. LaLonde, 1984. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," Working Papers 563, Princeton University, Department of Economics, Industrial Relations Section..
- A. Smith, Jeffrey & E. Todd, Petra, 2005.
"Does matching overcome LaLonde's critique of nonexperimental estimators?,"
Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
- Jeffrey Smith & Petra Todd, 2003. "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?," University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers 20035, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
- Steven Glazerman & Dan M. Levy & David Myers, 2003.
"Nonexperimental Versus Experimental Estimates of Earnings Impacts,"
The ANNALS of the American Academy of Political and Social Science, , vol. 589(1), pages 63-93, September.
- Steven Glazerman & Dan M. Levy & David Myers, "undated". "Nonexperimental Versus Experimental Estimates of Earnings Impacts," Mathematica Policy Research Reports 7c8bd68ac8db47caa57c70ee1, Mathematica Policy Research.
- Christina Clark Tuttle & Bing-ru Teh & Ira Nichols-Barrer & Brian P. Gill & Philip Gleason, "undated". "Student Characteristics and Achievement in 22 KIPP Middle Schools," Mathematica Policy Research Reports 69064a347d534ffa8947d7b6e, Mathematica Policy Research.
- Michael Lechner, 2000.
"An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany,"
Journal of Human Resources, University of Wisconsin Press, vol. 35(2), pages 347-375.
- Lechner, Michael, 1996. "An Evaluation of Public Sector Sponsored Continuous Vocational Training Programs in East Germany," Discussion Papers 539, Institut fuer Volkswirtschaftslehre und Statistik, Abteilung fuer Volkswirtschaftslehre.
- Lechner, Michael, 1999. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," IZA Discussion Papers 93, Institute of Labor Economics (IZA).
- Peikes, Deborah N. & Moreno, Lorenzo & Orzol, Sean Michael, 2008. "Propensity Score Matching: A Note of Caution for Evaluators of Social Programs," The American Statistician, American Statistical Association, vol. 62, pages 222-231, August.
- Shadish, William R. & Clark, M. H. & Steiner, Peter M., 2008. "Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1334-1344.
- Roberto Agodini & Mark Dynarski, 2004. "Are Experiments the Only Option? A Look at Dropout Prevention Programs," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 180-194, February.
- Guido W. Imbens & Jeffrey M. Wooldridge, 2009.
"Recent Developments in the Econometrics of Program Evaluation,"
Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
- Guido M. Imbens & Jeffrey M. Wooldridge, 2008. "Recent Developments in the Econometrics of Program Evaluation," NBER Working Papers 14251, National Bureau of Economic Research, Inc.
- Wooldridge, Jeffrey M. & Imbens, Guido, 2009. "Recent Developments in the Econometrics of Program Evaluation," Scholarly Articles 3043416, Harvard University Department of Economics.
- Guido Imbens & Jeffrey M. Wooldridge, 2008. "Recent developments in the econometrics of program evaluation," CeMMAP working papers CWP24/08, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
- Imbens, Guido W. & Wooldridge, Jeffrey M., 2008. "Recent Developments in the Econometrics of Program Evaluation," IZA Discussion Papers 3640, Institute of Labor Economics (IZA).
- Will Dobbie & Roland G. Fryer, 2011. "Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children's Zone," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 158-187, July.
- Alberto Abadie & Guido W. Imbens, 2008.
"On the Failure of the Bootstrap for Matching Estimators,"
Econometrica, Econometric Society, vol. 76(6), pages 1537-1557, November.
- Alberto Abadie & Guido W. Imbens, 2006. "On the Failure of the Bootstrap for Matching Estimators," NBER Technical Working Papers 0325, National Bureau of Economic Research, Inc.
- Imbens, Guido & Abadie, Alberto, 2008. "On the Failure of the Bootstrap for Matching Estimators," Scholarly Articles 3043415, Harvard University Department of Economics.
- repec:mpr:mprres:3694 is not listed on IDEAS
- Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, vol. 85(4), pages 923-937, September.
- Atila Abdulkadiroğlu & Joshua D. Angrist & Susan M. Dynarski & Thomas J. Kane & Parag A. Pathak, 2011.
"Accountability and Flexibility in Public Schools: Evidence from Boston's Charters And Pilots,"
The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(2), pages 699-748.
- Atila Abdulkadiroglu & Joshua Angrist & Susan Dynarski & Thomas J. Kane & Parag Pathak, 2009. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters and Pilots," NBER Working Papers 15549, National Bureau of Economic Research, Inc.
- repec:mpr:mprres:6673 is not listed on IDEAS
- Kenneth A. Couch & Robert Bifulco, 2012. "Can Nonexperimental Estimates Replicate Estimates Based on Random Assignment in Evaluations of School Choice? A Within‐Study Comparison," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 31(3), pages 729-751, June.
- Elizabeth Ty Wilde & Robinson Hollister, 2007. "How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 455-477.
- Thomas Fraker & Rebecca Maynard, 1987. "The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs," Journal of Human Resources, University of Wisconsin Press, vol. 22(2), pages 194-227.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Kenneth Fortson & Randall Blair & Kathryn Gonzalez, 2015. "Evaluation of a Rural Road Rehabilitation Project in Armenia," Mathematica Policy Research Reports 3dad6663ad0343f28cb24633d, Mathematica Policy Research.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Fortson, Kenneth & Gleason, Philip & Kopa, Emma & Verbitsky-Savitz, Natalya, 2015. "Horseshoes, hand grenades, and treatment effects? Reassessing whether nonexperimental estimators are biased," Economics of Education Review, Elsevier, vol. 44(C), pages 100-113.
- Kenneth Fortson & Natalya Verbitsky-Savitz & Emma Kopa & Philip Gleason, 2012. "Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates," Mathematica Policy Research Reports 27f871b5b7b94f3a80278a593, Mathematica Policy Research.
- Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
- Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
- Ben Weidmann & Luke Miratrix, 2021. "Lurking Inferential Monsters? Quantifying Selection Bias In Evaluations Of School Programs," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(3), pages 964-986, June.
- Lechner, Michael & Wunsch, Conny, 2013.
"Sensitivity of matching-based program evaluations to the availability of control variables,"
Labour Economics, Elsevier, vol. 21(C), pages 111-121.
- Lechner, Michael & Wunsch, Conny, 2011. "Sensitivity of matching-based program evaluations to the availability of control variables," CEPR Discussion Papers 8294, C.E.P.R. Discussion Papers.
- Lechner, Michael & Wunsch, Conny, 2011. "Sensitivity of matching-based program evaluations to the availability of control variables," Economics Working Paper Series 1105, University of St. Gallen, School of Economics and Political Science.
- Michael Lechner & Conny Wunsch, 2011. "Sensitivity of Matching-Based Program Evaluations to the Availability of Control Variables," CESifo Working Paper Series 3381, CESifo.
- Lechner, Michael & Wunsch, Conny, 2011. "Sensitivity of Matching-Based Program Evaluations to the Availability of Control Variables," IZA Discussion Papers 5553, Institute of Labor Economics (IZA).
- Fatih Unlu & Douglas Lee Lauen & Sarah Crittenden Fuller & Tiffany Berglund & Elc Estrera, 2021. "Can Quasi‐Experimental Evaluations That Rely On State Longitudinal Data Systems Replicate Experimental Results?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(2), pages 572-613, March.
- Andrew P. Jaciw, 2016. "Applications of a Within-Study Comparison Approach for Evaluating Bias in Generalized Causal Inferences From Comparison Groups Studies," Evaluation Review, , vol. 40(3), pages 241-276, June.
- Andrew P. Jaciw, 2016. "Assessing the Accuracy of Generalized Inferences From Comparison Group Studies Using a Within-Study Comparison Approach," Evaluation Review, , vol. 40(3), pages 199-240, June.
- Katherine Baicker & Theodore Svoronos, 2019. "Testing the Validity of the Single Interrupted Time Series Design," NBER Working Papers 26080, National Bureau of Economic Research, Inc.
- Elizabeth Ty Wilde & Robinson Hollister, 2007. "How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 455-477.
- Katherine Baicker & Theodore Svoronos, 2019. "Testing the Validity of the Single Interrupted Time Series Design," CID Working Papers 364, Center for International Development at Harvard University.
- Peter M. Steiner & Vivian C. Wong, 2018. "Assessing Correspondence Between Experimental and Nonexperimental Estimates in Within-Study Comparisons," Evaluation Review, , vol. 42(2), pages 214-247, April.
- Flores, Carlos A. & Mitnik, Oscar A., 2009.
"Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data,"
IZA Discussion Papers
4451, Institute of Labor Economics (IZA).
- Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-9, University of Miami, Department of Economics.
- Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
- Guido W. Imbens & Jeffrey M. Wooldridge, 2009.
"Recent Developments in the Econometrics of Program Evaluation,"
Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
- Guido M. Imbens & Jeffrey M. Wooldridge, 2008. "Recent Developments in the Econometrics of Program Evaluation," NBER Working Papers 14251, National Bureau of Economic Research, Inc.
- Wooldridge, Jeffrey M. & Imbens, Guido, 2009. "Recent Developments in the Econometrics of Program Evaluation," Scholarly Articles 3043416, Harvard University Department of Economics.
- Guido Imbens & Jeffrey M. Wooldridge, 2008. "Recent developments in the econometrics of program evaluation," CeMMAP working papers CWP24/08, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
- Imbens, Guido W. & Wooldridge, Jeffrey M., 2008. "Recent Developments in the Econometrics of Program Evaluation," IZA Discussion Papers 3640, Institute of Labor Economics (IZA).
- Heinrich, Carolyn J. & Mueser, Peter R. & Troske, Kenneth & Jeon, Kyung-Seong & Kahvecioglu, Daver C., 2009.
"New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program,"
IZA Discussion Papers
4569, Institute of Labor Economics (IZA).
- Peter R. Mueser & Carolyn J. Heinrich & Kenneth R. Troske & Kyung-Seong Jeon & Daver C. Kahvecioglu, 2010. "New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program," Working Papers 1003, Department of Economics, University of Missouri.
- Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007.
"Using State Administrative Data to Measure Program Performance,"
The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
- Peter R. Mueser & Kenneth Troske & Alexey Gorislavsky, 2003. "Using State Administrative Data to Measure Program Performance," Working Papers 0309, Department of Economics, University of Missouri.
- Mueser, Peter R. & Troske, Kenneth & Gorislavsky, Alexey, 2003. "Using State Administrative Data to Measure Program Performance," IZA Discussion Papers 786, Institute of Labor Economics (IZA).
- Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," Working Papers 0702, Department of Economics, University of Missouri.
- repec:mpr:mprres:7443 is not listed on IDEAS
- Vivian C. Wong & Peter M. Steiner, 2018. "Designs of Empirical Evaluations of Nonexperimental Methods in Field Settings," Evaluation Review, , vol. 42(2), pages 176-213, April.
- Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
- Henrik Hansen & Ninja Ritter Klejnstrup & Ole Winckler Andersen, 2011. "A Comparison of Model-based and Design-based Impact Evaluations of Interventions in Developing Countries," IFRO Working Paper 2011/16, University of Copenhagen, Department of Food and Resource Economics.
More about this item
Keywords
RCTs Randomized Controlled Trials; Nonexperimental Estimators; Education; Working Paper 16;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:1c24988cd5454dd3be51fbc2c07860e6. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George The email address of this maintainer does not seem to be valid anymore. Please ask Cindy George to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.