IDEAS home Printed from https://ideas.repec.org/p/ecm/nasm04/252.html
   My bibliography  Save this paper

Estimating Dynamic Treatment Effects from Project STAR

Author

Listed:
  • Steven Lehrer
  • Weili Ding

Abstract

This paper considers the analysis of data from randomized trials which offer a sequence of interventions and suffer from a variety of problems in implementation. In experiments that provide treatment in multiple periods (T>1), subjects have up to 2^{T}-1 counterfactual outcomes to be estimated to determine the full sequence of causal effects from the study. Traditional program evaluation and non-experimental estimators are unable to recover parameters of interest to policy makers in this setting, particularly if there is non-ignorable attrition. We examine these issues in the context of Tennessee's highly influential randomized class size study, Project STAR. We demonstrate how a researcher can estimate the full sequence of dynamic treatment effects using a sequential difference in difference strategy that accounts for attrition due to observables using inverse probability weighting M-estimators. These estimates allow us to recover the structural parameters of the small class effects in the underlying education production function and construct dynamic average treatment effects. We present a complete and different picture of the effectiveness of reduced class size and find that accounting for both attrition due to observables and selection due to unobservables is crucial and necessary with data from Project STAR

Suggested Citation

  • Steven Lehrer & Weili Ding, 2004. "Estimating Dynamic Treatment Effects from Project STAR," Econometric Society 2004 North American Summer Meetings 252, Econometric Society.
    • Handle: RePEc:ecm:nasm04:252
    as

    Download full text from publisher

    File URL: http://repec.org/esNASM04/up.26073.1075342942.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 497-532.
    2. Constantine E. Frangakis & Donald B. Rubin, 2002. "Principal Stratification in Causal Inference," Biometrics, The International Biometric Society, vol. 58(1), pages 21-29, March.
    3. James Heckman & Jeffrey Smith & Christopher Taber, 1994. "Accounting for Dropouts in Evaluations of Social Experiments," NBER Technical Working Papers 0166, National Bureau of Economic Research, Inc.
    4. John Fitzgerald & Peter Gottschalk & Robert Moffitt, 1998. "An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics," Journal of Human Resources, University of Wisconsin Press, vol. 33(2), pages 251-299.
    5. Caroline M. Hoxby, 2000. "The Effects of Class Size on Student Achievement: New Evidence from Population Variation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(4), pages 1239-1285.
    6. Yau L.H.Y. & Little R.J., 2001. "Inference for the Complier-Average Causal Effect From Longitudinal Data Subject to Noncompliance and Missing Data, With Application to a Job Training Assessment for the Unemployed," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1232-1244, December.
    7. James Heckman & Neil Hohmann & Jeffrey Smith & Michael Khoo, 2000. "Substitution and Dropout Bias in Social Experiments: A Study of an Influential Social Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(2), pages 651-694.
    8. Becketti, Sean & Gould, William & Lillard, Lee & Welch, Finis, 1988. "The Panel Study of Income Dynamics after Fourteen Years: An Evaluatio n," Journal of Labor Economics, University of Chicago Press, vol. 6(4), pages 472-492, October.
    9. Hausman, Jerry A & Newey, Whitney K & Taylor, William E, 1987. "Efficient Estimation and Identification of Simultaneous Equation Models with Covariance Restrictions," Econometrica, Econometric Society, vol. 55(4), pages 849-874, July.
    10. Krueger, Alan B & Whitmore, Diane M, 2001. "The Effect of Attending a Small Class in the Early Grades on College-Test Taking and Middle School Test Results: Evidence from Project STAR," Economic Journal, Royal Economic Society, vol. 111(468), pages 1-28, January.
    11. Michael A. Boozer & Stephen E. Cacciola, 2001. "Inside the 'Black Box' of Project STAR: Estimation of Peer Effects Using Experimental Data," Working Papers 832, Economic Growth Center, Yale University.
    12. Jeffrey M. Wooldridge, 2002. "Inverse probability weighted M-estimators for sample selection, attrition, and stratification," Portuguese Economic Journal, Springer;Instituto Superior de Economia e Gestao, vol. 1(2), pages 117-139, August.
    13. Caroline Hoxby, 2000. "Peer Effects in the Classroom: Learning from Gender and Race Variation," NBER Working Papers 7867, National Bureau of Economic Research, Inc.
    14. Ruth Miquel, 2002. "Identification of Dynamic Treatment Effects by Instrumental Variables," University of St. Gallen Department of Economics working paper series 2002 2002-11, Department of Economics, University of St. Gallen.
    15. Guido W. Imbens & Donald B. Rubin, 1997. "Estimating Outcome Distributions for Compliers in Instrumental Variables Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 555-574.
    16. Joshua D. Angrist & Guido W. Imbens & D.B. Rubin, 1993. "Identification of Causal Effects Using Instrumental Variables," NBER Technical Working Papers 0136, National Bureau of Economic Research, Inc.
    17. Boozer, Michael A. & Cacciola, Stephen E., 2001. "Inside the 'Black Box' of Project Star: Estimation of Peer Effects Using Experimental Data," Center Discussion Papers 28524, Yale University, Economic Growth Center.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Weili Ding & Steven F. Lehrer, 2010. "Estimating Treatment Effects from Contaminated Multiperiod Education Experiments: The Dynamic Impacts of Class Size Reductions," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 31-42, February.
    2. Steven Lehrer, 2005. "Class Size And Student Achievement: Experimental Estimates Of Who Benefits And Who Loses From Reductions," Working Paper 1046, Economics Department, Queen's University.
    3. Michael Lechner & Ruth Miquel, 2010. "Identification of the effects of dynamic treatments by sequential conditional independence assumptions," Empirical Economics, Springer, vol. 39(1), pages 111-137, August.
    4. Sally Hudson, "undated". "The Effects of Performance-Based Teacher Pay on Student Achievement," Discussion Papers 09-023, Stanford Institute for Economic Policy Research.
    5. Lechner, Michael, 2013. "Treatment effects and panel data," Economics Working Paper Series 1314, University of St. Gallen, School of Economics and Political Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weili Ding & Steven F. Lehrer, 2010. "Estimating Treatment Effects from Contaminated Multiperiod Education Experiments: The Dynamic Impacts of Class Size Reductions," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 31-42, February.
    2. Andreas Ammermueller & Jörn-Steffen Pischke, 2006. "Peer Effects in European Primary Schools: Evidence from PIRLS," CEE Discussion Papers 0065, Centre for the Economics of Education, LSE.
    3. Eric D. Gould & Victor Lavy & M. Daniele Paserman, 2009. "Does Immigration Affect the Long-Term Educational Outcomes of Natives? Quasi-Experimental Evidence," Economic Journal, Royal Economic Society, vol. 119(540), pages 1243-1269, October.
    4. Alexandra de Gendre & Nicolás Salamanca, 2020. "On the Mechanisms of Ability Peer Effects," Melbourne Institute Working Paper Series wp2020n19, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    5. Eric D. Gould & Victor Lavy & M. Daniele Paserman, 2004. "Immigrating to Opportunity: Estimating the Effect of School Quality Using a Natural Experiment on Ethiopians in Israel," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(2), pages 489-526.
    6. Diane Whitmore, 2005. "Resource and Peer Impacts on Girls' Academic Achievement: Evidence from a Randomized Experiment," American Economic Review, American Economic Association, vol. 95(2), pages 199-203, May.
    7. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    8. Vincent Boucher & Yann Bramoullé & Habiba Djebbari & Bernard Fortin, 2014. "Do Peers Affect Student Achievement? Evidence From Canada Using Group Size Variation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(1), pages 91-109, January.
    9. Fertig, Michael, 2003. "Educational Production, Endogenous Peer Group Formation and Class Composition - Evidence From the PISA 2000 Study," RWI Discussion Papers 2, RWI - Leibniz-Institut für Wirtschaftsforschung.
    10. Elizabeth U. Cascio & Diane Whitmore Schanzenbach, 2016. "First in the Class? Age and the Education Production Function," Education Finance and Policy, MIT Press, vol. 11(3), pages 225-250, Summer.
    11. Salvador Contreras & Frank Badua & Mitchell Adrian, 2012. "Peer Effects on Undergraduate Business Student Performance," International Review of Economic Education, Economics Network, University of Bristol, vol. 11(1), pages 57-66.
    12. ÖZGÜR, Onur & BISIN, Alberto, 2011. "Dynamic Linear Economies with Social Interactions," Cahiers de recherche 04-2011, Centre interuniversitaire de recherche en économie quantitative, CIREQ.
    13. Brock, William A. & Durlauf, Steven N., 2007. "Identification of binary choice models with social interactions," Journal of Econometrics, Elsevier, vol. 140(1), pages 52-75, September.
    14. Eric D. Gould & Victor Lavy & M. Daniele Paserman, 2011. "Sixty Years after the Magic Carpet Ride: The Long-Run Effect of the Early Childhood Environment on Social and Economic Outcomes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 78(3), pages 938-973.
    15. Yongyun Shin, 2012. "Do Black Children Benefit More From Small Classes? Multivariate Instrumental Variable Estimators With Ignorable Missing Data," Journal of Educational and Behavioral Statistics, , vol. 37(4), pages 543-574, August.
    16. Martin Huber & Anna Solovyeva, 2020. "Direct and Indirect Effects under Sample Selection and Outcome Attrition," Econometrics, MDPI, vol. 8(4), pages 1-25, December.
    17. Dinand Webbink, 2005. "Causal Effects in Education," Journal of Economic Surveys, Wiley Blackwell, vol. 19(4), pages 535-560, September.
    18. Yongyun Shin & Stephen W. Raudenbush, 2011. "The Causal Effect of Class Size on Academic Achievement," Journal of Educational and Behavioral Statistics, , vol. 36(2), pages 154-185, April.
    19. Murguia Baysse, Juan Manuel, 2013. "Essays on agricultural, financial economics and education," ISU General Staff Papers 201301010800004458, Iowa State University, Department of Economics.
    20. Wo[ss]mann, Ludger & West, Martin, 2006. "Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS," European Economic Review, Elsevier, vol. 50(3), pages 695-736, April.

    More about this item

    Keywords

    Education; Attrition; Non-Compliance; Sequential Difference in Difference; Class Size reduction;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecm:nasm04:252. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/essssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.