IDEAS home Printed from https://ideas.repec.org/p/red/sed010/1336.html
   My bibliography  Save this paper

Inference with Imperfect Randomization: The Case of the Perry Preschool Program

Author

Listed:
  • Rodrigo Pinto

    (University of Chicago)

  • Azeem Shaikh

    (University of Chicago)

  • Adam Yavitz

    (University of Chicago)

  • James Heckman

    (University of Chicago)

Abstract

This paper considers the problem of inference about the effect of a program on multiple outcomes when assignment of treatment status is imperfectly randomized. Here, by imperfect randomization we mean that treatment status may have been reassigned after the initial randomization on the basis of observed or unobserved characteristics. We develop our methodology in the context of the High/Scope Perry Preschool program. We find significant effects of the program on a number of different outcomes of interest, including outcomes related to criminal activity for males, even after accounting for the imperfectness of the randomization and the multiplicity of hypotheses. On the other hand, effects of the program on other outcomes of interest, including outcomes related to criminal behavior for females, are no longer significant after more careful scrutiny.

Suggested Citation

  • Rodrigo Pinto & Azeem Shaikh & Adam Yavitz & James Heckman, 2010. "Inference with Imperfect Randomization: The Case of the Perry Preschool Program," 2010 Meeting Papers 1336, Society for Economic Dynamics.
    • Handle: RePEc:red:sed010:1336
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joseph P. Romano & Azeem M. Shaikh, 2010. "Inference for the Identified Set in Partially Identified Econometric Models," Econometrica, Econometric Society, vol. 78(1), pages 169-211, January.
    2. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program," Quantitative Economics, Econometric Society, vol. 1(1), pages 1-46, July.
    3. James Heckman & Rodrigo Pinto & Peter Savelyev, 2013. "Understanding the Mechanisms through Which an Influential Early Childhood Program Boosted Adult Outcomes," American Economic Review, American Economic Association, vol. 103(6), pages 2052-2086, October.
    4. Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008. "Formalized Data Snooping Based On Generalized Error Rates," Econometric Theory, Cambridge University Press, vol. 24(2), pages 404-447, April.
    5. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    6. Heckman, James J. & Moon, Seong Hyeok & Pinto, Rodrigo & Savelyev, Peter A. & Yavitz, Adam, 2010. "The rate of return to the HighScope Perry Preschool Program," Journal of Public Economics, Elsevier, vol. 94(1-2), pages 114-128, February.
    7. James J. Heckman & Ganesh Karapakula, 2019. "The Perry Preschoolers at Late Midlife: A Study in Design-Specific Inference," Working Papers 2019-034, Human Capital and Economic Opportunity Working Group.
    8. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2010. "Hypothesis Testing in Econometrics," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 75-104, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    2. Aviva Aron-Dine & Liran Einav & Amy Finkelstein, 2013. "The RAND Health Insurance Experiment, Three Decades Later," Journal of Economic Perspectives, American Economic Association, vol. 27(1), pages 197-222, Winter.
    3. Arouna, Aminou & Michler, Jeffrey D. & Lokossou, Jourdain C., 2021. "Contract farming and rural transformation: Evidence from a field experiment in Benin," Journal of Development Economics, Elsevier, vol. 151(C).
    4. Daniela Del Boca & Christopher Flinn & Matthew Wiswall, 2016. "Transfers to Households with Children and Child Development," Economic Journal, Royal Economic Society, vol. 126(596), pages 136-183, October.
    5. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    6. Sneha Elango & Jorge Luis García & James J. Heckman & Andrés Hojman, 2015. "Early Childhood Education," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 235-297, National Bureau of Economic Research, Inc.
    7. James J. Heckman & Ganesh Karapakula, 2019. "The Perry Preschoolers at Late Midlife: A Study in Design-Specific Inference," Working Papers 2019-034, Human Capital and Economic Opportunity Working Group.
    8. Laurent Davezies & Guillaume Hollard & Pedro Vergara Merino, 2024. "Revisiting Randomization with the Cube Method," Papers 2407.13613, arXiv.org.
    9. James J Heckman & Ganesh Karapakula, 2021. "Using a satisficing model of experimenter decision-making to guide finite-sample inference for compromised experiments," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 1-39.
    10. John A. List & Azeem M. Shaikh & Atom Vayalinkal, 2023. "Multiple testing with covariate adjustment in experimental economics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(6), pages 920-939, September.
    11. Yuehao Bai & Joseph P. Romano & Azeem M. Shaikh, 2022. "Inference in Experiments With Matched Pairs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 1726-1737, October.
    12. Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.
    13. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    14. Asoni, Andrea, 2011. "Intelligence, Self-confidence and Entrepreneurship," Working Paper Series 887, Research Institute of Industrial Economics.
    15. Fougère, Denis & Filatriau, Olivier & Tô, Maxime, 2013. "Will Sooner Be Better? The Impact of Early Preschool Enrollment on Cognitive and Noncognitive Achievement of Children," CEPR Discussion Papers 9480, C.E.P.R. Discussion Papers.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Heckman, James & Pinto, Rodrigo & Shaikh, Azeem M., 2024. "Dealing with imperfect randomization: Inference for the highscope perry preschool program," Journal of Econometrics, Elsevier, vol. 243(1).
    2. Doyle, Orla & Harmon, Colm & Heckman, James J. & Logue, Caitriona & Moon, Seong Hyeok, 2017. "Early skill formation and the efficiency of parental investment: A randomized controlled trial of home visiting," Labour Economics, Elsevier, vol. 45(C), pages 40-58.
    3. John A. List & Azeem M. Shaikh & Atom Vayalinkal, 2023. "Multiple testing with covariate adjustment in experimental economics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(6), pages 920-939, September.
    4. James J Heckman & Ganesh Karapakula, 2021. "Using a satisficing model of experimenter decision-making to guide finite-sample inference for compromised experiments," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 1-39.
    5. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. repec:spo:wpmain:info:hdl:2441/6s39gt704s95upu27ma7s3p6q8 is not listed on IDEAS
    7. Fabian Kosse & Thomas Deckers & Pia Pinger & Hannah Schildberg-Hörisch & Armin Falk, 2020. "The Formation of Prosociality: Causal Evidence on the Role of Social Environment," Journal of Political Economy, University of Chicago Press, vol. 128(2), pages 434-467.
    8. James J. Heckman & Stefano Mosso, 2014. "The Economics of Human Development and Social Mobility," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 689-733, August.
    9. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1593-1660.
    10. Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021. "Enhancing Human Capital in Children: A Case Study on Scaling," TSE Working Papers 21-1196, Toulouse School of Economics (TSE), revised Oct 2023.
    11. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    12. Orla Doyle & Colm Harmon & James J. Heckman & Caitriona Logue & Seong Moon, 2013. "Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes," NBER Working Papers 19316, National Bureau of Economic Research, Inc.
    13. Sandner, Malte & Jungmann, Tanja, 2017. "Gender-specific effects of early childhood intervention: Evidence from a randomized controlled trial," Labour Economics, Elsevier, vol. 45(C), pages 59-78.
    14. Orla Doyle, 2017. "The First 2,000 Days and Child Skills: Evidence from a Randomized Experiment of Home Visiting," Working Papers 2017-054, Human Capital and Economic Opportunity Working Group.
    15. Cortés, Darwin & Maldonado, Darío & Gallego, Juan & Charpak, Nathalie & Tessier, Rejean & Ruiz, Juan Gabriel & Hernandez, José Tiberio & Uriza, Felipe & Pico, Julieth, 2022. "Comparing long-term educational effects of two early childhood health interventions," Journal of Health Economics, Elsevier, vol. 86(C).
    16. Yann Algan & Elizabeth Beasley & Frank Vitaro & Richard Tremblay, 2014. "The Impact of Non-Cognitive Skills Training on Academic and Non-academic Trajectories: From Childhood to Early Adulthood," Working Papers hal-03429906, HAL.
    17. Fougère, Denis & Filatriau, Olivier & Tô, Maxime, 2013. "Will Sooner Be Better? The Impact of Early Preschool Enrollment on Cognitive and Noncognitive Achievement of Children," CEPR Discussion Papers 9480, C.E.P.R. Discussion Papers.
    18. Jorge Luis García & James J. Heckman, 2023. "Parenting Promotes Social Mobility Within and Across Generations," Annual Review of Economics, Annual Reviews, vol. 15(1), pages 349-388, September.
    19. Brutti, Zelda & Montolio, Daniel, 2021. "Preventing criminal minds: Early education access and adult offending behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 97-126.
    20. Yann Algan & Elizabeth Beasley & Frank Vitaro & Richard E Tremblay, 2014. "The Impact of Non-Cognitive Skills Training on Academic and Non-academic Trajectories: From Childhood to Early Adulthood," Sciences Po publications info:hdl:2441/6s39gt704s9, Sciences Po.
    21. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.

    More about this item

    JEL classification:

    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • J13 - Labor and Demographic Economics - - Demographic Economics - - - Fertility; Family Planning; Child Care; Children; Youth

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:red:sed010:1336. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christian Zimmermann (email available below). General contact details of provider: https://edirc.repec.org/data/sedddea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.