IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/a0c005c003c242308a92c02dcca08793.html
   My bibliography  Save this paper

Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs

Author

Listed:
  • Peter Z. Schochet

Abstract

This report describes a novel statistical theory for estimating impacts for randomized controlled trials (RCTs) for a wide range of designs used in social policy research.

Suggested Citation

  • Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:a0c005c003c242308a92c02dcca08793
    as

    Download full text from publisher

    File URL: https://www.mathematica.org/-/media/publications/pdfs/education/statistical_theory.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Z. Schochet, 2013. "Estimators for Clustered Education RCTs Using the Neyman Model for Causal Inference," Journal of Educational and Behavioral Statistics, , vol. 38(3), pages 219-238, June.
    2. Guido W. Imbens & Michal Kolesár, 2016. "Robust Standard Errors in Small Samples: Some Practical Advice," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 701-712, October.
    3. Robert B. Olsen & Larry L. Orr & Stephen H. Bell & Elizabeth A. Stuart, 2013. "External Validity in Policy Evaluations That Choose Sites Purposively," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(1), pages 107-121, January.
    4. James Heckman & Jeffrey Smith & Christopher Taber, 1994. "Accounting for Dropouts in Evaluations of Social Experiments," NBER Technical Working Papers 0166, National Bureau of Economic Research, Inc.
    5. Peter Z. Schochet, "undated". "An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations," Mathematica Policy Research Reports dbb4b45215a740819b08ca578, Mathematica Policy Research.
    6. Peter Z. Schochet & Hanley S. Chiang, 2011. "Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs," Journal of Educational and Behavioral Statistics, , vol. 36(3), pages 307-345, June.
    7. James J. Heckman, 2008. "Econometric Causality," International Statistical Review, International Statistical Institute, vol. 76(1), pages 1-27, April.
    8. repec:mpr:mprres:7090 is not listed on IDEAS
    9. Peter Z. Schochet, "undated". "Statistical Power for Random Assignment Evaluations of Education Programs (Journal Article)," Mathematica Policy Research Reports a110ff428db043f4a6f55d349, Mathematica Policy Research.
    10. Philip Gleason & Melissa Clark & Christina Clark Tuttle & Emily Dwoyer, "undated". "The Evaluation of Charter School Impacts," Mathematica Policy Research Reports 3066da11915a4b04a77b38848, Mathematica Policy Research.
    11. Elizabeth Tipton, 2013. "Improving Generalizations From Experiments Using Propensity Score Subclassification," Journal of Educational and Behavioral Statistics, , vol. 38(3), pages 239-266, June.
    12. Hong, Guanglei & Raudenbush, Stephen W., 2006. "Evaluating Kindergarten Retention Policy: A Case Study of Causal Inference for Multilevel Observational Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 901-910, September.
    13. Yang L. & Tsiatis A. A., 2001. "Efficiency Study of Estimators for a Treatment Effect in a Pretest-Posttest Trial," The American Statistician, American Statistical Association, vol. 55, pages 314-321, November.
    14. Henry May & Irma Perez-Johnson & Joshua Haimson & Samina Sattar & Phil Gleason, "undated". "Using State Tests in Education Experiments: A Discussion of the Issues," Mathematica Policy Research Reports 18840c8e7e1f4f78a40d1acbe, Mathematica Policy Research.
    15. repec:mpr:mprres:7071 is not listed on IDEAS
    16. repec:mpr:mprres:6574 is not listed on IDEAS
    17. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    18. MacKinnon, James G., 2011. "Thirty Years of Heteroskedasticity-Robust Inference," Queen's Economics Department Working Papers 273816, Queen's University - Department of Economics.
    19. repec:mpr:mprres:6573 is not listed on IDEAS
    20. Peter Z. Schochet, "undated". "Do Typical RCTs of Education Interventions Have Sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes? (Journal Article)," Mathematica Policy Research Reports 80367b32df294d0dae76b082d, Mathematica Policy Research.
    21. Green, Donald P. & Vavreck, Lynn, 2008. "Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches," Political Analysis, Cambridge University Press, vol. 16(2), pages 138-152, April.
    22. repec:mpr:mprres:3180 is not listed on IDEAS
    23. repec:mpr:mprres:5863 is not listed on IDEAS
    24. Hausman, Jerry & Palmer, Christopher, 2012. "Heteroskedasticity-robust inference in finite samples," Economics Letters, Elsevier, vol. 116(2), pages 232-235.
    25. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    26. Peter Z. Schochet, 2008. "Statistical Power for Random Assignment Evaluations of Education Programs," Journal of Educational and Behavioral Statistics, , vol. 33(1), pages 62-87, March.
    27. repec:mpr:mprres:6676 is not listed on IDEAS
    28. repec:mpr:mprres:3181 is not listed on IDEAS
    29. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    30. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    31. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported by the Neyman Model for Causal Inference? (Presentation)," Mathematica Policy Research Reports abfc39d59c714499b2fe42f68, Mathematica Policy Research.
    32. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported By the Neyman Model for Causal Inference?," Mathematica Policy Research Reports 782da2242fba458eb61752f96, Mathematica Policy Research.
    33. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    34. Joshua D. Angrist & Victor Lavy, 2002. "The Effect of High School Matriculation Awards: Evidence from Randomized Trials," NBER Working Papers 9389, National Bureau of Economic Research, Inc.
    35. repec:mpr:mprres:7293 is not listed on IDEAS
    36. repec:mpr:mprres:8128 is not listed on IDEAS
    37. repec:mpr:mprres:6720 is not listed on IDEAS
    38. Luke W. Miratrix & Jasjeet S. Sekhon & Bin Yu, 2013. "Adjusting treatment effect estimates by post-stratification in randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(2), pages 369-396, March.
    39. repec:mpr:mprres:6440 is not listed on IDEAS
    40. Philip Gleason & Melissa Clark & Christina Clark Tuttle & Emily Dwoyer, 2010. "The Evaluation of Charter School Impacts (Presentation)," Mathematica Policy Research Reports 770e250b2ef343a3b1ec8c932, Mathematica Policy Research.
    41. A. D. Roy, 1951. "Some Thoughts On The Distribution Of Earnings," Oxford Economic Papers, Oxford University Press, vol. 3(2), pages 135-146.
    42. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, January.
    43. Daniel P. Mayer & Paul E. Peterson & David E. Myers & Christina Clark Tuttle & William G. Howell, 2002. "School Choice in New York City After Three Years: An Evaluation of the School Choice Scholarships Program," Mathematica Policy Research Reports bd29adb569094778a5981be0e, Mathematica Policy Research.
    44. Peter Z. Schochet, "undated". "Statistical Power for Random Assignment Evaluations of Education Programs," Mathematica Policy Research Reports 6749d31ad72d4acf988f7dce5, Mathematica Policy Research.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Z. Schochet, 2020. "Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 32-57, February.
    2. Tim Kautz & Peter Z. Schochet & Charles Tilley, "undated". "Comparing Impact Findings from Design-Based and Model-Based Methods: An Empirical Investigation," Mathematica Policy Research Reports b7656ddce20f4007b71836e99, Mathematica Policy Research.
    3. Peter Z. Schochet, "undated". "The Late Pretest Problem in Randomized Control Trials of Education Interventions," Mathematica Policy Research Reports fb514df5dbb84a5dbea79865c, Mathematica Policy Research.
    4. Peter Z. Schochet & Hanley Chiang, "undated". "Technical Methods Report: Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs," Mathematica Policy Research Reports 947d1823e3ff42208532a763d, Mathematica Policy Research.
    5. Peter Z. Schochet, 2010. "The Late Pretest Problem in Randomized Control Trials of Education Interventions," Journal of Educational and Behavioral Statistics, , vol. 35(4), pages 379-406, August.
    6. Peter Z. Schochet, "undated". "Technical Methods Report: Statistical Power for Regression Discontinuity Designs in Education Evaluations," Mathematica Policy Research Reports 61fb6c057561451a8a6074508, Mathematica Policy Research.
    7. repec:mpr:mprres:8128 is not listed on IDEAS
    8. Susan Athey & Guido Imbens, 2016. "The Econometrics of Randomized Experiments," Papers 1607.00698, arXiv.org.
    9. Aronow Peter M. & Middleton Joel A., 2013. "A Class of Unbiased Estimators of the Average Treatment Effect in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 1(1), pages 135-154, June.
    10. repec:mpr:mprres:6286 is not listed on IDEAS
    11. Kenneth Fortson & Natalya Verbitsky-Savitz & Emma Kopa & Philip Gleason, 2012. "Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates," Mathematica Policy Research Reports 27f871b5b7b94f3a80278a593, Mathematica Policy Research.
    12. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    13. repec:mpr:mprres:6094 is not listed on IDEAS
    14. Donald P. Green & Winston Lin & Claudia Gerber, 2018. "Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments," Evaluation Review, , vol. 42(4), pages 391-422, August.
    15. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    16. Middleton Joel A. & Aronow Peter M., 2015. "Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments," Statistics, Politics and Policy, De Gruyter, vol. 6(1-2), pages 39-75, December.
    17. James J. Heckman & Rodrigo Pinto, 2022. "Causality and Econometrics," NBER Working Papers 29787, National Bureau of Economic Research, Inc.
    18. Heckman, James & Pinto, Rodrigo, 2024. "Econometric causality: The central role of thought experiments," Journal of Econometrics, Elsevier, vol. 243(1).
    19. Jianghao Chu & Tae-Hwy Lee & Aman Ullah & Haifeng Xu, 2020. "Exact Distribution of the F-statistic under Heteroskedasticity of Unknown Form for Improved Inference," Working Papers 202027, University of California at Riverside, Department of Economics.
    20. Peter Z. Schochet, 2013. "Estimators for Clustered Education RCTs Using the Neyman Model for Causal Inference," Journal of Educational and Behavioral Statistics, , vol. 38(3), pages 219-238, June.
    21. repec:mpr:mprres:6372 is not listed on IDEAS
    22. Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.
    23. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    24. Peter Z. Schochet, 2021. "Statistical Power for Estimating Treatment Effects Using Difference-in-Differences and Comparative Interrupted Time Series Designs with Variation in Treatment Timing," Papers 2102.06770, arXiv.org, revised Oct 2021.

    More about this item

    Keywords

    Randomized Controlled Trials; Impact Estimation; Quantitative Methods; Causal Inference; Education Interventions;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:a0c005c003c242308a92c02dcca08793. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.