IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/b7656ddce20f4007b71836e99dd8bd87.html
   My bibliography  Save this paper

Comparing Impact Findings from Design-Based and Model-Based Methods: An Empirical Investigation

Author

Listed:
  • Tim Kautz
  • Peter Z. Schochet
  • Charles Tilley

Abstract

This study investigates how much of a practical difference it makes to use design-based methods versus more conventional model-based methods.

Suggested Citation

  • Tim Kautz & Peter Z. Schochet & Charles Tilley, "undated". "Comparing Impact Findings from Design-Based and Model-Based Methods: An Empirical Investigation," Mathematica Policy Research Reports b7656ddce20f4007b71836e99, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:b7656ddce20f4007b71836e99dd8bd87
    as

    Download full text from publisher

    File URL: https://www.mathematica.org/-/media/publications/pdfs/education/2017/design-model-based-methods.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Z. Schochet, 2013. "Estimators for Clustered Education RCTs Using the Neyman Model for Causal Inference," Journal of Educational and Behavioral Statistics, , vol. 38(3), pages 219-238, June.
    2. repec:mpr:mprres:6174 is not listed on IDEAS
    3. Philip Gleason & Melissa Clark & Christina Clark Tuttle & Emily Dwoyer, "undated". "The Evaluation of Charter School Impacts," Mathematica Policy Research Reports 3066da11915a4b04a77b38848, Mathematica Policy Research.
    4. Jill Constantine & Daniel Player & Tim Silva & Kristin Hallgren & Mary Grider & John Deke, 2009. "An Evaluation of Teachers Trained Through Different Routes to Certification," Mathematica Policy Research Reports 1a9e2910a01843c2babbc8c64, Mathematica Policy Research.
    5. Paul T. Decker & Daniel P. Mayer & Steven Glazerman, "undated". "The Effects of Teach For America on Students: Findings from a National Evaluation," Mathematica Policy Research Reports c8b5eb6d499c465c86a96bee4, Mathematica Policy Research.
    6. repec:mpr:mprres:6573 is not listed on IDEAS
    7. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    8. repec:mpr:mprres:6676 is not listed on IDEAS
    9. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    10. Peter Z. Schochet & John Burghardt & Steven Glazerman, 2001. "National Job Corps Study: The Impacts of Job Corps on Participants' Employment and Related Outcomes," Mathematica Policy Research Reports db6c4204b8e1408bb0c6289ec, Mathematica Policy Research.
    11. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported by the Neyman Model for Causal Inference? (Presentation)," Mathematica Policy Research Reports abfc39d59c714499b2fe42f68, Mathematica Policy Research.
    12. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported By the Neyman Model for Causal Inference?," Mathematica Policy Research Reports 782da2242fba458eb61752f96, Mathematica Policy Research.
    13. Duncan Chaplin & Martha Bleeker & Kevin Booker, "undated". "Roads to Success: Estimated Impacts of an Education and Career Planning Program During Middle School," Mathematica Policy Research Reports 68a9bbd46c67405fba0358d5c, Mathematica Policy Research.
    14. repec:mpr:mprres:4150 is not listed on IDEAS
    15. repec:mpr:mprres:7962 is not listed on IDEAS
    16. repec:mpr:mprres:7293 is not listed on IDEAS
    17. Melissa Clark & Hanley Chiang & Sheena McConnell & Tim Silva & Kathy Sonnenfeld & Anastasia Erbe & Mike Puma, "undated". "The Effectiveness of Secondary Math Teachers from Teach For America and the Teaching Fellows Programs (Presentation)," Mathematica Policy Research Reports 32d4ed3f97b14a208a78435db, Mathematica Policy Research.
    18. Jill Constantine, 2009. "Evaluation of Teachers Trained Through Different Routes to Certification," Mathematica Policy Research Reports 6e759a791c77408a816add035, Mathematica Policy Research.
    19. repec:mpr:mprres:7890 is not listed on IDEAS
    20. repec:mpr:mprres:6720 is not listed on IDEAS
    21. repec:mpr:mprres:6175 is not listed on IDEAS
    22. repec:mpr:mprres:7338 is not listed on IDEAS
    23. Philip Gleason & Melissa Clark & Christina Clark Tuttle & Emily Dwoyer, 2010. "The Evaluation of Charter School Impacts (Presentation)," Mathematica Policy Research Reports 770e250b2ef343a3b1ec8c932, Mathematica Policy Research.
    24. repec:mpr:mprres:6556 is not listed on IDEAS
    25. repec:mpr:mprres:8154 is not listed on IDEAS
    26. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, October.
    27. Melissa A. Clark & Hanley S. Chiang & Tim Silva & Sheena McConnell & Kathy Sonnenfeld & Anastasia Erbe & Michael Puma, "undated". "The Effectiveness of Secondary Math Teachers from Teach For America and the Teaching Fellows Programs," Mathematica Policy Research Reports ad5192faecc9490288484de35, Mathematica Policy Research.
    28. repec:mpr:mprres:4012 is not listed on IDEAS
    29. repec:mpr:mprres:2951 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peter Z. Schochet, "undated". "What is Design-Based Causal Inference and Why Should I Use It?," Mathematica Policy Research Reports 82a207630f374ef6a7dfd4a60, Mathematica Policy Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lisa Dragoset & Jaime Thomas & Mariesa Herrmann & John Deke & Susanne James-Burdumy & Cheryl Graczewski & Andrea Boyle & Rachel Upton & Courtney Tanenbaum & Jessica Giffin, "undated". "School Improvement Grants: Implementation and Effectiveness (Final Report)," Mathematica Policy Research Reports 76bce3f4bb0944f29a481fae0, Mathematica Policy Research.
    2. Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    3. Lisa Dragoset & Jaime Thomas & Mariesa Herrmann & John Deke & Susanne James-Burdumy & Cheryl Graczewski & Andrea Boyle & Courtney Tanenbaum & Jessica Giffin & Rachel Upton, "undated". "Race to the Top: Implementation and Relationship to Student Outcomes," Mathematica Policy Research Reports 4169441544e144b08777d967e, Mathematica Policy Research.
    4. Roland G. Fryer, Jr, 2016. "The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments," NBER Working Papers 22130, National Bureau of Economic Research, Inc.
    5. Lisa Dragoset & Susanne James-Burdumy & Kristin Hallgren & Irma Perez-Johnson & Mariesa Herrmann & Christina Tuttle & Megan Hague Angus & Rebecca Herman & Matthew Murray & Courtney Tanenbaum & Cheryl , 2015. "Usage of Practices Promoted by School Improvement Grants," Mathematica Policy Research Reports 8e99f01663504ef5b9f8357f6, Mathematica Policy Research.
    6. Lisa Dragoset & Susanne James-Burdumy & Kristin Hallgren & Irma Perez-Johnson & Mariesa Herrmann & Christina Tuttle & Megan Hague Angus & Rebecca Herman & Matthew Murray & Corutney Tanenbaum & Cheryl , 2015. "Usage of Policies and Practices Promoted by Race to the Top," Mathematica Policy Research Reports 4b296d4310f843acad1f80b5a, Mathematica Policy Research.
    7. Peter Z. Schochet, 2020. "Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 32-57, February.
    8. Justus J. Randolph & Anaya Bryson & Lakshmi Menon & Stephen Michaels & Debra Leigh Walls Rosenstein & Warren McPherson, 2016. "PROTOCOL: Montessori Education for Improving Academic and Social/Behavioral Outcomes for Elementary Students," Campbell Systematic Reviews, John Wiley & Sons, vol. 12(1), pages 1-32.
    9. Matthew Johnson & Alicia Demers & Cleo Jacobs Johnson & Claudia Gentile, "undated". "Ewing Marion Kauffman School Evaluation Impact Report Year 4," Mathematica Policy Research Reports 5175e15711054006b860f263e, Mathematica Policy Research.
    10. Matthew Johnson & Eric Lundquist & Cleo Jacobs Johnson & Claudia Gentile, "undated". "Kauffman School Evaluation Long-Term Outcomes Report: Year 2," Mathematica Policy Research Reports 08feceb41e2a451e88fd9b926, Mathematica Policy Research.
    11. repec:mpr:mprres:8128 is not listed on IDEAS
    12. Matthew Johnson & Alicia Demers, "undated". "Ewing Marion Kauffman School Year 6 Impacts," Mathematica Policy Research Reports bd1f54f237124b218114062d5, Mathematica Policy Research.
    13. Matthew Johnson & Alicia Demers & Cleo Jacobs Johnson & Claudia Gentile, "undated". "Ewing Marion Kauffman School Evaluation Impact Report Year 5," Mathematica Policy Research Reports 917049103ecc4f93ba12260b5, Mathematica Policy Research.
    14. Vegas, E & Ganimian, A. J., 2013. "Theory and Evidence on Teacher Policies in Developed and Developing Countries," Working Paper 104291, Harvard University OpenScholar.
    15. Vilsa E. Curto & Roland G. Fryer Jr., 2014. "The Potential of Urban Boarding Schools for the Poor: Evidence from SEED," Journal of Labor Economics, University of Chicago Press, vol. 32(1), pages 65-93.
    16. Hanley S. Chiang & Melissa A. Clark & Sheena McConnell, "undated". "Supplying Disadvantaged Schools with Effective Teachers: Experimental Evidence on Secondary Math Teachers from Teach For America," Mathematica Policy Research Reports 4623b3b72e154403be21e3878, Mathematica Policy Research.
    17. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    18. Lu, Jiannan, 2016. "On randomization-based and regression-based inferences for 2K factorial designs," Statistics & Probability Letters, Elsevier, vol. 112(C), pages 72-78.
    19. Antecol, Heather & Eren, Ozkan & Ozbeklik, Serkan, 2013. "The effect of Teach for America on the distribution of student achievement in primary school: Evidence from a randomized experiment," Economics of Education Review, Elsevier, vol. 37(C), pages 113-125.
    20. Peter Z. Schochet & Hanley Chiang, "undated". "Technical Methods Report: Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs," Mathematica Policy Research Reports 947d1823e3ff42208532a763d, Mathematica Policy Research.
    21. Joe Regan-Stansfield, 2016. "Do good primary schools perform even better as academies?," Working Papers 141167564, Lancaster University Management School, Economics Department.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:b7656ddce20f4007b71836e99dd8bd87. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.