IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/eedf2eac4d4c4d8e869052c1d4b2d566.html
   My bibliography  Save this paper

Multi-Armed RCTs: A Design-Based Framework

Author

Listed:
  • Peter Z. Schochet

Abstract

This report describes how design-based estimators for the two-group design need to be modified for the multi-armed design when comparing pairs of research groups to each other.

Suggested Citation

  • Peter Z. Schochet, "undated". "Multi-Armed RCTs: A Design-Based Framework," Mathematica Policy Research Reports eedf2eac4d4c4d8e869052c1d, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:eedf2eac4d4c4d8e869052c1d4b2d566
    as

    Download full text from publisher

    File URL: https://www.mathematica.org/-/media/publications/pdfs/education/2017/multi-armed-rcts.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Z. Schochet, "undated". "An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations," Mathematica Policy Research Reports dbb4b45215a740819b08ca578, Mathematica Policy Research.
    2. Peter Z. Schochet & Hanley S. Chiang, 2011. "Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs," Journal of Educational and Behavioral Statistics, , vol. 36(3), pages 307-345, June.
    3. Howard S. Bloom, 1984. "Accounting for No-Shows in Experimental Evaluation Designs," Evaluation Review, , vol. 8(2), pages 225-246, April.
    4. repec:mpr:mprres:6192 is not listed on IDEAS
    5. Castleman, Benjamin L. & Page, Lindsay C., 2015. "Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates?," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 144-160.
    6. Paul A. Games & John F. Howell, 1976. "Pairwise Multiple Comparison Procedures with Unequal N’s and/or Variances: A Monte Carlo Study," Journal of Educational and Behavioral Statistics, , vol. 1(2), pages 113-125, June.
    7. repec:mpr:mprres:7071 is not listed on IDEAS
    8. repec:mpr:mprres:7723 is not listed on IDEAS
    9. repec:mpr:mprres:6574 is not listed on IDEAS
    10. Kraft, Matthew A. & Rogers, Todd, 2015. "The underutilized potential of teacher-to-parent communication: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
    11. Tan, Zhiqiang, 2010. "Marginal and Nested Structural Models Using Instrumental Variables," Journal of the American Statistical Association, American Statistical Association, vol. 105(489), pages 157-169.
    12. Peter Z. Schochet, 2009. "An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations," Evaluation Review, , vol. 33(6), pages 539-567, December.
    13. Donald B. Rubin, 1977. "Assignment to Treatment Group on the Basis of a Covariate," Journal of Educational and Behavioral Statistics, , vol. 2(1), pages 1-26, March.
    14. Roberto Agodini & Barbara Harris & Sally Atkins-Burnett & Sheila Heaviside & Timothy Novak & Robert Murphy, "undated". "Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools," Mathematica Policy Research Reports 467194cd2d6f4cbaba7e23745, Mathematica Policy Research.
    15. Qi Long & Roderick J. A. Little & Xihong Lin, 2010. "Estimating causal effects in trials involving multitreatment arms subject to non‐compliance: a Bayesian framework," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 59(3), pages 513-531, May.
    16. Roberto Agodini, 2009. "Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools (Conference Paper)," Mathematica Policy Research Reports a08d661306a843fa89aff4c30, Mathematica Policy Research.
    17. Jing Cheng & Dylan S. Small, 2006. "Bounds on causal effects in three‐arm trials with non‐compliance," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(5), pages 815-836, November.
    18. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, January.
    19. repec:mpr:mprres:7337 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dana Rotz & Paul Burkander & Kenneth Fortson & Sheena McConnell & Peter Schochet & Mary Grider & Linda Molinari & Elias Sanchez-Eppler, "undated". "Providing Public Workforce Services to Job Seekers: 15-Month Impact Findings on the WIA Adult and Dislocated Worker Programs (Technical Supplement)," Mathematica Policy Research Reports 1153336bee1c4a969d5341ef4, Mathematica Policy Research.
    2. Peter Z. Schochet, 2020. "The Complier Average Causal Effect Parameter for Multiarmed RCTs," Evaluation Review, , vol. 44(5-6), pages 410-436, October.
    3. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    4. Dana Rotz & Paul Burkander & Mary Grider & Kenneth Fortson & Linda Molinari & Elias Sanchez-Eppler & Lindsay Cattell, "undated". "Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs, Technical Supplement," Mathematica Policy Research Reports db04b33db50a4d45824da5a65, Mathematica Policy Research.
    5. repec:mpr:mprres:7031 is not listed on IDEAS
    6. Duncan Chaplin & Arif Mamun & Ali Protik & John Schurrer & Divya Vohra & Kristine Bos & Hannah Burak & Laura Meyer & Anca Dumitrescu & Christopher Ksoll & Thomas Cook, "undated". "Grid Electricity Expansion in Tanzania by MCC: Findings from a Rigorous Impact Evaluation, Final Report," Mathematica Policy Research Reports 144768f69008442e96369195e, Mathematica Policy Research.
    7. Emilie Bagby & Kristine Bos & Anca Dumitrescu & Nicholas Ingwersen & Matt Sloan, "undated". "Niger NECS Impact Evaluation Report," Mathematica Policy Research Reports a759ff92090a4c2aba4caefe6, Mathematica Policy Research.
    8. Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    9. Arif A. Mamun & Lori Timmins & David C. Stapleton, "undated". "Prospects for an Impact Evaluation of Project SEARCH: An Evaluability Assessment," Mathematica Policy Research Reports 29231db872d5434984051a1bd, Mathematica Policy Research.
    10. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    11. Jiannan Lu & Peng Ding & Tirthankar Dasgupta, 2018. "Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 540-567, October.
    12. Peter Z. Schochet & Ronald D'Amico & Jillian Berk & Sarah Dolfin & Nathan Wozny, "undated". "Estimated Impacts for Participants in the Trade Adjustment Assistance (TAA) Program Under the 2002 Amendments," Mathematica Policy Research Reports 582d8723f6884d4eb7a3f95a4, Mathematica Policy Research.
    13. Karnani, Mohit, 2016. "Freshmen teachers and college major choice: Evidence from a random assignment in Chile," MPRA Paper 76062, University Library of Munich, Germany.
    14. Berlinski, Samuel & Busso, Matías & Dinkelman, Taryn & Martínez, Claudia, 2021. "Reducing Parent-School Information Gaps and Improving Education Outcomes: Evidence from High-Frequency Text Messages," IDB Publications (Working Papers) 11234, Inter-American Development Bank.
    15. Douglas H. Clements & Roberto Agodini & Barbara Harris, "undated". "Instructional Practices and Student Math Achievement: Correlations from a Study of Math Curricula (Issue Brief)," Mathematica Policy Research Reports 1e18ce07a82f42479daa643f7, Mathematica Policy Research.
    16. Purvi Sevak & Matthew Kehn & Todd Honeycutt & Gina Livermore, "undated". "Minnesota Substantial Gainful Activity (SGA) Project Demonstration: Final Evaluation Report," Mathematica Policy Research Reports 4e67883bec6d4c29b0b0df529, Mathematica Policy Research.
    17. repec:mpr:mprres:7907 is not listed on IDEAS
    18. repec:mpr:mprres:7273 is not listed on IDEAS
    19. Kalena E. Cortes & Hans Fricke & Susanna Loeb & David S. Song, 2018. "Too little or too much? Actionable Advice in an Early-Childhood Text Messaging Experiment," NBER Working Papers 24827, National Bureau of Economic Research, Inc.
    20. Kalena E. Cortes & Hans D.U. Fricke & Susanna Loeb & David S. Song & Benjamin N. York, 2019. "When Behavioral Barriers are Too High or Low – How Timing Matters for Parenting Interventions," NBER Working Papers 25964, National Bureau of Economic Research, Inc.
    21. Kimberly Boller & Sally Atkins-Burnett & Elizabeth M. Malone & Gail P. Baxter & Jerry West, "undated". "Compendium of Student, Teacher, and Classroom Measures Used in NCEE Evaluations of Educational Interventions, Volume I: Measures Selection Approaches and Compendium Development Methods," Mathematica Policy Research Reports 07a8a75ba5634b3a813e22bc6, Mathematica Policy Research.
    22. Bonilla-Mejía, Leonardo & Bottan, Nicolas L. & Ham, Andrés, 2019. "Information policies and higher education choices experimental evidence from Colombia," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 83(C).
    23. John Deke & Lisa Dragoset, "undated". "Statistical Power for Regression Discontinuity Designs in Education: Empirical Estimates of Design Effects Relative to Randomized Controlled Trials," Mathematica Policy Research Reports a4f1d03eb7bf427a8983d4736, Mathematica Policy Research.

    More about this item

    Keywords

    Randomized controlled trials; multi-armed designs; impact estimation; quantitative methods; causal inference; education interventions;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:eedf2eac4d4c4d8e869052c1d4b2d566. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.