IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2206.07845.html
   My bibliography  Save this paper

Optimality of Matched-Pair Designs in Randomized Controlled Trials

Author

Listed:
  • Yuehao Bai

Abstract

In randomized controlled trials (RCTs), treatment is often assigned by stratified randomization. I show that among all stratified randomization schemes which treat all units with probability one half, a certain matched-pair design achieves the maximum statistical precision for estimating the average treatment effect (ATE). In an important special case, the optimal design pairs units according to the baseline outcome. In a simulation study based on datasets from 10 RCTs, this design lowers the standard error for the estimator of the ATE by 10% on average, and by up to 34%, relative to the original designs.

Suggested Citation

  • Yuehao Bai, 2022. "Optimality of Matched-Pair Designs in Randomized Controlled Trials," Papers 2206.07845, arXiv.org.
    • Handle: RePEc:arx:papers:2206.07845
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2206.07845
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Miriam Bruhn & David McKenzie, 2009. "In Pursuit of Balance: Randomization in Practice in Development Field Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
    2. Victor Chernozhukov & Iván Fernández‐Val & Ye Luo, 2018. "The Sorted Effects Method: Discovering Heterogeneous Effects Beyond Their Averages," Econometrica, Econometric Society, vol. 86(6), pages 1911-1938, November.
    3. Felipe Barrera-Osorio & Leigh L. Linden & Juan E. Saavedra, 2019. "Medium- and Long-Term Educational Consequences of Alternative Conditional Cash Transfer Designs: Experimental Evidence from Colombia," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 54-91, July.
    4. Suresh de Mel & David McKenzie & Christopher Woodruff, 2019. "Labor Drops: Experimental Evidence on the Return to Additional Labor in Microenterprises," American Economic Journal: Applied Economics, American Economic Association, vol. 11(1), pages 202-235, January.
    5. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    6. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    7. repec:adr:anecst:y:2008:i:91-92:p:09 is not listed on IDEAS
    8. Bold, Tessa & Kimenyi, Mwangi & Mwabu, Germano & Ng’ang’a, Alice & Sandefur, Justin, 2018. "Experimental evidence on scaling up education reforms in Kenya," Journal of Public Economics, Elsevier, vol. 168(C), pages 1-20.
    9. Kari Lock Morgan & Donald B. Rubin, 2015. "Rerandomization to Balance Tiers of Covariates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1412-1421, December.
    10. Erika Deserranno & Miri Stryjan & Munshi Sulaiman, 2019. "Leader Selection and Service Delivery in Community Groups: Experimental Evidence from Uganda," American Economic Journal: Applied Economics, American Economic Association, vol. 11(4), pages 240-267, October.
    11. Bruno Crépon & Florencia Devoto & Esther Duflo & William Parienté, 2015. "Estimating the Impact of Microcredit on Those Who Take It Up: Evidence from a Randomized Experiment in Morocco," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 123-150, January.
    12. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2019. "Inference under covariate‐adaptive randomization with multiple treatments," Quantitative Economics, Econometric Society, vol. 10(4), pages 1747-1785, November.
    13. Joshua Angrist & Victor Lavy, 2009. "The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial," American Economic Review, American Economic Association, vol. 99(4), pages 1384-1414, September.
    14. Rachel Glennerster & Kudzai Takavarasha, 2013. "Running Randomized Evaluations: A Practical Guide," Economics Books, Princeton University Press, edition 1, number 10085.
    15. Martin Abel & Rulof Burger & Eliana Carranza & Patrizio Piraino, 2019. "Bridging the Intention-Behavior Gap? The Effect of Plan-Making Prompts on Job Search and Employment," American Economic Journal: Applied Economics, American Economic Association, vol. 11(2), pages 284-301, April.
    16. P. A. Riach & J. Rich, 2002. "Field Experiments of Discrimination in the Market Place," Economic Journal, Royal Economic Society, vol. 112(483), pages 480-518, November.
    17. Alberto Abadie & Guido W. Imbens, 2008. "Estimation of the Conditional Variance in Paired Experiments," Annals of Economics and Statistics, GENES, issue 91-92, pages 175-187.
    18. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    19. Alan Gerber & Mitchell Hoffman & John Morgan & Collin Raymond, 2020. "One in a Million: Field Experiments on Perceived Closeness of the Election and Voter Turnout," American Economic Journal: Applied Economics, American Economic Association, vol. 12(3), pages 287-325, July.
    20. Colin B Fogarty, 2018. "Regression-assisted inference for the average treatment effect in paired experiments," Biometrika, Biometrika Trust, vol. 105(4), pages 994-1000.
    21. Jean N. Lee & Jonathan Morduch & Saravana Ravindran & Abu Shonchoy & Hassan Zaman, 2021. "Poverty and Migration in the Digital Age: Experimental Evidence on Mobile Banking in Bangladesh," American Economic Journal: Applied Economics, American Economic Association, vol. 13(1), pages 38-71, January.
    22. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    23. Martin Abel & Rulof Burger & Patrizio Piraino, 2020. "The Value of Reference Letters: Experimental Evidence from South Africa," American Economic Journal: Applied Economics, American Economic Association, vol. 12(3), pages 40-71, July.
    24. Colin B. Fogarty, 2018. "On mitigating the analytical limitations of finely stratified experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(5), pages 1035-1056, November.
    25. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    26. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, September.
    27. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    28. Jeanne Lafortune & Julio Riutort & José Tessada, 2018. "Role Models or Individual Consulting: The Impact of Personalizing Micro-entrepreneurship Training," American Economic Journal: Applied Economics, American Economic Association, vol. 10(4), pages 222-245, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jiang, Liang & Phillips, Peter C.B. & Tao, Yubo & Zhang, Yichong, 2023. "Regression-adjusted estimation of quantile treatment effects under covariate-adaptive randomizations," Journal of Econometrics, Elsevier, vol. 234(2), pages 758-776.
    2. Jizhou Liu, 2023. "Inference for Two-stage Experiments under Covariate-Adaptive Randomization," Papers 2301.09016, arXiv.org, revised Oct 2024.
    3. Yuehao Bai & Meng Hsuan Hsieh & Jizhou Liu & Max Tabord-Meehan, 2022. "Revisiting the Analysis of Matched-Pair and Stratified Experiments in the Presence of Attrition," Papers 2209.11840, arXiv.org, revised Oct 2023.
    4. Laurent Davezies & Guillaume Hollard & Pedro Vergara Merino, 2024. "Revisiting Randomization with the Cube Method," Papers 2407.13613, arXiv.org.
    5. Jinglong Zhao, 2023. "Adaptive Neyman Allocation," Papers 2309.08808, arXiv.org, revised Sep 2023.
    6. Bugni, Federico A. & Gao, Mengsi, 2023. "Inference under covariate-adaptive randomization with imperfect compliance," Journal of Econometrics, Elsevier, vol. 237(1).
    7. Yuehao Bai & Jizhou Liu & Azeem M. Shaikh & Max Tabord-Meehan, 2023. "On the Efficiency of Finely Stratified Experiments," Papers 2307.15181, arXiv.org, revised Aug 2024.
    8. Cai, Yong & Rafi, Ahnaf, 2024. "On the performance of the Neyman Allocation with small pilots," Journal of Econometrics, Elsevier, vol. 242(1).
    9. Guillermo Cruces & Dario Tortarolo & Gonzalo Vazquez-Bare, 2022. "Design of two-stage experiments with an application to spillovers in tax compliance," IFS Working Papers W22/32, Institute for Fiscal Studies.
    10. Yuehao Bai & Meng Hsuan Hsieh & Jizhou Liu & Max Tabord‐Meehan, 2024. "Revisiting the analysis of matched‐pair and stratified experiments in the presence of attrition," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(2), pages 256-268, March.
    11. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    12. Max Cytrynbaum, 2024. "Finely Stratified Rerandomization Designs," Papers 2407.03279, arXiv.org, revised Jul 2024.
    13. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    14. Yuehao Bai & Jizhou Liu & Max Tabord-Meehan, 2022. "Inference for Matched Tuples and Fully Blocked Factorial Designs," Papers 2206.04157, arXiv.org, revised Nov 2023.
    15. Yuehao Bai & Jizhou Liu & Azeem M. Shaikh & Max Tabord-Meehan, 2022. "Inference in Cluster Randomized Trials with Matched Pairs," Papers 2211.14903, arXiv.org, revised Aug 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liang Jiang & Xiaobin Liu & Peter C. B. Phillips & Yichong Zhang, 2024. "Bootstrap Inference for Quantile Treatment Effects in Randomized Experiments with Matched Pairs," The Review of Economics and Statistics, MIT Press, vol. 106(2), pages 542-556, March.
    2. Yuehao Bai, 2022. "Optimality of Matched-Pair Designs in Randomized Controlled Trials," American Economic Review, American Economic Association, vol. 112(12), pages 3911-3940, December.
    3. Yuehao Bai & Meng Hsuan Hsieh & Jizhou Liu & Max Tabord‐Meehan, 2024. "Revisiting the analysis of matched‐pair and stratified experiments in the presence of attrition," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(2), pages 256-268, March.
    4. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    5. Yuehao Bai & Joseph P. Romano & Azeem M. Shaikh, 2022. "Inference in Experiments With Matched Pairs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 1726-1737, October.
    6. Bai, Yuehao & Jiang, Liang & Romano, Joseph P. & Shaikh, Azeem M. & Zhang, Yichong, 2024. "Covariate adjustment in experiments with matched pairs," Journal of Econometrics, Elsevier, vol. 241(1).
    7. Yuehao Bai & Meng Hsuan Hsieh & Jizhou Liu & Max Tabord-Meehan, 2022. "Revisiting the Analysis of Matched-Pair and Stratified Experiments in the Presence of Attrition," Papers 2209.11840, arXiv.org, revised Oct 2023.
    8. Clément de Chaisemartin & Jaime Ramirez-Cuellar, 2024. "At What Level Should One Cluster Standard Errors in Paired and Small-Strata Experiments?," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 193-212, January.
    9. Pedro Carneiro & Sokbae Lee & Daniel Wilhelm, 2020. "Optimal data collection for randomized control trials," The Econometrics Journal, Royal Economic Society, vol. 23(1), pages 1-31.
    10. Jiang, Liang & Phillips, Peter C.B. & Tao, Yubo & Zhang, Yichong, 2023. "Regression-adjusted estimation of quantile treatment effects under covariate-adaptive randomizations," Journal of Econometrics, Elsevier, vol. 234(2), pages 758-776.
    11. Victor Chernozhukov & Mert Demirer & Esther Duflo & Ivan Fernandez-Val, 2017. "Generic machine learning inference on heterogenous treatment effects in randomized experiments," CeMMAP working papers 61/17, Institute for Fiscal Studies.
    12. Yuehao Bai & Jizhou Liu & Max Tabord-Meehan, 2022. "Inference for Matched Tuples and Fully Blocked Factorial Designs," Papers 2206.04157, arXiv.org, revised Nov 2023.
    13. Bugni, Federico A. & Gao, Mengsi, 2023. "Inference under covariate-adaptive randomization with imperfect compliance," Journal of Econometrics, Elsevier, vol. 237(1).
    14. Yichong Zhang & Xin Zheng, 2020. "Quantile treatment effects and bootstrap inference under covariate‐adaptive randomization," Quantitative Economics, Econometric Society, vol. 11(3), pages 957-982, July.
    15. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    16. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    17. Antonia Grohmann & Lukas Menkhoff & Helke Seitz, 2022. "The Effect of Personalized Feedback on Small Enterprises’ Finances in Uganda," Economic Development and Cultural Change, University of Chicago Press, vol. 70(3), pages 1197-1227.
    18. Novella, Rafael & Rosas-Shady, David & Freund, Richard, 2024. "Is online job training for all? Experimental evidence on the effects of a Coursera program in Costa Rica," Journal of Development Economics, Elsevier, vol. 169(C).
    19. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    20. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2019. "Inference under covariate‐adaptive randomization with multiple treatments," Quantitative Economics, Econometric Society, vol. 10(4), pages 1747-1785, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2206.07845. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.