IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2302.03687.html
   My bibliography  Save this paper

Covariate Adjustment in Stratified Experiments

Author

Listed:
  • Max Cytrynbaum

Abstract

This paper studies covariate adjusted estimation of the average treatment effect in stratified experiments. We work in a general framework that includes matched tuples designs, coarse stratification, and complete randomization as special cases. Regression adjustment with treatment-covariate interactions is known to weakly improve efficiency for completely randomized designs. By contrast, we show that for stratified designs such regression estimators are generically inefficient, potentially even increasing estimator variance relative to the unadjusted benchmark. Motivated by this result, we derive the asymptotically optimal linear covariate adjustment for a given stratification. We construct several feasible estimators that implement this efficient adjustment in large samples. In the special case of matched pairs, for example, the regression including treatment, covariates, and pair fixed effects is asymptotically optimal. We also provide novel asymptotically exact inference methods that allow researchers to report smaller confidence intervals, fully reflecting the efficiency gains from both stratification and adjustment. Simulations and an empirical application demonstrate the value of our proposed methods.

Suggested Citation

  • Max Cytrynbaum, 2023. "Covariate Adjustment in Stratified Experiments," Papers 2302.03687, arXiv.org, revised Jul 2024.
  • Handle: RePEc:arx:papers:2302.03687
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2302.03687
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    2. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    3. Alberto Abadie & Guido W. Imbens, 2008. "Estimation of the Conditional Variance in Paired Experiments," Annals of Economics and Statistics, GENES, issue 91-92, pages 175-187.
    4. Akanksha Negi & Jeffrey M. Wooldridge, 2021. "Revisiting regression adjustment in experiments with heterogeneous treatment effects," Econometric Reviews, Taylor & Francis Journals, vol. 40(5), pages 504-534, April.
    5. Amy Finkelstein & Sarah Taubman & Bill Wright & Mira Bernstein & Jonathan Gruber & Joseph P. Newhouse & Heidi Allen & Katherine Baicker, 2012. "The Oregon Health Insurance Experiment: Evidence from the First Year," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(3), pages 1057-1106.
    6. repec:adr:anecst:y:2008:i:91-92:p:09 is not listed on IDEAS
    7. Colin B Fogarty, 2018. "Regression-assisted inference for the average treatment effect in paired experiments," Biometrika, Biometrika Trust, vol. 105(4), pages 994-1000.
    8. repec:cup:cbooks:9780521885881 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuehao Bai & Hongchang Guo & Azeem M. Shaikh & Max Tabord-Meehan, 2023. "Inference in Experiments with Matched Pairs and Imperfect Compliance," Papers 2307.13094, arXiv.org, revised Jun 2024.
    2. Bai, Yuehao & Jiang, Liang & Romano, Joseph P. & Shaikh, Azeem M. & Zhang, Yichong, 2024. "Covariate adjustment in experiments with matched pairs," Journal of Econometrics, Elsevier, vol. 241(1).
    3. Max Cytrynbaum, 2024. "Finely Stratified Rerandomization Designs," Papers 2407.03279, arXiv.org, revised Jul 2024.
    4. Liang Jiang & Liyao Li & Ke Miao & Yichong Zhang, 2023. "Adjustment with Many Regressors Under Covariate-Adaptive Randomizations," Papers 2304.08184, arXiv.org, revised Feb 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiang, Liang & Phillips, Peter C.B. & Tao, Yubo & Zhang, Yichong, 2023. "Regression-adjusted estimation of quantile treatment effects under covariate-adaptive randomizations," Journal of Econometrics, Elsevier, vol. 234(2), pages 758-776.
    2. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    3. Bai, Yuehao & Jiang, Liang & Romano, Joseph P. & Shaikh, Azeem M. & Zhang, Yichong, 2024. "Covariate adjustment in experiments with matched pairs," Journal of Econometrics, Elsevier, vol. 241(1).
    4. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
    5. Liang Jiang & Oliver B. Linton & Haihan Tang & Yichong Zhang, 2022. "Improving Estimation Efficiency via Regression-Adjustment in Covariate-Adaptive Randomizations with Imperfect Compliance," Papers 2201.13004, arXiv.org, revised Jun 2023.
    6. Yuehao Bai, 2022. "Optimality of Matched-Pair Designs in Randomized Controlled Trials," Papers 2206.07845, arXiv.org.
    7. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    8. Liang Jiang & Xiaobin Liu & Peter C. B. Phillips & Yichong Zhang, 2024. "Bootstrap Inference for Quantile Treatment Effects in Randomized Experiments with Matched Pairs," The Review of Economics and Statistics, MIT Press, vol. 106(2), pages 542-556, March.
    9. Timothy B. Armstrong & Michal Kolesár, 2021. "Finite‐Sample Optimal Estimation and Inference on Average Treatment Effects Under Unconfoundedness," Econometrica, Econometric Society, vol. 89(3), pages 1141-1177, May.
    10. Yuehao Bai & Jizhou Liu & Max Tabord-Meehan, 2022. "Inference for Matched Tuples and Fully Blocked Factorial Designs," Papers 2206.04157, arXiv.org, revised Nov 2023.
    11. John A. List & Ian Muir & Gregory K. Sun, 2022. "Using Machine Learning for Efficient Flexible Regression Adjustment in Economic Experiments," NBER Working Papers 30756, National Bureau of Economic Research, Inc.
    12. Isaiah Andrews & Emily Oster, 2017. "A Simple Approximation for Evaluating External Validity Bias," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    13. Max Cytrynbaum, 2024. "Finely Stratified Rerandomization Designs," Papers 2407.03279, arXiv.org, revised Jul 2024.
    14. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    15. Liang Jiang & Liyao Li & Ke Miao & Yichong Zhang, 2023. "Adjustment with Many Regressors Under Covariate-Adaptive Randomizations," Papers 2304.08184, arXiv.org, revised Feb 2024.
    16. Sookyo Jeong & Hongseok Namkoong, 2020. "Assessing External Validity Over Worst-case Subpopulations," Papers 2007.02411, arXiv.org, revised Feb 2022.
    17. Elira Kuka, 2020. "Quantifying the Benefits of Social Insurance: Unemployment Insurance and Health," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 490-505, July.
    18. Charles Courtemanche & James Marton & Benjamin Ukert & Aaron Yelowitz & Daniela Zapata, 2018. "Early Effects of the Affordable Care Act on Health Care Access, Risky Health Behaviors, and Self‐Assessed Health," Southern Economic Journal, John Wiley & Sons, vol. 84(3), pages 660-691, January.
    19. Clément de Chaisemartin & Jaime Ramirez-Cuellar, 2024. "At What Level Should One Cluster Standard Errors in Paired and Small-Strata Experiments?," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 193-212, January.
    20. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2302.03687. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.