IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2109.09220.html
   My bibliography  Save this paper

Unifying Design-based Inference: On Bounding and Estimating the Variance of any Linear Estimator in any Experimental Design

Author

Listed:
  • Joel A. Middleton

Abstract

This paper provides a design-based framework for variance (bound) estimation in experimental analysis. Results are applicable to virtually any combination of experimental design, linear estimator (e.g., difference-in-means, OLS, WLS) and variance bound, allowing for unified treatment and a basis for systematic study and comparison of designs using matrix spectral analysis. A proposed variance estimator reproduces Eicker-Huber-White (aka. "robust", "heteroskedastic consistent", "sandwich", "White", "Huber-White", "HC", etc.) standard errors and "cluster-robust" standard errors as special cases. While past work has shown algebraic equivalences between design-based and the so-called "robust" standard errors under some designs, this paper motivates them for a wide array of design-estimator-bound triplets. In so doing, it provides a clearer and more general motivation for variance estimators.

Suggested Citation

  • Joel A. Middleton, 2021. "Unifying Design-based Inference: On Bounding and Estimating the Variance of any Linear Estimator in any Experimental Design," Papers 2109.09220, arXiv.org.
    • Handle: RePEc:arx:papers:2109.09220
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2109.09220
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported by the Neyman Model for Causal Inference? (Presentation)," Mathematica Policy Research Reports abfc39d59c714499b2fe42f68, Mathematica Policy Research.
    2. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported By the Neyman Model for Causal Inference?," Mathematica Policy Research Reports 782da2242fba458eb61752f96, Mathematica Policy Research.
    3. Hansen, Ben B. & Bowers, Jake, 2009. "Attributing Effects to a Cluster-Randomized Get-Out-the-Vote Campaign," Journal of the American Statistical Association, American Statistical Association, vol. 104(487), pages 873-885.
    4. Betsy Sinclair & Margaret McConnell & Donald P. Green, 2012. "Detecting Spillover Effects: Design and Analysis of Multilevel Experiments," American Journal of Political Science, John Wiley & Sons, vol. 56(4), pages 1055-1069, October.
    5. Middleton, Joel A., 2008. "Bias of the regression estimator for experiments using clustered random assignment," Statistics & Probability Letters, Elsevier, vol. 78(16), pages 2654-2659, November.
    6. Middleton Joel A. & Aronow Peter M., 2015. "Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments," Statistics, Politics and Policy, De Gruyter, vol. 6(1-2), pages 39-75, December.
    7. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    8. repec:mpr:mprres:6573 is not listed on IDEAS
    9. Samii, Cyrus & Aronow, Peter M., 2012. "On equivalencies between design-based and regression-based variance estimators for randomized experiments," Statistics & Probability Letters, Elsevier, vol. 82(2), pages 365-370.
    10. Lu, Jiannan, 2016. "Covariate adjustment in randomization-based causal inference for 2K factorial designs," Statistics & Probability Letters, Elsevier, vol. 119(C), pages 11-20.
    11. Arceneaux, Kevin & Nickerson, David W., 2009. "Modeling Certainty with Clustered Data: A Comparison of Methods," Political Analysis, Cambridge University Press, vol. 17(2), pages 177-190, April.
    12. Aronow Peter M. & Middleton Joel A., 2013. "A Class of Unbiased Estimators of the Average Treatment Effect in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 1(1), pages 135-154, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aronow Peter M. & Middleton Joel A., 2013. "A Class of Unbiased Estimators of the Average Treatment Effect in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 1(1), pages 135-154, June.
    2. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    3. Lu, Jiannan, 2016. "On randomization-based and regression-based inferences for 2K factorial designs," Statistics & Probability Letters, Elsevier, vol. 112(C), pages 72-78.
    4. Peter Z. Schochet, 2020. "Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 32-57, February.
    5. Middleton Joel A. & Aronow Peter M., 2015. "Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments," Statistics, Politics and Policy, De Gruyter, vol. 6(1-2), pages 39-75, December.
    6. Susan Athey & Guido Imbens, 2016. "The Econometrics of Randomized Experiments," Papers 1607.00698, arXiv.org.
    7. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    8. Zach Branson & Tirthankar Dasgupta, 2020. "Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework," International Statistical Review, International Statistical Institute, vol. 88(1), pages 101-121, April.
    9. repec:mpr:mprres:7638 is not listed on IDEAS
    10. repec:mpr:mprres:6965 is not listed on IDEAS
    11. Haoge Chang & Joel Middleton & P. M. Aronow, 2021. "Exact Bias Correction for Linear Adjustment of Randomized Controlled Trials," Papers 2110.08425, arXiv.org, revised Oct 2021.
    12. repec:mpr:mprres:6286 is not listed on IDEAS
    13. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    14. repec:mpr:mprres:7273 is not listed on IDEAS
    15. Kenneth Fortson & Natalya Verbitsky-Savitz & Emma Kopa & Philip Gleason, 2012. "Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates," Mathematica Policy Research Reports 27f871b5b7b94f3a80278a593, Mathematica Policy Research.
    16. repec:mpr:mprres:6094 is not listed on IDEAS
    17. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    18. John Deke, 2016. "Design and Analysis Considerations for Cluster Randomized Controlled Trials That Have a Small Number of Clusters," Evaluation Review, , vol. 40(5), pages 444-486, October.
    19. Peter Z. Schochet & Hanley Chiang, "undated". "Technical Methods Report: Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs," Mathematica Policy Research Reports 947d1823e3ff42208532a763d, Mathematica Policy Research.
    20. Melissa A. Clark & Philip Gleason & Christina Clark Tuttle & Marsha K. Silverberg, 2011. "Do Charter Schools Improve Student Achievement? Evidence from a National Randomized Study," Mathematica Policy Research Reports af41392138504f369930e6f2b, Mathematica Policy Research.
    21. Peter Z. Schochet, 2010. "The Late Pretest Problem in Randomized Control Trials of Education Interventions," Journal of Educational and Behavioral Statistics, , vol. 35(4), pages 379-406, August.
    22. repec:mpr:mprres:7443 is not listed on IDEAS
    23. Peter Z. Schochet, "undated". "The Late Pretest Problem in Randomized Control Trials of Education Interventions," Mathematica Policy Research Reports fb514df5dbb84a5dbea79865c, Mathematica Policy Research.
    24. Laura Blue & Gregory Peterson & Keith Kranker & Tessa Huffman & Alli Steiner & Amanda Markovitz & Malcolm Williams & Kate Stewart & Julia Rollison & Jia Pu & Thomas Concannon & Liisa Hiatt & Nabeel Qu, "undated". "Evaluation of the Million Hearts® Cardiovascular Disease Risk Reduction Model: Third Annual Report," Mathematica Policy Research Reports d4649f0778804c4eb0adcf2db, Mathematica Policy Research.
    25. Peter Z. Schochet, "undated". "Technical Methods Report: Statistical Power for Regression Discontinuity Designs in Education Evaluations," Mathematica Policy Research Reports 61fb6c057561451a8a6074508, Mathematica Policy Research.
    26. Ding Peng & Li Xinran & Miratrix Luke W., 2017. "Bridging Finite and Super Population Causal Inference," Journal of Causal Inference, De Gruyter, vol. 5(2), pages 1-8, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2109.09220. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.