IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v37y2013i3-4p170-196.html
   My bibliography  Save this article

Covariance Adjustments for the Analysis of Randomized Field Experiments

Author

Listed:
  • Richard Berk
  • Emil Pitkin
  • Lawrence Brown
  • Andreas Buja
  • Edward George
  • Linda Zhao

Abstract

Background: It has become common practice to analyze randomized experiments using linear regression with covariates. Improved precision of treatment effect estimates is the usual motivation. In a series of important articles, David Freedman showed that this approach can be badly flawed. Recent work by Winston Lin offers partial remedies, but important problems remain. Results: In this article, we address those problems through a reformulation of the Neyman causal model. We provide a practical estimator and valid standard errors for the average treatment effect. Proper generalizations to well-defined populations can follow. Conclusion: In most applications, the use of covariates to improve precision is not worth the trouble.

Suggested Citation

  • Richard Berk & Emil Pitkin & Lawrence Brown & Andreas Buja & Edward George & Linda Zhao, 2013. "Covariance Adjustments for the Analysis of Randomized Field Experiments," Evaluation Review, , vol. 37(3-4), pages 170-196, June.
    • Handle: RePEc:sae:evarev:v:37:y:2013:i:3-4:p:170-196
    DOI: 10.1177/0193841X13513025
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X13513025
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X13513025?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Luke W. Miratrix & Jasjeet S. Sekhon & Bin Yu, 2013. "Adjusting treatment effect estimates by post-stratification in randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(2), pages 369-396, March.
    2. Heller, Ruth & Rosenbaum, Paul R. & Small, Dylan S., 2009. "Split Samples and Design Sensitivity in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 104(487), pages 1090-1101.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Matt Taddy & Matt Gardner & Liyun Chen & David Draper, 2016. "A Nonparametric Bayesian Analysis of Heterogenous Treatment Effects in Digital Experimentation," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 661-672, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.
    2. Hennessy Jonathan & Dasgupta Tirthankar & Miratrix Luke & Pattanayak Cassandra & Sarkar Pradipta, 2016. "A Conditional Randomization Test to Account for Covariate Imbalance in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 4(1), pages 61-80, March.
    3. Susan Athey & Dean Eckles & Guido W. Imbens, 2018. "Exact p-Values for Network Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 230-240, January.
    4. Raphael Calel & Antoine Dechezleprêtre, 2016. "Environmental Policy and Directed Technological Change: Evidence from the European Carbon Market," The Review of Economics and Statistics, MIT Press, vol. 98(1), pages 173-191, March.
    5. Bernard Black & José-Antonio Espín-Sánchez & Eric French & Kate Litvak, 2017. "The Long-Term Effect of Health Insurance on Near-Elderly Health and Mortality," American Journal of Health Economics, University of Chicago Press, vol. 3(3), pages 281-311, Summer.
    6. Anubhav Gupta & Thomas Luke Spreen, 2024. "Do tax credits benefit charities? Evidence from two states," Contemporary Economic Policy, Western Economic Association International, vol. 42(1), pages 94-109, January.
    7. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    8. Bas van der Klaauw & Sandra Vriend, 2015. "A Nonparametric Method for Predicting Survival Probabilities," Tinbergen Institute Discussion Papers 15-126/V, Tinbergen Institute.
    9. Donald P. Green & Winston Lin & Claudia Gerber, 2018. "Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments," Evaluation Review, , vol. 42(4), pages 391-422, August.
    10. Peter Z. Schochet, 2020. "Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 32-57, February.
    11. Kikuta,Kyosuke, 2022. "The drowning-out effect: voter turnout, uncertainty, and protests," IDE Discussion Papers 867, Institute of Developing Economies, Japan External Trade Organization(JETRO).
    12. Timothy R. Wojan, 2023. "Registered Report: Exploratory Analysis of Ownership Diversity and Innovation in the Annual Business Survey," Working Papers 23-11, Center for Economic Studies, U.S. Census Bureau.
    13. Ron Berman & Elea McDonnell Feit, 2024. "Latent Stratification for Incrementality Experiments," Marketing Science, INFORMS, vol. 43(4), pages 903-917, July.
    14. Jason J. Sauppe & Sheldon H. Jacobson, 2017. "The role of covariate balance in observational studies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(4), pages 323-344, June.
    15. Nicole E. Pashley & Luke W. Miratrix, 2021. "Insights on Variance Estimation for Blocked and Matched Pairs Designs," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 271-296, June.
    16. Bernhard Swoboda & Cathrin Puchert & Dirk Morschett, 2016. "Explaining the differing effects of corporate reputation across nations: a multilevel analysis," Journal of the Academy of Marketing Science, Springer, vol. 44(4), pages 454-473, July.
    17. Woon Sau Leung & Wei Song & Jie Chen, 2018. "Does Bank Stakeholder Orientation Enhance Financial Stability? Evidence from a Natural Experiment," Working Papers 2018-14, Swansea University, School of Management.
    18. Guihua Wang & Jun Li & Wallace J. Hopp, 2022. "An Instrumental Variable Forest Approach for Detecting Heterogeneous Treatment Effects in Observational Studies," Management Science, INFORMS, vol. 68(5), pages 3399-3418, May.
    19. Susan Athey & Guido Imbens, 2016. "The Econometrics of Randomized Experiments," Papers 1607.00698, arXiv.org.
    20. Lu, Jiannan, 2016. "Covariate adjustment in randomization-based causal inference for 2K factorial designs," Statistics & Probability Letters, Elsevier, vol. 119(C), pages 11-20.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:37:y:2013:i:3-4:p:170-196. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.