IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v37y2013i3-4p170-196.html
   My bibliography  Save this article

Covariance Adjustments for the Analysis of Randomized Field Experiments

Author

Listed:
  • Richard Berk
  • Emil Pitkin
  • Lawrence Brown
  • Andreas Buja
  • Edward George
  • Linda Zhao

Abstract

Background: It has become common practice to analyze randomized experiments using linear regression with covariates. Improved precision of treatment effect estimates is the usual motivation. In a series of important articles, David Freedman showed that this approach can be badly flawed. Recent work by Winston Lin offers partial remedies, but important problems remain. Results: In this article, we address those problems through a reformulation of the Neyman causal model. We provide a practical estimator and valid standard errors for the average treatment effect. Proper generalizations to well-defined populations can follow. Conclusion: In most applications, the use of covariates to improve precision is not worth the trouble.

Suggested Citation

  • Richard Berk & Emil Pitkin & Lawrence Brown & Andreas Buja & Edward George & Linda Zhao, 2013. "Covariance Adjustments for the Analysis of Randomized Field Experiments," Evaluation Review, , vol. 37(3-4), pages 170-196, June.
    • Handle: RePEc:sae:evarev:v:37:y:2013:i:3-4:p:170-196
    DOI: 10.1177/0193841X13513025
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X13513025
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X13513025?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Luke W. Miratrix & Jasjeet S. Sekhon & Bin Yu, 2013. "Adjusting treatment effect estimates by post-stratification in randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(2), pages 369-396, March.
    2. Heller, Ruth & Rosenbaum, Paul R. & Small, Dylan S., 2009. "Split Samples and Design Sensitivity in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 104(487), pages 1090-1101.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Matt Taddy & Matt Gardner & Liyun Chen & David Draper, 2016. "A Nonparametric Bayesian Analysis of Heterogenous Treatment Effects in Digital Experimentation," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 661-672, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.
    2. Haoge Chang & Joel Middleton & P. M. Aronow, 2021. "Exact Bias Correction for Linear Adjustment of Randomized Controlled Trials," Papers 2110.08425, arXiv.org, revised Oct 2021.
    3. Hennessy Jonathan & Dasgupta Tirthankar & Miratrix Luke & Pattanayak Cassandra & Sarkar Pradipta, 2016. "A Conditional Randomization Test to Account for Covariate Imbalance in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 4(1), pages 61-80, March.
    4. Susan Athey & Dean Eckles & Guido W. Imbens, 2018. "Exact p-Values for Network Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 230-240, January.
    5. Raphael Calel & Antoine Dechezleprêtre, 2016. "Environmental Policy and Directed Technological Change: Evidence from the European Carbon Market," The Review of Economics and Statistics, MIT Press, vol. 98(1), pages 173-191, March.
    6. Bernard Black & José-Antonio Espín-Sánchez & Eric French & Kate Litvak, 2017. "The Long-Term Effect of Health Insurance on Near-Elderly Health and Mortality," American Journal of Health Economics, University of Chicago Press, vol. 3(3), pages 281-311, Summer.
    7. Anubhav Gupta & Thomas Luke Spreen, 2024. "Do tax credits benefit charities? Evidence from two states," Contemporary Economic Policy, Western Economic Association International, vol. 42(1), pages 94-109, January.
    8. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    9. Bas van der Klaauw & Sandra Vriend, 2015. "A Nonparametric Method for Predicting Survival Probabilities," Tinbergen Institute Discussion Papers 15-126/V, Tinbergen Institute.
    10. Dimitris Alexakis & Leonidas G. Barbopoulos, 2020. "Incentive‐compatible contracts in merger negotiations: The role of acquirer idiosyncratic stock return volatility," Financial Markets, Institutions & Instruments, John Wiley & Sons, vol. 29(1), pages 3-40, February.
    11. Donald P. Green & Winston Lin & Claudia Gerber, 2018. "Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments," Evaluation Review, , vol. 42(4), pages 391-422, August.
    12. Ding, Peng, 2021. "The Frisch–Waugh–Lovell theorem for standard errors," Statistics & Probability Letters, Elsevier, vol. 168(C).
    13. Peter Z. Schochet, 2020. "Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 32-57, February.
    14. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
    15. Middleton Joel A. & Aronow Peter M., 2015. "Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments," Statistics, Politics and Policy, De Gruyter, vol. 6(1-2), pages 39-75, December.
    16. Peter M. Steiner, 2011. "Propensity Score Methods for Causal Inference: On the Relative Importance of Covariate Selection, Reliable Measurement, and Choice of Propensity Score Technique," Working Papers 09, AlmaLaurea Inter-University Consortium.
    17. Kikuta,Kyosuke, 2022. "The drowning-out effect: voter turnout, uncertainty, and protests," IDE Discussion Papers 867, Institute of Developing Economies, Japan External Trade Organization(JETRO).
    18. M. Deniz Dalman & Mari W. Buche & Junhong Min, 2019. "The Differential Influence of Identification on Ethical Judgment: The Role of Brand Love," Journal of Business Ethics, Springer, vol. 158(3), pages 875-891, September.
    19. Timothy R. Wojan, 2023. "Registered Report: Exploratory Analysis of Ownership Diversity and Innovation in the Annual Business Survey," Working Papers 23-11, Center for Economic Studies, U.S. Census Bureau.
    20. Ian D. Gow & David F. Larcker & Peter C. Reiss, 2016. "Causal Inference in Accounting Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 477-523, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:37:y:2013:i:3-4:p:170-196. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.