IDEAS home Printed from https://ideas.repec.org/a/cup/polals/v16y2008i02p138-152_00.html
   My bibliography  Save this article

Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches

Author

Listed:
  • Green, Donald P.
  • Vavreck, Lynn

Abstract

Analysts of cluster-randomized field experiments have an array of estimation techniques to choose from. Using Monte Carlo simulation, we evaluate the properties of point estimates and standard errors (SEs) generated by ordinary least squares (OLS) as applied to both individual-level and cluster-level data. We also compare OLS to alternative random effects estimators, such as generalized least squares (GLS). Our simulations assess efficiency across a variety of scenarios involving varying sample sizes and numbers of clusters. Our results confirm that conventional OLS SEs are severely biased downward and that, for all estimators, gains in efficiency come mainly from increasing the number of clusters, not increasing the number of individuals within clusters. We find relatively minor differences across alternative estimation approaches, but GLS seems to enjoy a slight edge in terms of the efficiency of its point estimates and the accuracy of its SEs. We illustrate the application of alternative estimation approaches using a clustered experiment in which Rock the Vote TV advertisements were used to encourage young voters in 85 cable TV markets to vote in the 2004 presidential election.

Suggested Citation

  • Green, Donald P. & Vavreck, Lynn, 2008. "Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches," Political Analysis, Cambridge University Press, vol. 16(2), pages 138-152, April.
    • Handle: RePEc:cup:polals:v:16:y:2008:i:02:p:138-152_00
    as

    Download full text from publisher

    File URL: https://www.cambridge.org/core/product/identifier/S1047198700006768/type/journal_article
    File Function: link to article abstract page
    Download Restriction: no
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fangzhou Su & Peng Ding, 2021. "Model‐assisted analyses of cluster‐randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 994-1015, November.
    2. Miltiadis S. Chalikias & Georgios X. Papageorgiou & Dimitrios P. Zarogiannis, 2024. "Estimator Comparison for the Prediction of Election Results," Stats, MDPI, vol. 7(3), pages 1-14, July.
    3. Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    4. Rabie Mahssouni & Mohamed Noureddine Touijer & Mohamed Makhroute, 2022. "Employee Compensation, Training and Financial Performance during the COVID-19 Pandemic," JRFM, MDPI, vol. 15(12), pages 1-19, November.
    5. Beatrice Magistro, 2020. "Financial literacy and support for free trade in the UK," The World Economy, Wiley Blackwell, vol. 43(8), pages 2050-2069, August.
    6. Middleton Joel A. & Aronow Peter M., 2015. "Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments," Statistics, Politics and Policy, De Gruyter, vol. 6(1-2), pages 39-75, December.
    7. Susanne Berger & Nathaniel Graham & Achim Zeileis, 2017. "Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R," Working Papers 2017-12, Faculty of Economics and Statistics, Universität Innsbruck.
    8. Odunayo Magret Olarewaju & Stephen Oseko Migiro & Mabutho Sibanda, 2017. "Effect of Agency Costs on Executive Compensation in South African Commercial Banks," Journal of Economics and Behavioral Studies, AMH International, vol. 9(4), pages 25-37.
    9. Aronow Peter M. & Middleton Joel A., 2013. "A Class of Unbiased Estimators of the Average Treatment Effect in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 1(1), pages 135-154, June.
    10. Elizabeth Levy Paluck, 2010. "The Promising Integration of Qualitative Methods and Field Experiments," The ANNALS of the American Academy of Political and Social Science, , vol. 628(1), pages 59-71, March.
    11. von Hippel, Paul T. & Bellows, Laura & Osborne, Cynthia & Lincove, Jane Arnold & Mills, Nick, 2016. "Teacher quality differences between teacher preparation programs: How big? How reliable? Which programs are different?," Economics of Education Review, Elsevier, vol. 53(C), pages 31-45.
    12. Arturas Rozenas & Roya Talibova & Yuri M. Zhukov, 2024. "Fighting for Tyranny: State Repression and Combat Motivation," American Economic Journal: Applied Economics, American Economic Association, vol. 16(3), pages 44-75, July.
    13. Charlotte Cavaill, 2015. "Deservingness, Self-Interest and the Welfare State: Why Some Care More about Deservingness than Others and Why It Matters," LIS Working papers 652, LIS Cross-National Data Center in Luxembourg.
    14. Jeffrey Harden & Thomas Carsey, 2012. "Balancing constituency representation and party responsiveness in the US Senate: the conditioning effect of state ideological heterogeneity," Public Choice, Springer, vol. 150(1), pages 137-154, January.
    15. repec:rre:publsh:v:38:y:2008:i:2:p:251-69 is not listed on IDEAS
    16. Harden Jeffrey J., 2012. "Improving Statistical Inference with Clustered Data," Statistics, Politics and Policy, De Gruyter, vol. 3(1), pages 1-30, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cup:polals:v:16:y:2008:i:02:p:138-152_00. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kirk Stebbing (email available below). General contact details of provider: https://www.cambridge.org/pan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.