IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0009788.html
   My bibliography  Save this article

A Robust Procedure for Comparing Multiple Means under Heteroscedasticity in Unbalanced Designs

Author

Listed:
  • Esther Herberich
  • Johannes Sikorski
  • Torsten Hothorn

Abstract

Investigating differences between means of more than two groups or experimental conditions is a routine research question addressed in biology. In order to assess differences statistically, multiple comparison procedures are applied. The most prominent procedures of this type, the Dunnett and Tukey-Kramer test, control the probability of reporting at least one false positive result when the data are normally distributed and when the sample sizes and variances do not differ between groups. All three assumptions are non-realistic in biological research and any violation leads to an increased number of reported false positive results. Based on a general statistical framework for simultaneous inference and robust covariance estimators we propose a new statistical multiple comparison procedure for assessing multiple means. In contrast to the Dunnett or Tukey-Kramer tests, no assumptions regarding the distribution, sample sizes or variance homogeneity are necessary. The performance of the new procedure is assessed by means of its familywise error rate and power under different distributions. The practical merits are demonstrated by a reanalysis of fatty acid phenotypes of the bacterium Bacillus simplex from the “Evolution Canyons” I and II in Israel. The simulation results show that even under severely varying variances, the procedure controls the number of false positive findings very well. Thus, the here presented procedure works well under biologically realistic scenarios of unbalanced group sizes, non-normality and heteroscedasticity.

Suggested Citation

  • Esther Herberich & Johannes Sikorski & Torsten Hothorn, 2010. "A Robust Procedure for Comparing Multiple Means under Heteroscedasticity in Unbalanced Designs," PLOS ONE, Public Library of Science, vol. 5(3), pages 1-8, March.
  • Handle: RePEc:plo:pone00:0009788
    DOI: 10.1371/journal.pone.0009788
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0009788
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0009788&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0009788?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Katharina J. Heinemann & Sun-Young Yang & Henryk Straube & Nieves Medina-Escobar & Marina Varbanova-Herde & Marco Herde & Sangkee Rhee & Claus-Peter Witte, 2021. "Initiation of cytosolic plant purine nucleotide catabolism involves a monospecific xanthosine monophosphate phosphatase," Nature Communications, Nature, vol. 12(1), pages 1-9, December.
    2. Daniel Caballe-Fontanet & Cristina Alvarez-Peregrina & Neus Busquet-Duran & Eduard Pedemonte-Sarrias & Miguel Angel Sanchez-Tena, 2020. "Improvement of the Quality of Life in Patients with Age-Related Macular Degeneration by Using Filters," IJERPH, MDPI, vol. 17(18), pages 1-7, September.
    3. Turner, Matthew D. & Moumouni, Oumarou, 2018. "The dividing of fields in Sudano-Sahelian West Africa: The roles of soil fertility variation and legal doctrine," Land Use Policy, Elsevier, vol. 77(C), pages 362-374.
    4. Marian Reiff & Erik Šoltés & Silvia Komara & Tatiana Šoltésová & Silvia Zelinová, 2022. "Segmentation and estimation of claim severity in motor third-party liability insurance through contrast analysis," Equilibrium. Quarterly Journal of Economics and Economic Policy, Institute of Economic Research, vol. 17(3), pages 803-842, September.
    5. Umlauft, Maria & Placzek, Marius & Konietschke, Frank & Pauly, Markus, 2019. "Wild bootstrapping rank-based procedures: Multiple testing in nonparametric factorial repeated measures designs," Journal of Multivariate Analysis, Elsevier, vol. 171(C), pages 176-192.
    6. Mario Hasler, 2016. "Heteroscedasticity: multiple degrees of freedom vs. sandwich estimation," Statistical Papers, Springer, vol. 57(1), pages 55-68, March.
    7. Iparraguirre, José Luis & Ma, Ruosi, 2015. "Efficiency in the provision of social care for older people. A three-stage Data Envelopment Analysis using self-reported quality of life," Socio-Economic Planning Sciences, Elsevier, vol. 49(C), pages 33-46.
    8. Konietschke Frank & Bösiger Sandra & Brunner Edgar & Hothorn Ludwig A., 2013. "Are Multiple Contrast Tests Superior to the ANOVA?," The International Journal of Biostatistics, De Gruyter, vol. 9(1), pages 63-73, August.
    9. Mario Hasler, 2016. "Heteroscedasticity: multiple degrees of freedom vs. sandwich estimation," Statistical Papers, Springer, vol. 57(1), pages 55-68, March.
    10. Philip Pallmann & Ludwig A. Hothorn, 2016. "Analysis of means: a generalized approach using R," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(8), pages 1541-1560, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0009788. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.