IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v65y2024i6d10.1007_s00362-024-01531-7.html
   My bibliography  Save this article

Welch’s t test is more sensitive to real world violations of distributional assumptions than student’s t test but logistic regression is more robust than either

Author

Listed:
  • David Curtis

    (UCL Genetics Institute)

Abstract

It has previously been pointed out that Student’s t test, which assumes that samples are drawn from populations with equal standard deviations, can have an inflated Type I error rate if this assumption is violated. Hence it has been recommended that Welch’s t test should be preferred. In the context of carrying out gene-wise weighted burden tests for detecting association of rare variants with psoriasis we observe that Welch’s test performs unsatisfactorily. We show that if the assumption of normality is violated and observations follow a Poisson distribution, then with unequal sample sizes Welch’s t test has an inflated Type I error rate, is systematically biased and is prone to produce extremely low p values. We argue that such data can arise in a variety of real world situations and believe that researchers should be aware of this issue. Student’s t test performs much better in this scenario but a likelihood ratio test based on logistic regression models performs better still and we suggest that this might generally be a preferable method to test for a difference in distributions between two samples. This research has been conducted using the UK Biobank Resource.

Suggested Citation

  • David Curtis, 2024. "Welch’s t test is more sensitive to real world violations of distributional assumptions than student’s t test but logistic regression is more robust than either," Statistical Papers, Springer, vol. 65(6), pages 3981-3989, August.
    • Handle: RePEc:spr:stpapr:v:65:y:2024:i:6:d:10.1007_s00362-024-01531-7
    DOI: 10.1007/s00362-024-01531-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-024-01531-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-024-01531-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dieter Rasch & Klaus Kubinger & Karl Moder, 2011. "The two-sample t test: pre-testing its assumptions does not pay off," Statistical Papers, Springer, vol. 52(1), pages 219-231, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leonie Kuen & Fiona Schürmann & Daniel Westmattelmann & Sophie Hartwig & Shay Tzafrir & Gerhard Schewe, 2023. "Trust transfer effects and associated risks in telemedicine adoption," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-22, December.
    2. Pablo Flores & María de Lourdes Palacios, 2024. "Goodness and lack of fit tests to pretest normality when comparing means," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 34(1), pages 119-129.
    3. Tsai, Arthur C. & Liou, Michelle & Simak, Maria & Cheng, Philip E., 2017. "On hyperbolic transformations to normality," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 250-266.
    4. Herbes, Carsten & Rilling, Benedikt & Holstenkamp, Lars, 2021. "Ready for new business models? Human and social capital in the management of renewable energy cooperatives in Germany," Energy Policy, Elsevier, vol. 156(C).
    5. I. Parra-Frutos, 2016. "Preliminary tests when comparing means," Computational Statistics, Springer, vol. 31(4), pages 1607-1631, December.
    6. David Weltman & Mark Eakin, 2014. "Incorporating Unusual Fonts and Planned Mistakes in Study Materials to Increase Business Student Focus and Retention," INFORMS Transactions on Education, INFORMS, vol. 15(1), pages 156-165, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:65:y:2024:i:6:d:10.1007_s00362-024-01531-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.