IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0230798.html
   My bibliography  Save this article

Applying univariate vs. multivariate statistics to investigate therapeutic efficacy in (pre)clinical trials: A Monte Carlo simulation study on the example of a controlled preclinical neurotrauma trial

Author

Listed:
  • Hristo Todorov
  • Emily Searle-White
  • Susanne Gerber

Abstract

Background: Small sample sizes combined with multiple correlated endpoints pose a major challenge in the statistical analysis of preclinical neurotrauma studies. The standard approach of applying univariate tests on individual response variables has the advantage of simplicity of interpretation, but it fails to account for the covariance/correlation in the data. In contrast, multivariate statistical techniques might more adequately capture the multi-dimensional pathophysiological pattern of neurotrauma and therefore provide increased sensitivity to detect treatment effects. Results: We systematically evaluated the performance of univariate ANOVA, Welch’s ANOVA and linear mixed effects models versus the multivariate techniques, ANOVA on principal component scores and MANOVA tests by manipulating factors such as sample and effect size, normality and homogeneity of variance in computer simulations. Linear mixed effects models demonstrated the highest power when variance between groups was equal or variance ratio was 1:2. In contrast, Welch’s ANOVA outperformed the remaining methods with extreme variance heterogeneity. However, power only reached acceptable levels of 80% in the case of large simulated effect sizes and at least 20 measurements per group or moderate effects with at least 40 replicates per group. In addition, we evaluated the capacity of the ordination techniques, principal component analysis (PCA), redundancy analysis (RDA), linear discriminant analysis (LDA), and partial least squares discriminant analysis (PLS-DA) to capture patterns of treatment effects without formal hypothesis testing. While LDA suffered from a high false positive rate due to multicollinearity, PCA, RDA, and PLS-DA were robust and PLS-DA outperformed PCA and RDA in capturing a true treatment effect pattern. Conclusions: Multivariate tests do not provide an appreciable increase in power compared to univariate techniques to detect group differences in preclinical studies. However, PLS-DA seems to be a useful ordination technique to explore treatment effect patterns without formal hypothesis testing.

Suggested Citation

  • Hristo Todorov & Emily Searle-White & Susanne Gerber, 2020. "Applying univariate vs. multivariate statistics to investigate therapeutic efficacy in (pre)clinical trials: A Monte Carlo simulation study on the example of a controlled preclinical neurotrauma trial," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-20, March.
  • Handle: RePEc:plo:pone00:0230798
    DOI: 10.1371/journal.pone.0230798
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0230798
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0230798&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0230798?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Beverly S Muhlhausler & Frank H Bloomfield & Matthew W Gillman, 2013. "Whole Animal Experiments Should Be More Like Human Randomized Controlled Trials," PLOS Biology, Public Library of Science, vol. 11(2), pages 1-6, February.
    2. Peter E Batchelor & Taryn E Wills & Peta Skeers & Camila R Battistuzzo & Malcolm R Macleod & David W Howells & Emily S Sena, 2013. "Meta-Analysis of Pre-Clinical Studies of Early Decompression in Acute Spinal Cord Injury: A Battle of Time and Pressure," PLOS ONE, Public Library of Science, vol. 8(8), pages 1-1, August.
    3. Jessica L. Nielson & Jesse Paquette & Aiwen W. Liu & Cristian F. Guandique & C. Amy Tovar & Tomoo Inoue & Karen-Amanda Irvine & John C. Gensel & Jennifer Kloke & Tanya C. Petrossian & Pek Y. Lum & Gun, 2015. "Topological data analysis for discovery in preclinical spinal cord injury and traumatic brain injury," Nature Communications, Nature, vol. 6(1), pages 1-12, December.
    4. Adam R Ferguson & Karen-Amanda Irvine & John C Gensel & Jessica L Nielson & Amity Lin & Johnathan Ly & Mark R Segal & Rajiv R Ratan & Jacqueline C Bresnahan & Michael S Beattie, 2013. "Derivation of Multivariate Syndromic Outcome Metrics for Consistent Testing across Multiple Models of Cervical Spinal Cord Injury in Rats," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-13, March.
    5. Carol Kilkenny & William J Browne & Innes C Cuthill & Michael Emerson & Douglas G Altman, 2010. "Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research," PLOS Biology, Public Library of Science, vol. 8(6), pages 1-5, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Constance Holman & Sophie K Piper & Ulrike Grittner & Andreas Antonios Diamantaras & Jonathan Kimmelman & Bob Siegerink & Ulrich Dirnagl, 2016. "Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-12, January.
    2. David Baker & Katie Lidster & Ana Sottomayor & Sandra Amor, 2014. "Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies," PLOS Biology, Public Library of Science, vol. 12(1), pages 1-6, January.
    3. Jennifer A Hirst & Jeremy Howick & Jeffrey K Aronson & Nia Roberts & Rafael Perera & Constantinos Koshiaris & Carl Heneghan, 2014. "The Need for Randomization in Animal Trials: An Overview of Systematic Reviews," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    4. Dean A Fergusson & Marc T Avey & Carly C Barron & Mathew Bocock & Kristen E Biefer & Sylvain Boet & Stephane L Bourque & Isidora Conic & Kai Chen & Yuan Yi Dong & Grace M Fox & Ronald B George & Neil , 2019. "Reporting preclinical anesthesia study (REPEAT): Evaluating the quality of reporting in the preclinical anesthesiology literature," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-15, May.
    5. Zhongwei Xu & Bingze Xu & Susanna L. Lundström & Àlex Moreno-Giró & Danxia Zhao & Myriam Martin & Erik Lönnblom & Qixing Li & Alexander Krämer & Changrong Ge & Lei Cheng & Bibo Liang & Dongmei Tong & , 2023. "A subset of type-II collagen-binding antibodies prevents experimental arthritis by inhibiting FCGR3 signaling in neutrophils," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    6. Nathalie Percie du Sert & Viki Hurst & Amrita Ahluwalia & Sabina Alam & Marc T Avey & Monya Baker & William J Browne & Alejandra Clark & Innes C Cuthill & Ulrich Dirnagl & Michael Emerson & Paul Garne, 2020. "The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-12, July.
    7. Vivian Leung & Frédérik Rousseau-Blass & Guy Beauchamp & Daniel S J Pang, 2018. "ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesi," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-13, May.
    8. Claudia Kurreck & Esmeralda Castaños-Vélez & Dorette Freyer & Sonja Blumenau & Ingo Przesdzing & Rene Bernard & Ulrich Dirnagl, 2020. "Improving quality of preclinical academic research through auditing: A feasibility study," PLOS ONE, Public Library of Science, vol. 15(10), pages 1-15, October.
    9. Beverly S Muhlhausler & Frank H Bloomfield & Matthew W Gillman, 2013. "Whole Animal Experiments Should Be More Like Human Randomized Controlled Trials," PLOS Biology, Public Library of Science, vol. 11(2), pages 1-6, February.
    10. Rebecca A Nishi & Anna Badner & Mitra J Hooshmand & Dana A Creasman & Hongli Liu & Aileen J Anderson, 2020. "The effects of mouse strain and age on a model of unilateral cervical contusion spinal cord injury," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-19, June.
    11. Bertha Estrella & Elena N. Naumova & Magda Cepeda & Trudy Voortman & Peter D. Katsikis & Hemmo A. Drexhage, 2019. "Effects of Air Pollution on Lung Innate Lymphoid Cells: Review of In Vitro and In Vivo Experimental Studies," IJERPH, MDPI, vol. 16(13), pages 1-15, July.
    12. Emily M Wong & Fern Tablin & Edward S Schelegle, 2020. "Comparison of nonparametric and parametric methods for time-frequency heart rate variability analysis in a rodent model of cardiovascular disease," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-15, November.
    13. Jonathan Mauricio Muñoz-Cabrera & Adrián Gabriel Sandoval-Hernández & Andrea Niño & Tatiana Báez & Angie Bustos-Rangel & Gloria Patricia Cardona-Gómez & Alejandro Múnera & Gonzalo Arboleda, 2019. "Bexarotene therapy ameliorates behavioral deficits and induces functional and molecular changes in very-old Triple Transgenic Mice model of Alzheimer´s disease," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-22, October.
    14. Konstantinos K Tsilidis & Orestis A Panagiotou & Emily S Sena & Eleni Aretouli & Evangelos Evangelou & David W Howells & Rustam Al-Shahi Salman & Malcolm R Macleod & John P A Ioannidis, 2013. "Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases," PLOS Biology, Public Library of Science, vol. 11(7), pages 1-10, July.
    15. Olivia Hogue & Tucker Harvey & Dena Crozier & Claire Sonneborn & Abagail Postle & Hunter Block-Beach & Eashwar Somasundaram & Francis J May & Monica Snyder Braun & Felicia L Pasadyn & Khandi King & Ca, 2022. "Statistical practice and transparent reporting in the neurosciences: Preclinical motor behavioral experiments," PLOS ONE, Public Library of Science, vol. 17(3), pages 1-17, March.
    16. Marta Liliana Musskopf & Amanda Finger Stadler & Ulf ME Wikesjö & Cristiano Susin, 2022. "The minipig intraoral dental implant model: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 17(2), pages 1-17, February.
    17. Stefan Boehme & Bastian Duenges & Klaus U Klein & Volker Hartwich & Beate Mayr & Jolanda Consiglio & James E Baumgardner & Klaus Markstaller & Reto Basciani & Andreas Vogt, 2013. "Multi Frequency Phase Fluorimetry (MFPF) for Oxygen Partial Pressure Measurement: Ex Vivo Validation by Polarographic Clark-Type Electrode," PLOS ONE, Public Library of Science, vol. 8(4), pages 1-8, April.
    18. Neves, Kleber & Amaral, Olavo Bohrer, 2019. "Addressing selective reporting of experiments – the case for predefined exclusion criteria," MetaArXiv a8gu5, Center for Open Science.
    19. Willie A Bidot & Aaron C Ericsson & Craig L Franklin, 2018. "Effects of water decontamination methods and bedding material on the gut microbiota," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-16, October.
    20. Emanuele Rinninella & Marco Cintoni & Pauline Raoul & Vincenzina Mora & Antonio Gasbarrini & Maria Cristina Mele, 2021. "Impact of Food Additive Titanium Dioxide on Gut Microbiota Composition, Microbiota-Associated Functions, and Gut Barrier: A Systematic Review of In Vivo Animal Studies," IJERPH, MDPI, vol. 18(4), pages 1-16, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0230798. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.