IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0166733.html
   My bibliography  Save this article

The Devil Is in the Details: Incomplete Reporting in Preclinical Animal Research

Author

Listed:
  • Marc T Avey
  • David Moher
  • Katrina J Sullivan
  • Dean Fergusson
  • Gilly Griffin
  • Jeremy M Grimshaw
  • Brian Hutton
  • Manoj M Lalu
  • Malcolm Macleod
  • John Marshall
  • Shirley H J Mei
  • Michael Rudnicki
  • Duncan J Stewart
  • Alexis F Turgeon
  • Lauralyn McIntyre
  • Canadian Critical Care Translational Biology Group

Abstract

Incomplete reporting of study methods and results has become a focal point for failures in the reproducibility and translation of findings from preclinical research. Here we demonstrate that incomplete reporting of preclinical research is not limited to a few elements of research design, but rather is a broader problem that extends to the reporting of the methods and results. We evaluated 47 preclinical research studies from a systematic review of acute lung injury that use mesenchymal stem cells (MSCs) as a treatment. We operationalized the ARRIVE (Animal Research: Reporting of In Vivo Experiments) reporting guidelines for pre-clinical studies into 109 discrete reporting sub-items and extracted 5,123 data elements. Overall, studies reported less than half (47%) of all sub-items (median 51 items; range 37–64). Across all studies, the Methods Section reported less than half (45%) and the Results Section reported less than a third (29%). There was no association between journal impact factor and completeness of reporting, which suggests that incomplete reporting of preclinical research occurs across all journals regardless of their perceived prestige. Incomplete reporting of methods and results will impede attempts to replicate research findings and maximize the value of preclinical studies.

Suggested Citation

  • Marc T Avey & David Moher & Katrina J Sullivan & Dean Fergusson & Gilly Griffin & Jeremy M Grimshaw & Brian Hutton & Manoj M Lalu & Malcolm Macleod & John Marshall & Shirley H J Mei & Michael Rudnicki, 2016. "The Devil Is in the Details: Incomplete Reporting in Preclinical Animal Research," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-13, November.
  • Handle: RePEc:plo:pone00:0166733
    DOI: 10.1371/journal.pone.0166733
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0166733
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0166733&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0166733?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Carol Kilkenny & Nick Parsons & Ed Kadyszewski & Michael F W Festing & Innes C Cuthill & Derek Fry & Jane Hutton & Douglas G Altman, 2009. "Survey of the Quality of Experimental Design, Statistical Analysis and Reporting of Research Using Animals," PLOS ONE, Public Library of Science, vol. 4(11), pages 1-11, November.
    2. Jennifer A Hirst & Jeremy Howick & Jeffrey K Aronson & Nia Roberts & Rafael Perera & Constantinos Koshiaris & Carl Heneghan, 2014. "The Need for Randomization in Animal Trials: An Overview of Systematic Reviews," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    3. Lauralyn A McIntyre & David Moher & Dean A Fergusson & Katrina J Sullivan & Shirley H J Mei & Manoj Lalu & John Marshall & Malcolm Mcleod & Gilly Griffin & Jeremy Grimshaw & Alexis Turgeon & Marc T Av, 2016. "Efficacy of Mesenchymal Stromal Cell Therapy for Acute Lung Injury in Preclinical Animal Models: A Systematic Review," PLOS ONE, Public Library of Science, vol. 11(1), pages 1-16, January.
    4. Valerie C Henderson & Jonathan Kimmelman & Dean Fergusson & Jeremy M Grimshaw & Dan G Hackam, 2013. "Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments," PLOS Medicine, Public Library of Science, vol. 10(7), pages 1-14, July.
    5. David Baker & Katie Lidster & Ana Sottomayor & Sandra Amor, 2014. "Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies," PLOS Biology, Public Library of Science, vol. 12(1), pages 1-6, January.
    6. Monya Baker, 2016. "1,500 scientists lift the lid on reproducibility," Nature, Nature, vol. 533(7604), pages 452-454, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Malcolm R Macleod & Aaron Lawson McLean & Aikaterini Kyriakopoulou & Stylianos Serghiou & Arno de Wilde & Nicki Sherratt & Theo Hirst & Rachel Hemblade & Zsanett Bahor & Cristina Nunes-Fonseca & Aparn, 2015. "Risk of Bias in Reports of In Vivo Research: A Focus for Improvement," PLOS Biology, Public Library of Science, vol. 13(10), pages 1-12, October.
    2. Pandora Pound & Christine J Nicol, 2018. "Retrospective harm benefit analysis of pre-clinical animal research for six treatment interventions," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-26, March.
    3. Dean A Fergusson & Marc T Avey & Carly C Barron & Mathew Bocock & Kristen E Biefer & Sylvain Boet & Stephane L Bourque & Isidora Conic & Kai Chen & Yuan Yi Dong & Grace M Fox & Ronald B George & Neil , 2019. "Reporting preclinical anesthesia study (REPEAT): Evaluating the quality of reporting in the preclinical anesthesiology literature," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-15, May.
    4. Vivian Leung & Frédérik Rousseau-Blass & Guy Beauchamp & Daniel S J Pang, 2018. "ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesi," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-13, May.
    5. SeungHye Han & Tolani F Olonisakin & John P Pribis & Jill Zupetic & Joo Heung Yoon & Kyle M Holleran & Kwonho Jeong & Nader Shaikh & Doris M Rubio & Janet S Lee, 2017. "A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review," PLOS ONE, Public Library of Science, vol. 12(9), pages 1-14, September.
    6. Fala Cramond & Cadi Irvine & Jing Liao & David Howells & Emily Sena & Gillian Currie & Malcolm Macleod, 2016. "Protocol for a retrospective, controlled cohort study of the impact of a change in Nature journals’ editorial policy for life sciences research on the completeness of reporting study design and execut," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 315-328, July.
    7. Judith van Luijk & Brenda Bakker & Maroeska M Rovers & Merel Ritskes-Hoitinga & Rob B M de Vries & Marlies Leenaars, 2014. "Systematic Reviews of Animal Studies; Missing Link in Translational Research?," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-5, March.
    8. Dennis Bontempi & Leonard Nuernberg & Suraj Pai & Deepa Krishnaswamy & Vamsi Thiriveedhi & Ahmed Hosny & Raymond H. Mak & Keyvan Farahani & Ron Kikinis & Andrey Fedorov & Hugo J. W. L. Aerts, 2024. "End-to-end reproducible AI pipelines in radiology using the cloud," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    9. Chai, Daniel & Ali, Searat & Brosnan, Mark & Hasso, Tim, 2024. "Understanding researchers' perceptions and experiences in finance research replication studies: A pre-registered report," Pacific-Basin Finance Journal, Elsevier, vol. 86(C).
    10. Yali Liu & Xingxing Zhao & Yuefen Mai & Xinxin Li & Jin Wang & Lili Chen & Jing Mu & Gengxue Jin & Hongping Gou & Wanting Sun & Yuchen Feng, 2016. "Adherence to ARRIVE Guidelines in Chinese Journal Reports on Neoplasms in Animals," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-12, May.
    11. Kimberley E Wever & Carlijn R Hooijmans & Niels P Riksen & Thomas B Sterenborg & Emily S Sena & Merel Ritskes-Hoitinga & Michiel C Warlé, 2015. "Determinants of the Efficacy of Cardiac Ischemic Preconditioning: A Systematic Review and Meta-Analysis of Animal Studies," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-17, November.
    12. Natasha A Karp & Terry F Meehan & Hugh Morgan & Jeremy C Mason & Andrew Blake & Natalja Kurbatova & Damian Smedley & Julius Jacobsen & Richard F Mott & Vivek Iyer & Peter Matthews & David G Melvin & S, 2015. "Applying the ARRIVE Guidelines to an In Vivo Database," PLOS Biology, Public Library of Science, vol. 13(5), pages 1-11, May.
    13. Morgan Taschuk & Greg Wilson, 2017. "Ten simple rules for making research software more robust," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-10, April.
    14. Peter Harremoës, 2019. "Replication Papers," Publications, MDPI, vol. 7(3), pages 1-8, July.
    15. Konrad Neumann & Ulrike Grittner & Sophie K Piper & Andre Rex & Oscar Florez-Vargas & George Karystianis & Alice Schneider & Ian Wellwood & Bob Siegerink & John P A Ioannidis & Jonathan Kimmelman & Ul, 2017. "Increasing efficiency of preclinical research by group sequential designs," PLOS Biology, Public Library of Science, vol. 15(3), pages 1-9, March.
    16. Fernando Hoces de la Guardia & Sean Grant & Edward Miguel, 2021. "A framework for open policy analysis," Science and Public Policy, Oxford University Press, vol. 48(2), pages 154-163.
    17. Antonella Lanati & Marinella Marzano & Caterina Manzari & Bruno Fosso & Graziano Pesole & Francesca De Leo, 2019. "Management at the service of research: ReOmicS, a quality management system for omics sciences," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-13, December.
    18. Joel Ferguson & Rebecca Littman & Garret Christensen & Elizabeth Levy Paluck & Nicholas Swanson & Zenan Wang & Edward Miguel & David Birke & John-Henry Pezzuto, 2023. "Survey of open science practices and attitudes in the social sciences," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    19. Thomas F. Heston, 2024. "Redefining Significance: Robustness and Percent Fragility Indices in Biomedical Research," Stats, MDPI, vol. 7(2), pages 1-12, June.
    20. Erastus Karanja & Aditya Sharma & Ibrahim Salama, 2020. "What does MIS survey research reveal about diversity and representativeness in the MIS field? A content analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1583-1628, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0166733. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.