IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/2006930.html
   My bibliography  Save this article

Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017

Author

Listed:
  • Joshua D Wallach
  • Kevin W Boyack
  • John P A Ioannidis

Abstract

Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000–2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks. We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]). Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]). Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed. Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.Author summary: Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical articles published from 2000–2014, the majority of studies did not share protocols and raw data or disclose funding or potential conflicts of interest. However, multiple recent efforts, which are attempting to address some of the existing concerns, may be resulting in genuine improvements in the transparency, openness, and reproducibility of the scientific literature. In this study, we investigate the reproducibility and transparency practices across the published biomedical literature from 2015–2017. We analyze reporting of public and/or private funding and conflicts of interests, sharing protocols and raw data, and independent replication and reproducibility checks. We also investigate what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. Our evaluation suggests that although there have been improvements over the last few years in some aspects of reproducibility and transparency (e.g., data sharing), opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.

Suggested Citation

  • Joshua D Wallach & Kevin W Boyack & John P A Ioannidis, 2018. "Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017," PLOS Biology, Public Library of Science, vol. 16(11), pages 1-20, November.
  • Handle: RePEc:plo:pbio00:2006930
    DOI: 10.1371/journal.pbio.2006930
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2006930
    Download Restriction: no

    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.2006930&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pbio.2006930?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Monya Baker, 2016. "1,500 scientists lift the lid on reproducibility," Nature, Nature, vol. 533(7604), pages 452-454, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dennis Bontempi & Leonard Nuernberg & Suraj Pai & Deepa Krishnaswamy & Vamsi Thiriveedhi & Ahmed Hosny & Raymond H. Mak & Keyvan Farahani & Ron Kikinis & Andrey Fedorov & Hugo J. W. L. Aerts, 2024. "End-to-end reproducible AI pipelines in radiology using the cloud," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    2. Chai, Daniel & Ali, Searat & Brosnan, Mark & Hasso, Tim, 2024. "Understanding researchers' perceptions and experiences in finance research replication studies: A pre-registered report," Pacific-Basin Finance Journal, Elsevier, vol. 86(C).
    3. Morgan Taschuk & Greg Wilson, 2017. "Ten simple rules for making research software more robust," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-10, April.
    4. Peter Harremoës, 2019. "Replication Papers," Publications, MDPI, vol. 7(3), pages 1-8, July.
    5. Fernando Hoces de la Guardia & Sean Grant & Edward Miguel, 2021. "A framework for open policy analysis," Science and Public Policy, Oxford University Press, vol. 48(2), pages 154-163.
    6. Antonella Lanati & Marinella Marzano & Caterina Manzari & Bruno Fosso & Graziano Pesole & Francesca De Leo, 2019. "Management at the service of research: ReOmicS, a quality management system for omics sciences," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-13, December.
    7. Joel Ferguson & Rebecca Littman & Garret Christensen & Elizabeth Levy Paluck & Nicholas Swanson & Zenan Wang & Edward Miguel & David Birke & John-Henry Pezzuto, 2023. "Survey of open science practices and attitudes in the social sciences," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    8. Thomas F. Heston, 2024. "Redefining Significance: Robustness and Percent Fragility Indices in Biomedical Research," Stats, MDPI, vol. 7(2), pages 1-12, June.
    9. Erastus Karanja & Aditya Sharma & Ibrahim Salama, 2020. "What does MIS survey research reveal about diversity and representativeness in the MIS field? A content analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1583-1628, March.
    10. Bor Luen Tang, 2023. "Some Insights into the Factors Influencing Continuous Citation of Retracted Scientific Papers," Publications, MDPI, vol. 11(4), pages 1-14, October.
    11. Sarah L R Stevens & Mateusz Kuzak & Carlos Martinez & Aurelia Moser & Petra Bleeker & Marc Galland, 2018. "Building a local community of practice in scientific programming for life scientists," PLOS Biology, Public Library of Science, vol. 16(11), pages 1-10, November.
    12. Rosenblatt, Lucas & Herman, Bernease & Holovenko, Anastasia & Lee, Wonkwon & Loftus, Joshua & McKinnie, Elizabeth & Rumezhak, Taras & Stadnik, Andrii & Howe, Bill & Stoyanovich, Julia, 2023. "Epistemic parity: reproducibility as an evaluation metric for differential privacy," LSE Research Online Documents on Economics 120493, London School of Economics and Political Science, LSE Library.
    13. Inga Patarčić & Jadranka Stojanovski, 2022. "Adoption of Transparency and Openness Promotion (TOP) Guidelines across Journals," Publications, MDPI, vol. 10(4), pages 1-10, November.
    14. Susanne Wieschowski & Svenja Biernot & Susanne Deutsch & Silke Glage & André Bleich & René Tolba & Daniel Strech, 2019. "Publication rates in animal research. Extent and characteristics of published and non-published animal studies followed up at two German university medical centres," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-8, November.
    15. Marc T Avey & David Moher & Katrina J Sullivan & Dean Fergusson & Gilly Griffin & Jeremy M Grimshaw & Brian Hutton & Manoj M Lalu & Malcolm Macleod & John Marshall & Shirley H J Mei & Michael Rudnicki, 2016. "The Devil Is in the Details: Incomplete Reporting in Preclinical Animal Research," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-13, November.
    16. Shinichi Nakagawa & Edward R. Ivimey-Cook & Matthew J. Grainger & Rose E. O’Dea & Samantha Burke & Szymon M. Drobniak & Elliot Gould & Erin L. Macartney & April Robin Martinig & Kyle Morrison & Matthi, 2023. "Method Reporting with Initials for Transparency (MeRIT) promotes more granularity and accountability for author contributions," Nature Communications, Nature, vol. 14(1), pages 1-5, December.
    17. Paul J. Ferraro & J. Dustin Tracy, 2022. "A reassessment of the potential for loss-framed incentive contracts to increase productivity: a meta-analysis and a real-effort experiment," Experimental Economics, Springer;Economic Science Association, vol. 25(5), pages 1441-1466, November.
    18. Micheletti, Tatiane & Wimmler, Marie-Christin & Berger, Uta & Grimm, Volker & McIntire, Eliot J., 2024. "Beyond guides, protocols and acronyms: Adoption of good modelling practices depends on challenging academia's status quo in ecology," Ecological Modelling, Elsevier, vol. 496(C).
    19. Ana Cecilia Quiroga Gutierrez & Daniel J. Lindegger & Ala Taji Heravi & Thomas Stojanov & Martin Sykora & Suzanne Elayan & Stephen J. Mooney & John A. Naslund & Marta Fadda & Oliver Gruebner, 2023. "Reproducibility and Scientific Integrity of Big Data Research in Urban Public Health and Digital Epidemiology: A Call to Action," IJERPH, MDPI, vol. 20(2), pages 1-15, January.
    20. Ahmed Al-Shafei & Hamidreza Zareipour & Yankai Cao, 2022. "High-Performance and Parallel Computing Techniques Review: Applications, Challenges and Potentials to Support Net-Zero Transition of Future Grids," Energies, MDPI, vol. 15(22), pages 1-58, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:2006930. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.