IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0092590.html
   My bibliography  Save this article

Measuring the Value of Research Data: A Citation Analysis of Oceanographic Data Sets

Author

Listed:
  • Christopher W Belter

Abstract

Evaluation of scientific research is becoming increasingly reliant on publication-based bibliometric indicators, which may result in the devaluation of other scientific activities - such as data curation – that do not necessarily result in the production of scientific publications. This issue may undermine the movement to openly share and cite data sets in scientific publications because researchers are unlikely to devote the effort necessary to curate their research data if they are unlikely to receive credit for doing so. This analysis attempts to demonstrate the bibliometric impact of properly curated and openly accessible data sets by attempting to generate citation counts for three data sets archived at the National Oceanographic Data Center. My findings suggest that all three data sets are highly cited, with estimated citation counts in most cases higher than 99% of all the journal articles published in Oceanography during the same years. I also find that methods of citing and referring to these data sets in scientific publications are highly inconsistent, despite the fact that a formal citation format is suggested for each data set. These findings have important implications for developing a data citation format, encouraging researchers to properly curate their research data, and evaluating the bibliometric impact of individuals and institutions.

Suggested Citation

  • Christopher W Belter, 2014. "Measuring the Value of Research Data: A Citation Analysis of Oceanographic Data Sets," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-9, March.
  • Handle: RePEc:plo:pone00:0092590
    DOI: 10.1371/journal.pone.0092590
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0092590
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0092590&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0092590?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ludo Waltman & Clara Calero‐Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan van Eck & Thed N. van Leeuwen & Anthony F.J. van Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    2. Christine L. Borgman, 2012. "The conundrum of sharing research data," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(6), pages 1059-1078, June.
    3. Liz Allen & Ceri Jones & Kevin Dolby & David Lynn & Mark Walport, 2009. "Looking for Landmarks: The Role of Expert Review and Bibliometric Analysis in Evaluating Scientific Publication Outputs," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-8, June.
    4. Kousha, Kayvan & Thelwall, Mike & Rezaie, Somayeh, 2010. "Using the Web for research evaluation: The Integrated Online Impact indicator," Journal of Informetrics, Elsevier, vol. 4(1), pages 124-135.
    5. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    6. Hendrik P. van Dalen & Kène Henkens, 2012. "Intended and unintended consequences of a publish‐or‐perish culture: A worldwide survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(7), pages 1282-1293, July.
    7. Henk F Moed, 2007. "The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review," Science and Public Policy, Oxford University Press, vol. 34(8), pages 575-583, October.
    8. Carol Tenopir & Suzie Allard & Kimberly Douglass & Arsev Umur Aydinoglu & Lei Wu & Eleanor Read & Maribeth Manoff & Mike Frame, 2011. "Data Sharing by Scientists: Practices and Perceptions," PLOS ONE, Public Library of Science, vol. 6(6), pages 1-21, June.
    9. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    10. Young-Ho Eom & Santo Fortunato, 2011. "Characterizing and Modeling Citation Dynamics," PLOS ONE, Public Library of Science, vol. 6(9), pages 1-7, September.
    11. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    12. Peter Guttorp & Stephan R. Sain & Christopher K. Wikle & Richard E. Chandler & Peter Thorne & Jay Lawrimore & Kate Willett, 2012. "Building trust in climate science: data products for the 21st century," Environmetrics, John Wiley & Sons, Ltd., vol. 23(5), pages 373-381, August.
    13. Heather A. Piwowar & Todd J. Vision & Michael C. Whitlock, 2011. "Data archiving is a good investment," Nature, Nature, vol. 473(7347), pages 285-285, May.
    14. Rodrigo Costas & Thed N. Leeuwen & Anthony F. J. Raan, 2011. "The “Mendel syndrome” in science: durability of scientific literature and its effects on bibliometric analysis of individual scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 177-205, October.
    15. Christine L. Borgman, 2012. "The conundrum of sharing research data," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(6), pages 1059-1078, June.
    16. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "National research assessment exercises: a comparison of peer review and bibliometrics rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 929-941, December.
    17. Kostoff, Ronald N. & Geisler, Elie, 2007. "The unintended consequences of metrics in technology evaluation," Journal of Informetrics, Elsevier, vol. 1(2), pages 103-114.
    18. Heather A Piwowar & Roger S Day & Douglas B Fridsma, 2007. "Sharing Detailed Research Data Is Associated with Increased Citation Rate," PLOS ONE, Public Library of Science, vol. 2(3), pages 1-5, March.
    19. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    2. Isabella Peters & Peter Kraker & Elisabeth Lex & Christian Gumpenberger & Juan Gorraiz, 2016. "Research data explored: an extended analysis of citations and altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 723-744, May.
    3. Nicola Milia & Alessandra Congiu & Paolo Anagnostou & Francesco Montinaro & Marco Capocasa & Emanuele Sanna & Giovanni Destro Bisol, 2012. "Mine, Yours, Ours? Sharing Data on Human Genetic Variation," PLOS ONE, Public Library of Science, vol. 7(6), pages 1-8, June.
    4. Benedikt Fecher & Sascha Friesike & Marcel Hebing, 2014. "What Drives Academic Data Sharing?," SOEPpapers on Multidisciplinary Panel Data Research 655, DIW Berlin, The German Socio-Economic Panel (SOEP).
    5. Keren Weinshall & Lee Epstein, 2020. "Developing High‐Quality Data Infrastructure for Legal Analytics: Introducing the Israeli Supreme Court Database," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 17(2), pages 416-434, June.
    6. Jesper W. Schneider & Thed Leeuwen & Martijn Visser & Kaare Aagaard, 2019. "Examining national citation impact by comparing developments in a fixed and a dynamic journal set," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 973-985, May.
    7. Liwei Zhang & Liang Ma, 2021. "Does open data boost journal impact: evidence from Chinese economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3393-3419, April.
    8. Shibayama, Sotaro & Lawson, Cornelia, 2021. "The use of rewards in the sharing of research resources," Research Policy, Elsevier, vol. 50(7).
    9. Jianhua Hou & Xiucai Yang & Chaomei Chen, 2018. "Emerging trends and new developments in information science: a document co-citation analysis (2009–2016)," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 869-892, May.
    10. Liwei Zhang & Liang Ma, 2023. "Is open science a double-edged sword?: data sharing and the changing citation pattern of Chinese economics articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2803-2818, May.
    11. Stefan Reichmann & Thomas Klebel & Ilire Hasani‐Mavriqi & Tony Ross‐Hellauer, 2021. "Between administration and research: Understanding data management practices in an institutional context," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(11), pages 1415-1431, November.
    12. Nicolas Robinson-Garcia & Benjamín Vargas-Quesada & Daniel Torres-Salinas & Zaida Chinchilla-Rodríguez & Juan Gorraiz, 2024. "Errors of measurement in scientometrics: classification schemes and document types in citation and publication rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6455-6475, October.
    13. Benedikt Fecher & Sascha Friesike & Marcel Hebing, 2015. "What Drives Academic Data Sharing?," PLOS ONE, Public Library of Science, vol. 10(2), pages 1-25, February.
    14. Benedikt Fecher & Sascha Friesike & Marcel Hebing, 2014. "What Drives Academic Data Sharing?," RatSWD Working Papers 236, German Data Forum (RatSWD).
    15. Álvaro Cabezas-Clavijo & Nicolás Robinson-García & Manuel Escabias & Evaristo Jiménez-Contreras, 2013. "Reviewers’ Ratings and Bibliometric Indicators: Hand in Hand When Assessing Over Research Proposals?," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-1, June.
    16. Robinson-Garcia, Nicolas & Mongeon, Philippe & Jeng, Wei & Costas, Rodrigo, 2017. "DataCite as a novel bibliometric source: Coverage, strengths and limitations," Journal of Informetrics, Elsevier, vol. 11(3), pages 841-854.
    17. Hou, Jianhua & Wang, Yuanyuan & Zhang, Yang & Wang, Dongyi, 2022. "How do scholars and non-scholars participate in dataset dissemination on Twitter," Journal of Informetrics, Elsevier, vol. 16(1).
    18. Mike Thelwall & Kayvan Kousha & Emma Stuart & Meiko Makita & Mahshid Abdoli & Paul Wilson & Jonathan Levitt, 2023. "In which fields are citations indicators of research quality?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(8), pages 941-953, August.
    19. Hoekman, Jarno & Rake, Bastian, 2024. "Geography of authorship: How geography shapes authorship attribution in big team science," Research Policy, Elsevier, vol. 53(2).
    20. Marco Cavallaro & Benedetto Lepori, 2021. "Institutional barriers to participation in EU framework programs: contrasting the Swiss and UK cases," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1311-1328, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0092590. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.