IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v111y2017i3d10.1007_s11192-017-2374-2.html
   My bibliography  Save this article

Letter to the Editor: About the quality and impact of scientific articles

Author

Listed:
  • Martin Ricker

    (Universidad Nacional Autónoma de México (UNAM))

Abstract

It is argued that counting the total number of times a scientific article is cited by others, does neither result in a proxy for its cognitive impact nor for its quality. One would have to distinguish at least substitutable and fundamental references. A supposed correlation between peer review assessments and citation counts is conceptually problematic, because peer review includes objective as well as subjective considerations (convictions). With refined methods, however, a differential citation analysis might be able in the future to answer if a given article did or did not have positive cognitive impact on subsequent research.

Suggested Citation

  • Martin Ricker, 2017. "Letter to the Editor: About the quality and impact of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1851-1855, June.
    • Handle: RePEc:spr:scient:v:111:y:2017:i:3:d:10.1007_s11192-017-2374-2
    DOI: 10.1007/s11192-017-2374-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-017-2374-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-017-2374-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ding, Ying & Liu, Xiaozhong & Guo, Chun & Cronin, Blaise, 2013. "The distribution of references across texts: Some implications for citation analysis," Journal of Informetrics, Elsevier, vol. 7(3), pages 583-592.
    2. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom-cited influences," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    3. Milojević, Staša, 2015. "Quantifying the cognitive extent of science," Journal of Informetrics, Elsevier, vol. 9(4), pages 962-973.
    4. Lin Zhang & Bart Thijs & Wolfgang Glänzel, 2013. "What does scientometrics share with other “metrics” sciences?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(7), pages 1515-1518, July.
    5. Lin Zhang & Bart Thijs & Wolfgang Glänzel, 2013. "What does scientometrics share with other “metrics” sciences?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(7), pages 1515-1518, July.
    6. M. V. Simkin & V. P. Roychowdhury, 2005. "Stochastic modeling of citation slips," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(3), pages 367-384, March.
    7. Brian M. Belcher & Katherine E. Rasmussen & Matthew R. Kemshaw & Deborah A. Zornes, 2016. "Defining and assessing research quality in a transdisciplinary context," Research Evaluation, Oxford University Press, vol. 25(1), pages 1-17.
    8. Gagolewski, Marek, 2013. "Scientific impact assessment cannot be fair," Journal of Informetrics, Elsevier, vol. 7(4), pages 792-802.
    9. Marc Bertin & Iana Atanassova & Cassidy R. Sugimoto & Vincent Lariviere, 2016. "The linguistic patterns and rhetorical structure of citation context: an approach using n-grams," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1417-1434, December.
    10. María‐del‐Mar Camacho‐Miñano & Manuel Núñez‐Nickel, 2009. "The multilayered nature of reference selection," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(4), pages 754-777, April.
    11. Martin Ricker, 2015. "A numerical algorithm with preference statements to evaluate the performance of scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 191-212, April.
    12. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom‐cited influences," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    13. Waltman, Ludo & van Eck, Nees Jan & Wouters, Paul, 2013. "Counting publications and citations: Is more always better?," Journal of Informetrics, Elsevier, vol. 7(3), pages 635-641.
    14. Barbara Good & Niki Vermeulen & Brigitte Tiefenthaler & Erik Arnold, 2015. "Counting quality? The Czech performance-based research funding system," Research Evaluation, Oxford University Press, vol. 24(2), pages 91-105.
    15. Marcelo Alves Ramos & Joabe Gomes Melo & Ulysses Paulino Albuquerque, 2012. "Citation behavior in popular scientific papers: what is behind obscure citations? The case of ethnobotany," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 711-719, September.
    16. Lutz Bornmann & Robin Haunschild, 2017. "Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 937-943, February.
    17. Dag W Aksnes & Randi Elisabeth Taxt, 2004. "Peer reviews and bibliometric indicators: a comparative study at a Norwegian university," Research Evaluation, Oxford University Press, vol. 13(1), pages 33-41, April.
    18. Kaur, Jasleen & Ferrara, Emilio & Menczer, Filippo & Flammini, Alessandro & Radicchi, Filippo, 2015. "Quality versus quantity in scientific impact," Journal of Informetrics, Elsevier, vol. 9(4), pages 800-808.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Drahomira Herrmannova & Robert M. Patton & Petr Knoth & Christopher G. Stahl, 2018. "Do citations and readership identify seminal publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 239-262, April.
    2. de Carvalho, Gustavo Dambiski Gomes & Sokulski, Carla Cristiane & da Silva, Wesley Vieira & de Carvalho, Hélio Gomes & de Moura, Rafael Vignoli & de Francisco, Antonio Carlos & da Veiga, Claudimar Per, 2020. "Bibliometrics and systematic reviews: A comparison between the Proknow-C and the Methodi Ordinatio," Journal of Informetrics, Elsevier, vol. 14(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ehsan Mohammadi & Mike Thelwall, 2013. "Assessing non-standard article impact using F1000 labels," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 383-395, November.
    2. Horbach, Serge & Aagaard, Kaare & Schneider, Jesper W., 2021. "Meta-Research: How problematic citing practices distort science," MetaArXiv aqyhg, Center for Open Science.
    3. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    4. Les Oxley, 2016. "Elites and Secret Handshakes Versus Metrics and Rule-Based Acclamation: A Comment on "Measuring the Unmeasurable"," Econometric Reviews, Taylor & Francis Journals, vol. 35(1), pages 44-49, January.
    5. Iman Tahamtan & Lutz Bornmann, 2019. "What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1635-1684, December.
    6. Sten F Odenwald, 2020. "A citation study of earth science projects in citizen science," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-26, July.
    7. Hu, Zewen & Wu, Yishan, 2014. "Regularity in the time-dependent distribution of the percentage of never-cited papers: An empirical pilot study based on the six journals," Journal of Informetrics, Elsevier, vol. 8(1), pages 136-146.
    8. Lawrence Smolinsky & Aaron Lercher, 2012. "Citation rates in mathematics: a study of variation by subdiscipline," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 911-924, June.
    9. Tobias Opthof & Loet Leydesdorff, 2011. "A comment to the paper by Waltman et al., Scientometrics, 87, 467–481, 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 1011-1016, September.
    10. Lutz Bornmann & Robin Haunschild, 2017. "Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 937-943, February.
    11. Jianhua Hou & Jiantao Ye, 2020. "Are uncited papers necessarily all nonimpact papers? A quantitative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1631-1662, August.
    12. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    13. Mike Thelwall, 2019. "The influence of highly cited papers on field normalised indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 519-537, February.
    14. Michael Calver & Kate Bryant & Grant Wardell-Johnson, 2018. "Quantifying the internationality and multidisciplinarity of authors and journals using ecological statistics," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 731-748, May.
    15. Isabel Bernal, 2013. "Open Access and the Changing Landscape of Research Impact Indicators: New Roles for Repositories," Publications, MDPI, vol. 1(2), pages 1-22, July.
    16. Bornmann, Lutz & Schier, Hermann & Marx, Werner & Daniel, Hans-Dieter, 2012. "What factors determine citation counts of publications in chemistry besides their quality?," Journal of Informetrics, Elsevier, vol. 6(1), pages 11-18.
    17. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    18. Zhi Li & Qinke Peng & Che Liu, 2016. "Two citation-based indicators to measure latent referential value of papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1299-1313, September.
    19. Christin Katharina Kreutz & Premtim Sahitaj & Ralf Schenkel, 2020. "Evaluating semantometrics from computer science publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2915-2954, December.
    20. Jiang, Zhuoren & Lin, Tianqianjin & Huang, Cui, 2023. "Deep representation learning of scientific paper reveals its potential scholarly impact," Journal of Informetrics, Elsevier, vol. 17(1).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:111:y:2017:i:3:d:10.1007_s11192-017-2374-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.