IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v122y2020i2d10.1007_s11192-019-03318-2.html
   My bibliography  Save this article

A new database of the references on international clinical practice guidelines: a facility for the evaluation of clinical research

Author

Listed:
  • Magnus Eriksson

    (Minso Solutions AB)

  • Annika Billhult

    (Minso Solutions AB
    University of Boras)

  • Tommy Billhult

    (Minso Solutions AB)

  • Elena Pallari

    (King’s College London
    University College London)

  • Grant Lewison

    (King’s College London
    Evaluametrics Ltd)

Abstract

Although there are now several bibliographic databases of research publications, such as Google Scholar, Pubmed, Scopus, and the Web of Science (WoS), and some also include counts of citations, there is at present no similarly comprehensive database of the rapidly growing number of clinical practice guidelines (CPGs), with their references, which sometimes number in the hundreds. CPGs have been shown to be useful for the evaluation of clinical (as opposed to basic) biomedical research, which often suffers from relatively low counts of citations in the serial literature. The objectives were to introduce a new citation database, clinical impact®, and demonstrate how it can be used to evaluate research impact of clinical research publications by exploring the characteristics of CPG citations of two sets of papers, as well as show temporal variation of clinical impact® and the WoS. The paper includes the methodology used to retain the data and also the rationale adopted to achieve data quality. The analysis showed that although CPGs tend preferentially to cite papers from their own country, this is not always the case. It also showed that cited papers tend to have a more clinical research level than uncited papers. An analysis of diachronous citations in both clinical impact® and the WoS showed that although the WoS citations showed a decreasing trend after a peak at 2–3 years after publication, this was less clear for CPG citations and a longer timescale would be needed to evaluate their impact on these documents.

Suggested Citation

  • Magnus Eriksson & Annika Billhult & Tommy Billhult & Elena Pallari & Grant Lewison, 2020. "A new database of the references on international clinical practice guidelines: a facility for the evaluation of clinical research," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1221-1235, February.
    • Handle: RePEc:spr:scient:v:122:y:2020:i:2:d:10.1007_s11192-019-03318-2
    DOI: 10.1007/s11192-019-03318-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03318-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-019-03318-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    2. Javier Ruiz-Castillo, 2013. "The role of statistics in establishing the similarity of citation distributions in a static and a dynamic context," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 173-181, July.
    3. Aristoklis D. Anastasiadis & Marcelo P. Albuquerque & Marcio P. Albuquerque & Diogo B. Mussi, 2010. "Tsallis q-exponential describes the distribution of scientific citations—a new characterization of the impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 205-218, April.
    4. ., 1998. "Scientific Explanation," Chapters, in: John B. Davis & D. W. Hands & Uskali Mäki (ed.), The Handbook of Economic Methodology, chapter 109, Edward Elgar Publishing.
    5. Lokman I. Meho & Kiduk Yang, 2007. "Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2105-2125, November.
    6. Victoria Bakare & Grant Lewison, 2017. "Country over-citation ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1199-1207, November.
    7. Anthony F. J. Van Raan, 2001. "Competition amongst scientists for publication status:Toward a model of scientific publication and citation distributions," Scientometrics, Springer;Akadémiai Kiadó, vol. 51(1), pages 347-357, April.
    8. Loet Leydesdorff & Félix Moya-Anegón & Wouter Nooy, 2016. "Aggregated journal–journal citation relations in scopus and web of science matched and compared in terms of networks, maps, and interactive overlays," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(9), pages 2194-2211, September.
    9. ., 1998. "Sociology of Scientific Knowledge, The," Chapters, in: John B. Davis & D. W. Hands & Uskali Mäki (ed.), The Handbook of Economic Methodology, chapter 118, Edward Elgar Publishing.
    10. Belén Álvarez-Bornstein & Fernanda Morillo & María Bordons, 2017. "Funding acknowledgments in the Web of Science: completeness and accuracy of collected data," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1793-1812, September.
    11. Grant Lewison & Richard Sullivan, 2015. "Conflicts of interest statements on biomedical papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2151-2159, March.
    12. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    13. Giovanni Abramo & Ciriaco Andrea D’Angelo & Anastasiia Soldatenkova, 2016. "The dispersion of the citation distribution of top scientists’ publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1711-1724, December.
    14. Grant Lewison & Guillermo Paraje, 2004. "The classification of biomedical journals by research level," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 145-157, June.
    15. ., 1998. "Methodology of Scientific Research Programmes," Chapters, in: John B. Davis & D. W. Hands & Uskali Mäki (ed.), The Handbook of Economic Methodology, chapter 70, Edward Elgar Publishing.
    16. Lokman I. Meho & Cassidy R. Sugimoto, 2009. "Assessing the scholarly impact of information studies: A tale of two citation databases—Scopus and Web of Science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(12), pages 2499-2508, December.
    17. Jonathan Grant, 1999. "Evaluating the outcomes of biomedical research on healthcare," Research Evaluation, Oxford University Press, vol. 8(1), pages 33-38, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christopher Traylor & Christoph Herrmann-Lingen, 2023. "Does the journal impact factor reflect the impact of German medical guideline contributions?," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1951-1962, March.
    2. Xin Li & Xuli Tang & Wei Lu, 2024. "How biomedical papers accumulated their clinical citations: a large-scale retrospective analysis based on PubMed," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3315-3339, June.
    3. Li, Xin & Tang, Xuli & Cheng, Qikai, 2022. "Predicting the clinical citation count of biomedical papers using multilayer perceptron neural network," Journal of Informetrics, Elsevier, vol. 16(4).
    4. Dongyu Zang & Chunli Liu, 2023. "Exploring the clinical translation intensity of papers published by the world’s top scientists in basic medicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2371-2416, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frederique Bordignon, 2020. "Self-correction of science: a comparative study of negative citations and post-publication peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1225-1239, August.
    2. Andrea Fronzetti Colladon & Ciriaco Andrea D’Angelo & Peter A. Gloor, 2020. "Predicting the future success of scientific publications through social network and semantic analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 357-377, July.
    3. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    4. Lakshmi Balachandran Nair & Michael Gibbert, 2016. "What makes a ‘good’ title and (how) does it matter for citations? A review and general model of article title attributes in management science," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1331-1359, June.
    5. Parul Khurana & Kiran Sharma, 2022. "Impact of h-index on author’s rankings: an improvement to the h-index for lower-ranked authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4483-4498, August.
    6. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    7. Amir Karami & Melek Yildiz Spinel & C. Nicole White & Kayla Ford & Suzanne Swan, 2021. "A Systematic Literature Review of Sexual Harassment Studies with Text Mining," Sustainability, MDPI, vol. 13(12), pages 1-24, June.
    8. Gerhard Reichmann & Christian Schlögl, 2022. "On the possibilities of presenting the research performance of an institute over a long period of time: the case of the Institute of Information Science at the University of Graz in Austria," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3193-3223, June.
    9. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    10. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.
    11. Michael Gusenbauer, 2022. "Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2683-2745, May.
    12. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    13. Danan Gu & Qiushi Feng, 2018. "Psychological Resilience of Chinese Centenarians and Its Associations With Survival and Health: A Fuzzy Set Analysis," The Journals of Gerontology: Series B, The Gerontological Society of America, vol. 73(5), pages 880-889.
    14. Junwen Zhu & Weishu Liu, 2020. "A tale of two databases: the use of Web of Science and Scopus in academic papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 321-335, April.
    15. Michael Gusenbauer, 2019. "Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 177-214, January.
    16. Fernanda Morillo & Belén Álvarez-Bornstein, 2018. "How to automatically identify major research sponsors selecting keywords from the WoS Funding Agency field," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1755-1770, December.
    17. Sultana, Atia & Lewison, Grant & Pallari, Elena, 2019. "The evaluation of mental disorders research reported in British and Irish newspapers between 2002 and 2013, and a comparison with the relative disease burdens and with research outputs in the two coun," Health Policy, Elsevier, vol. 123(4), pages 419-426.
    18. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    19. Dongyu Zang & Chunli Liu, 2023. "Exploring the clinical translation intensity of papers published by the world’s top scientists in basic medicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2371-2416, April.
    20. Vítor Vasata Macchi Silva & José Luis Duarte Ribeiro & Gonzalo Rubén Alvarez & Sonia Elisa Caregnato, 2019. "Competence-Based Management Research in the Web of Science and Scopus Databases: Scientific Production, Collaboration, and Impact," Publications, MDPI, vol. 7(4), pages 1-21, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:122:y:2020:i:2:d:10.1007_s11192-019-03318-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.