IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0122239.html
   My bibliography  Save this article

Measuring the Outcome of Biomedical Research: A Systematic Literature Review

Author

Listed:
  • Frédérique Thonon
  • Rym Boulkedid
  • Tristan Delory
  • Sophie Rousseau
  • Mahasti Saghatchian
  • Wim van Harten
  • Claire O’Neill
  • Corinne Alberti

Abstract

Background: There is an increasing need to evaluate the production and impact of medical research produced by institutions. Many indicators exist, yet we do not have enough information about their relevance. The objective of this systematic review was (1) to identify all the indicators that could be used to measure the output and outcome of medical research carried out in institutions and (2) enlist their methodology, use, positive and negative points. Methodology: We have searched 3 databases (Pubmed, Scopus, Web of Science) using the following keywords: [Research outcome* OR research output* OR bibliometric* OR scientometric* OR scientific production] AND [indicator* OR index* OR evaluation OR metrics]. We included articles presenting, discussing or evaluating indicators measuring the scientific production of an institution. The search was conducted by two independent authors using a standardised data extraction form. For each indicator we extracted its definition, calculation, its rationale and its positive and negative points. In order to reduce bias, data extraction and analysis was performed by two independent authors. Findings: We included 76 articles. A total of 57 indicators were identified. We have classified those indicators into 6 categories: 9 indicators of research activity, 24 indicators of scientific production and impact, 5 indicators of collaboration, 7 indicators of industrial production, 4 indicators of dissemination, 8 indicators of health service impact. The most widely discussed and described is the h-index with 31 articles discussing it. Discussion: The majority of indicators found are bibliometric indicators of scientific production and impact. Several indicators have been developed to improve the h-index. This indicator has also inspired the creation of two indicators to measure industrial production and collaboration. Several articles propose indicators measuring research impact without detailing a methodology for calculating them. Many bibliometric indicators identified have been created but have not been used or further discussed.

Suggested Citation

  • Frédérique Thonon & Rym Boulkedid & Tristan Delory & Sophie Rousseau & Mahasti Saghatchian & Wim van Harten & Claire O’Neill & Corinne Alberti, 2015. "Measuring the Outcome of Biomedical Research: A Systematic Literature Review," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    • Handle: RePEc:plo:pone00:0122239
    DOI: 10.1371/journal.pone.0122239
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0122239
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0122239&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0122239?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Grant Lewison, 2002. "From biomedical research to health improvement," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(2), pages 179-192, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    2. García-Romero, Antonio & Escribano, Álvaro & Tribó, Josep A., 2017. "The impact of health research on length of stay in Spanish public hospitals," Research Policy, Elsevier, vol. 46(3), pages 591-604.
    3. M. Luisa Lascurain-Sánchez & Carlos García-Zorita & Carmen Martín-Moreno & Carlos Suárez-Balseiro & Elías Sanz-Casado, 2008. "Impact of health science research on the Spanish health system, based on bibliometric and healthcare indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(1), pages 131-146, October.
    4. Jason Priem & Paul Groth & Dario Taraborelli, 2012. "The Altmetrics Collection," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-2, November.
    5. Antonio García Romero & José Navarrete Cortés & Cristina Escudero & Juan Antonio Fernández López & Juan Antonio Chaichío Moreno, 2009. "Measuring the influence of clinical trials citations on several bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 747-760, September.
    6. Grant Lewison & Thomas Turnbull, 2010. "News in brief and features in New Scientist magazine and the biomedical research papers that they cite, August 2008 to July 2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 345-359, October.
    7. Luis M. Plaza & Armando Albert, 2008. "Scientific literature cited in USPTO patent documents as indicators for the evaluation and analysis of Spanish scientific research in biomedical disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(3), pages 429-438, September.

    More about this item

    Keywords

    [research outcome* or research output* or bibliometric* or scientometric* or scientific production] and [indicator* or index* or evaluation or metrics]. we included articles presenting; discussing or evaluating indicators measuring the scientific production of an institution. the search was conducted by two independent authors using a standardised data extraction form. for each indicator we extracted its definition; calculation; its rationale and its positive and negative points. in order to reduce bias; data extraction and analysis was performed by two independent authors. findings: we included 76 articles. a total of 57 indicators were identified. we have classified those indicators into 6 categories: 9 indicators of research activity; 24 indicators of scientific production and impact; 5 indicators of collaboration; 7 indicators of industrial production; 4 indicators of dissemination; 8 indicators of health service impact. the most widely discussed and described is the h-index with 31 articles discussing it. discussion: the majority of indicators found are bibliometric indicators of scientific production and impact. several indicators have been developed to improve the h-index. this indicator has also inspired the creation of two indicators to measure industrial production and collaboration. several articles propose indicators measuring research impact without detailing a methodology for calculating them. many bibliometric indicators identified have been created but have not been used or further discussed.;
    All these keywords.

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0122239. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.