IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0200597.html
   My bibliography  Save this article

High-impact and transformative science (HITS) metrics: Definition, exemplification, and comparison

Author

Listed:
  • Joseph Staudt
  • Huifeng Yu
  • Robert P Light
  • Gerald Marschke
  • Katy Börner
  • Bruce A Weinberg

Abstract

Countries, research institutions, and scholars are interested in identifying and promoting high-impact and transformative scientific research. This paper presents a novel set of text- and citation-based metrics that can be used to identify high-impact and transformative works. The 11 metrics can be grouped into seven types: Radical-Generative, Radical-Destructive, Risky, Multidisciplinary, Wide Impact, Growing Impact, and Impact (overall). The metrics are exemplified, validated, and compared using a set of 10,778,696 MEDLINE articles matched to the Science Citation Index ExpandedTM. Articles are grouped into six 5-year periods (spanning 1983–2012) using publication year and into 6,159 fields constructed using comparable MeSH terms, with which each article is tagged. The analysis is conducted at the level of a field-period pair, of which 15,051 have articles and are used in this study. A factor analysis shows that transformativeness and impact are positively related (ρ = .402), but represent distinct phenomena. Looking at the subcomponents of transformativeness, there is no evidence that transformative work is adopted slowly or that the generation of important new concepts coincides with the obsolescence of existing concepts. We also find that the generation of important new concepts and highly cited work is more risky. Finally, supporting the validity of our metrics, we show that work that draws on a wider range of research fields is used more widely.

Suggested Citation

  • Joseph Staudt & Huifeng Yu & Robert P Light & Gerald Marschke & Katy Börner & Bruce A Weinberg, 2018. "High-impact and transformative science (HITS) metrics: Definition, exemplification, and comparison," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-23, July.
    • Handle: RePEc:plo:pone00:0200597
    DOI: 10.1371/journal.pone.0200597
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0200597
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0200597&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0200597?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hanning Guo & Scott Weingart & Katy Börner, 2011. "Mixed-indicators model for identifying emerging research areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 421-435, October.
    2. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    3. Ufuk Akcigit & Murat Celik & Daron Acemoglu, 2014. "Young, Restless and Creative: Openness to Disruption and Creative Innovations," 2014 Meeting Papers 377, Society for Economic Dynamics.
    4. Bettencourt, Luís M.A. & Kaiser, David I. & Kaur, Jasleen, 2009. "Scientific discovery and topological transitions in collaboration networks," Journal of Informetrics, Elsevier, vol. 3(3), pages 210-221.
    5. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    6. Katy Börner & Adam H. Simpson & Andreas Bueckle & Robert L. Goldstone, 2018. "Science map metaphors: a comparison of network versus hexmap-based visualizations," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 409-426, February.
    7. B Ian Hutchins & Xin Yuan & James M Anderson & George M Santangelo, 2016. "Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level," PLOS Biology, Public Library of Science, vol. 14(9), pages 1-25, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Joseph Staudt, 2020. "Mandating access: assessing the NIH’s public access policy," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 35(102), pages 269-304.
    2. Wei Cheng & Bruce A. Weinberg, 2021. "Marginalized and Overlooked? Minoritized Groups and the Adoption of New Scientific Ideas," NBER Working Papers 29179, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter van den Besselaar & Ulf Sandström, 2019. "Measuring researcher independence using bibliometric data: A proposal for a new performance indicator," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-20, March.
    2. Jay Bhattacharya & Mikko Packalen, 2020. "Stagnation and Scientific Incentives," NBER Working Papers 26752, National Bureau of Economic Research, Inc.
    3. Min, Chao & Bu, Yi & Sun, Jianjun, 2021. "Predicting scientific breakthroughs based on knowledge structure variations," Technological Forecasting and Social Change, Elsevier, vol. 164(C).
    4. Latefa Ali Dardas & Malik Sallam & Amanda Woodward & Nadia Sweis & Narjes Sweis & Faleh A. Sawair, 2023. "Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive," Publications, MDPI, vol. 11(1), pages 1-16, January.
    5. Rodríguez-Navarro, Alonso & Brito, Ricardo, 2024. "Rank analysis of most cited publications, a new approach for research assessments," Journal of Informetrics, Elsevier, vol. 18(2).
    6. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    7. Rotolo, Daniele & Hicks, Diana & Martin, Ben R., 2015. "What is an emerging technology?," Research Policy, Elsevier, vol. 44(10), pages 1827-1843.
    8. Gaetan de Rassenfosse & Kyle Higham & Orion Penner, 2022. "Scientific rewards for biomedical specialization are large and persistent," Working Papers 19, Chair of Science, Technology, and Innovation Policy.
    9. Jiang, Zhuoren & Lin, Tianqianjin & Huang, Cui, 2023. "Deep representation learning of scientific paper reveals its potential scholarly impact," Journal of Informetrics, Elsevier, vol. 17(1).
    10. Small, Henry & Boyack, Kevin W. & Klavans, Richard, 2014. "Identifying emerging topics in science and technology," Research Policy, Elsevier, vol. 43(8), pages 1450-1467.
    11. Winnink, J.J. & Tijssen, Robert J.W. & van Raan, A.F.J., 2019. "Searching for new breakthroughs in science: How effective are computerised detection algorithms?," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 673-686.
    12. Jacob Wood & Gohar Feroz Khan, 2015. "International trade negotiation analysis: network and semantic knowledge infrastructure," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(1), pages 537-556, October.
    13. Zhao, Star X. & Rousseau, Ronald & Ye, Fred Y., 2011. "h-Degree as a basic measure in weighted networks," Journal of Informetrics, Elsevier, vol. 5(4), pages 668-677.
    14. Kaur, Jasleen & Radicchi, Filippo & Menczer, Filippo, 2013. "Universality of scholarly impact metrics," Journal of Informetrics, Elsevier, vol. 7(4), pages 924-932.
    15. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.
    16. Gao, Qiang & Liang, Zhentao & Wang, Ping & Hou, Jingrui & Chen, Xiuxiu & Liu, Manman, 2021. "Potential index: Revealing the future impact of research topics based on current knowledge networks," Journal of Informetrics, Elsevier, vol. 15(3).
    17. Çağatay Bircan & Ralph De Haas, 2020. "The Limits of Lending? Banks and Technology Adoption across Russia," The Review of Financial Studies, Society for Financial Studies, vol. 33(2), pages 536-609.
    18. Corsini, Alberto & Pezzoni, Michele, 2023. "Does grant funding foster research impact? Evidence from France," Journal of Informetrics, Elsevier, vol. 17(4).
    19. Citron, Daniel T. & Way, Samuel F., 2018. "Network assembly of scientific communities of varying size and specificity," Journal of Informetrics, Elsevier, vol. 12(1), pages 181-190.
    20. Guoqiang Liang & Ying Lou & Haiyan Hou, 2022. "Revisiting the disruptive index: evidence from the Nobel Prize-winning articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5721-5730, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0200597. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.