IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v17y2023i4s1751157723000810.html
   My bibliography  Save this article

The k-step h-index in citation networks at the paper, author, and institution levels

Author

Listed:
  • Yang, Alex Jie
  • Wu, Linwei
  • Zhang, Qi
  • Wang, Hao
  • Deng, Sanhong

Abstract

The evaluation of scientific impact plays a crucial role in assessing research contributions. In this study, we introduce the concept of the k-step h-index and investigate its applicability in citation networks at different levels, including papers, authors, and institutions. By incorporating higher generations of citation information, the k-step h-index provides a comprehensive and nuanced measure of scientific influence. It demonstrates exponential growth in k-step citations, capturing valuable information from the Hirsch core and tail. Through power law distribution analysis, we uncover the presence of highly influential entities coexisting with less influential ones, revealing the heterogeneity of impact within citation networks. To validate the effectiveness of the k-step h-index, we utilize a vast dataset from APS, conducting a thorough examination of its consistency and convergent validity. Our findings demonstrate strong correlations between the k-step h-index and conventional metrics, as well as alignment with measures of innovation. This confirms the reliability of the k-step h-index and its ability to capture innovative contributions. Notably, when compared to benchmarks, the k-step h-index outperforms in accurately ranking expert-selected items, including milestone papers, distinguished authors, and prestigious institutions. Higher values of the k-step h-index consistently exhibit superior performance, showcasing their predictive power in identifying prominent scientific entities. These findings hold significant implications for research evaluation, policy-making, and strategic planning, as they pave the way for a more holistic understanding of scholarly contributions.

Suggested Citation

  • Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).
    • Handle: RePEc:eee:infome:v:17:y:2023:i:4:s1751157723000810
    DOI: 10.1016/j.joi.2023.101456
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157723000810
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2023.101456?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mingyang Wang & Jiaqi Zhang & Shijia Jiao & Xiangrong Zhang & Na Zhu & Guangsheng Chen, 2020. "Important citation identification by exploiting the syntactic and contextual information of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2109-2129, December.
    2. Wang, Jingjing & Xu, Shuqi & Mariani, Manuel S. & Lü, Linyuan, 2021. "The local structure of citation networks uncovers expert-selected milestone papers," Journal of Informetrics, Elsevier, vol. 15(4).
    3. Shaibu Mohammed & Anthony Morgan & Emmanuel Nyantakyi, 2020. "On the influence of uncited publications on a researcher’s h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1791-1799, March.
    4. Zhao, Star X. & Rousseau, Ronald & Ye, Fred Y., 2011. "h-Degree as a basic measure in weighted networks," Journal of Informetrics, Elsevier, vol. 5(4), pages 668-677.
    5. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    6. Joshua Ettinger & Friederike E. L. Otto & E. Lisa F. Schipper, 2021. "Storytelling can be a powerful tool for science," Nature, Nature, vol. 589(7842), pages 352-352, January.
    7. Gangan Prathap, 2019. "Letter to the editor: Revisiting the h-index and the p-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1829-1833, December.
    8. Xin An & Xin Sun & Shuo Xu, 2022. "Important citations identification with semi-supervised classification model," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6533-6555, November.
    9. Zhai, Li & Yan, Xiangbin & Zhang, Guojing, 2018. "Bi-directional h-index: A new measure of node centrality in weighted and directed networks," Journal of Informetrics, Elsevier, vol. 12(1), pages 299-314.
    10. Michael Park & Erin Leahey & Russell J. Funk, 2023. "Papers and patents are becoming less disruptive over time," Nature, Nature, vol. 613(7942), pages 138-144, January.
    11. Fang Zhang & Shengli Wu, 2021. "Measuring academic entities’ impact by content-based citation analysis in a heterogeneous academic network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7197-7222, August.
    12. Mariani, Manuel Sebastian & Medo, Matúš & Zhang, Yi-Cheng, 2016. "Identification of milestone papers through time-balanced network centrality," Journal of Informetrics, Elsevier, vol. 10(4), pages 1207-1223.
    13. Lutz Bornmann & Loet Leydesdorff, 2018. "Count highly-cited papers instead of papers with h citations: use normalized citation counts and compare “like with like”!," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1119-1123, May.
    14. Yian Yin & Yang Wang & James A. Evans & Dashun Wang, 2019. "Quantifying the dynamics of failure across science, startups and security," Nature, Nature, vol. 575(7781), pages 190-194, November.
    15. Holly N. Wolcott & Matthew J. Fouch & Elizabeth R. Hsu & Leo G. DiJoseph & Catherine A. Bernaciak & James G. Corrigan & Duane E. Williams, 2016. "Modeling time-dependent and -independent indicators to facilitate identification of breakthrough research papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 807-817, May.
    16. Henry H. Bi, 2023. "Four problems of the h-index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2677-2691, May.
    17. Russell J. Funk & Jason Owen-Smith, 2017. "A Dynamic Network Measure of Technological Change," Management Science, INFORMS, vol. 63(3), pages 791-817, March.
    18. Jeff Alstott & Ed Bullmore & Dietmar Plenz, 2014. "powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-11, January.
    19. L. Egghe, 2010. "On the relation between Schubert’s h-index of a single paper and its total number of received citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 115-117, July.
    20. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    21. Bornmann, Lutz & Tekles, Alexander, 2021. "Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts," Journal of Informetrics, Elsevier, vol. 15(3).
    22. Brito, Ricardo & Navarro, Alonso Rodríguez, 2021. "The inconsistency of h-index: A mathematical analysis," Journal of Informetrics, Elsevier, vol. 15(1).
    23. Vladlen Koltun & David Hafner, 2021. "The h-index is no longer an effective correlate of scientific reputation," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-16, June.
    24. Linyuan Lü & Tao Zhou & Qian-Ming Zhang & H. Eugene Stanley, 2016. "The H-index of a network node and its relation to degree and coreness," Nature Communications, Nature, vol. 7(1), pages 1-7, April.
    25. Henry H. Bi, 2023. "Correction to: Four problems of the h-index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2693-2699, May.
    26. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    27. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h‐index," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    28. András Schubert, 2009. "Using the h-index for assessing single publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(3), pages 559-565, March.
    29. Zhai, Li & Yan, Xiangbin & Zhang, Guojing, 2019. "The bi-directional h-index and B-core decomposition in directed networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 531(C).
    30. Shalabh, 2021. "Statistical inference via data science," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 1155-1155, July.
    31. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2018. "How to evaluate rankings of academic entities using test data," Journal of Informetrics, Elsevier, vol. 12(3), pages 631-655.
    32. Xu, Shuqi & Mariani, Manuel Sebastian & Lü, Linyuan & Medo, Matúš, 2020. "Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data," Journal of Informetrics, Elsevier, vol. 14(1).
    33. Gerardo Iñiguez & Carlos Pineda & Carlos Gershenson & Albert-László Barabási, 2022. "Dynamics of ranking," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    34. Mariani, Manuel Sebastian & Medo, Matúš & Lafond, François, 2019. "Early identification of important patents: Design and validation of citation network metrics," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 644-654.
    35. Xiaojun Hu & Ronald Rousseau, 2017. "Nobel Prize winners 2016: Igniting or sparking foundational publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 1053-1063, February.
    36. Martin Rosvall & Alcides V. Esquivel & Andrea Lancichinetti & Jevin D. West & Renaud Lambiotte, 2014. "Memory in network flows and its effects on spreading dynamics and community detection," Nature Communications, Nature, vol. 5(1), pages 1-13, December.
    37. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    38. Fengli Xu & Lingfei Wu & James Evans, 2022. "Flat teams drive scientific innovation," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 119(23), pages 2200927119-, June.
    39. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    40. Shouhuai Xu & Moti Yung & Jingguo Wang, 2021. "Seeking Foundations for the Science of Cyber Security," Information Systems Frontiers, Springer, vol. 23(2), pages 263-267, April.
    41. Fontana, Magda & Iori, Martina & Montobbio, Fabio & Sinatra, Roberta, 2020. "New and atypical combinations: An assessment of novelty and interdisciplinarity," Research Policy, Elsevier, vol. 49(7).
    42. Wu, Lingfei & Kittur, Aniket & Youn, Hyejin & Milojević, Staša & Leahey, Erin & Fiore, Stephen M. & Ahn, Yong-Yeol, 2022. "Metrics and mechanisms: Measuring the unmeasurable in the science of science," Journal of Informetrics, Elsevier, vol. 16(2).
    43. Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    44. Ruan, Xuanmin & Lyu, Dongqing & Gong, Kaile & Cheng, Ying & Li, Jiang, 2021. "Rethinking the disruption index as a measure of scientific and technological advances," Technological Forecasting and Social Change, Elsevier, vol. 172(C).
    45. Yian Yin & Yuxiao Dong & Kuansan Wang & Dashun Wang & Benjamin F. Jones, 2022. "Public use and public funding of science," Nature Human Behaviour, Nature, vol. 6(10), pages 1344-1350, October.
    46. B Ian Hutchins & Xin Yuan & James M Anderson & George M Santangelo, 2016. "Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level," PLOS Biology, Public Library of Science, vol. 14(9), pages 1-25, September.
    47. Leo Egghe, 2008. "Modelling successive h-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(3), pages 377-387, December.
    48. Jean J. Wang & Sarah X. Shao & Fred Y. Ye, 2021. "Identifying 'seed' papers in sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6001-6011, July.
    49. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    50. Henry H. Bi, 2023. "Correction to: Four problems of the h‑index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2701-2701, May.
    51. András Schubert, 2007. "Successive h-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 70(1), pages 201-205, January.
    52. J. E. Hirsch, 2019. "hα: An index to quantify an individual’s scientific leadership," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 673-686, February.
    53. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang, Alex J., 2024. "Unveiling the impact and dual innovation of funded research," Journal of Informetrics, Elsevier, vol. 18(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Alex J., 2024. "Unveiling the impact and dual innovation of funded research," Journal of Informetrics, Elsevier, vol. 18(1).
    2. Wang, Jingjing & Xu, Shuqi & Mariani, Manuel S. & Lü, Linyuan, 2021. "The local structure of citation networks uncovers expert-selected milestone papers," Journal of Informetrics, Elsevier, vol. 15(4).
    3. Xu, Shuqi & Mariani, Manuel Sebastian & Lü, Linyuan & Medo, Matúš, 2020. "Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data," Journal of Informetrics, Elsevier, vol. 14(1).
    4. Zhang, Yang & Wang, Yang & Du, Haifeng & Havlin, Shlomo, 2024. "Delayed citation impact of interdisciplinary research," Journal of Informetrics, Elsevier, vol. 18(1).
    5. Tong, Tong & Wang, Wanru & Ye, Fred Y., 2024. "A complement to the novel disruption indicator based on knowledge entities," Journal of Informetrics, Elsevier, vol. 18(2).
    6. Ruijie Wang & Yuhao Zhou & An Zeng, 2023. "Evaluating scientists by citation and disruption of their representative works," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1689-1710, March.
    7. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    8. Yves Fassin, 2020. "The HF-rating as a universal complement to the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 965-990, November.
    9. Eitan Frachtenberg, 2022. "Multifactor Citation Analysis over Five Years: A Case Study of SIGMETRICS Papers," Publications, MDPI, vol. 10(4), pages 1-16, December.
    10. Wu, Lingfei & Kittur, Aniket & Youn, Hyejin & Milojević, Staša & Leahey, Erin & Fiore, Stephen M. & Ahn, Yong-Yeol, 2022. "Metrics and mechanisms: Measuring the unmeasurable in the science of science," Journal of Informetrics, Elsevier, vol. 16(2).
    11. Yue Wang & Ning Li & Bin Zhang & Qian Huang & Jian Wu & Yang Wang, 2023. "The effect of structural holes on producing novel and disruptive research in physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1801-1823, March.
    12. Pierre Pelletier & Kevin Wirtz, 2023. "Sails and Anchors: The Complementarity of Exploratory and Exploitative Scientists in Knowledge Creation," Papers 2312.10476, arXiv.org.
    13. Liyin Zhang & Yuchen Qian & Chao Ma & Jiang Li, 2023. "Continued collaboration shortens the transition period of scientists who move to another institution," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1765-1784, March.
    14. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2023. "Uncited papers in the structure of scientific communication," Journal of Informetrics, Elsevier, vol. 17(2).
    15. Leo Egghe & Ronald Rousseau, 2021. "The h-index formalism," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6137-6145, July.
    16. Loet Leydesdorff & Paul Wouters & Lutz Bornmann, 2016. "Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2129-2150, December.
    17. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    18. Malte Hückstädt, 2023. "Ten reasons why research collaborations succeed—a random forest approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1923-1950, March.
    19. Maziar Montazerian & Edgar Dutra Zanotto & Hellmut Eckert, 2019. "A new parameter for (normalized) evaluation of H-index: countries as a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1065-1078, March.
    20. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:17:y:2023:i:4:s1751157723000810. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.