IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v17y2023i1s1751157723000044.html
   My bibliography  Save this article

Article’s scientific prestige: Measuring the impact of individual articles in the web of science

Author

Listed:
  • Chen, Ying
  • Koch, Thorsten
  • Zakiyeva, Nazgul
  • Liu, Kailiang
  • Xu, Zhitong
  • Chen, Chun-houh
  • Nakano, Junji
  • Honda, Keisuke

Abstract

We performed a citation analysis on the Web of Science publications consisting of more than 63 million articles and over a billion citations on 254 subjects from 1981 to 2020. We proposed the Article’s Scientific Prestige (ASP) metric and compared this metric to number of citations (#Cit) and journal grade in measuring the scientific impact of individual articles in the large-scale hierarchical and multi-disciplined citation network. In contrast to #Cit, ASP, that is computed based on the eigenvector centrality, considers both direct and indirect citations, and provides steady-state evaluation cross different disciplines. We found that ASP and #Cit are not aligned for most articles, with a growing mismatch amongst the less cited articles. While both metrics are reliable for evaluating the prestige of articles such as Nobel Prize winning articles, ASP tends to provide more persuasive rankings than #Cit when the articles are not highly cited. The journal grade, that is eventually determined by a few highly cited articles, is unable to properly reflect the scientific impact of individual articles. The number of references and coauthors are less relevant to scientific impact, but subjects do make a difference.

Suggested Citation

  • Chen, Ying & Koch, Thorsten & Zakiyeva, Nazgul & Liu, Kailiang & Xu, Zhitong & Chen, Chun-houh & Nakano, Junji & Honda, Keisuke, 2023. "Article’s scientific prestige: Measuring the impact of individual articles in the web of science," Journal of Informetrics, Elsevier, vol. 17(1).
    • Handle: RePEc:eee:infome:v:17:y:2023:i:1:s1751157723000044
    DOI: 10.1016/j.joi.2023.101379
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157723000044
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2023.101379?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhao, Qihang & Feng, Xiaodong, 2022. "Utilizing citation network structure to predict paper citation counts: A Deep learning approach," Journal of Informetrics, Elsevier, vol. 16(1).
    2. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    3. Ying Ding, 2011. "Applying weighted PageRank to author citation networks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 236-245, February.
    4. Wu, Han-Ming & Tien, Yin-Jing & Chen, Chun-houh, 2010. "GAP: A graphical environment for matrix visualization and cluster analysis," Computational Statistics & Data Analysis, Elsevier, vol. 54(3), pages 767-778, March.
    5. Johan S. G. Chu & James A. Evans, 2021. "Slowed canonical progress in large fields of science," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(41), pages 2021636118-, October.
    6. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    7. González-Pereira, Borja & Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2010. "A new approach to the metric of journals’ scientific prestige: The SJR indicator," Journal of Informetrics, Elsevier, vol. 4(3), pages 379-391.
    8. Yubing Nie & Yifan Zhu & Qika Lin & Sifan Zhang & Pengfei Shi & Zhendong Niu, 2019. "Academic rising star prediction via scholar’s evaluation model and machine learning techniques," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 461-476, August.
    9. Chen, P. & Xie, H. & Maslov, S. & Redner, S., 2007. "Finding scientific gems with Google’s PageRank algorithm," Journal of Informetrics, Elsevier, vol. 1(1), pages 8-15.
    10. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    11. Ying Ding, 2011. "Applying weighted PageRank to author citation networks," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 236-245, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dejian Yu & Wanru Wang & Shuai Zhang & Wenyu Zhang & Rongyu Liu, 2017. "A multiple-link, mutually reinforced journal-ranking model to measure the prestige of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 521-542, April.
    2. Fiala, Dalibor, 2012. "Time-aware PageRank for bibliographic networks," Journal of Informetrics, Elsevier, vol. 6(3), pages 370-388.
    3. Yanan Wang & An Zeng & Ying Fan & Zengru Di, 2019. "Ranking scientific publications considering the aging characteristics of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 155-166, July.
    4. Nykl, Michal & Campr, Michal & Ježek, Karel, 2015. "Author ranking based on personalized PageRank," Journal of Informetrics, Elsevier, vol. 9(4), pages 777-799.
    5. Fiala, Dalibor & Šubelj, Lovro & Žitnik, Slavko & Bajec, Marko, 2015. "Do PageRank-based author rankings outperform simple citation counts?," Journal of Informetrics, Elsevier, vol. 9(2), pages 334-348.
    6. Dinesh Pradhan & Partha Sarathi Paul & Umesh Maheswari & Subrata Nandi & Tanmoy Chakraborty, 2017. "$$C^3$$ C 3 -index: a PageRank based multi-faceted metric for authors’ performance measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 253-273, January.
    7. Jianlin Zhou & An Zeng & Ying Fan & Zengru Di, 2016. "Ranking scientific publications with similarity-preferential mechanism," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 805-816, February.
    8. Zhang, Fang & Wu, Shengli, 2020. "Predicting future influence of papers, researchers, and venues in a dynamic academic network," Journal of Informetrics, Elsevier, vol. 14(2).
    9. Yanbo Zhou & Xin-Li Xu & Xu-Hua Yang & Qu Li, 2022. "The influence of disruption on evaluating the scientific significance of papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5931-5945, October.
    10. Ruijie Wang & Yuhao Zhou & An Zeng, 2023. "Evaluating scientists by citation and disruption of their representative works," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1689-1710, March.
    11. Hao Wang & Hua-Wei Shen & Xue-Qi Cheng, 2016. "Scientific credit diffusion: Researcher level or paper level?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 827-837, November.
    12. Zeng, Tong & Wu, Longfeng & Bratt, Sarah & Acuna, Daniel E., 2020. "Assigning credit to scientific datasets using article citation networks," Journal of Informetrics, Elsevier, vol. 14(2).
    13. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & J. Martinez-Baena, 2012. "On first quartile journals which are not of highest impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 925-943, March.
    14. Ding, Ying & Liu, Xiaozhong & Guo, Chun & Cronin, Blaise, 2013. "The distribution of references across texts: Some implications for citation analysis," Journal of Informetrics, Elsevier, vol. 7(3), pages 583-592.
    15. Yubing Nie & Yifan Zhu & Qika Lin & Sifan Zhang & Pengfei Shi & Zhendong Niu, 2019. "Academic rising star prediction via scholar’s evaluation model and machine learning techniques," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 461-476, August.
    16. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2011. "Overall prestige of journals with ranking score above a given threshold," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 229-243, October.
    17. Wenjia Zhu & Jiancheng Guan, 2013. "A bibliometric study of service innovation research: based on complex network analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1195-1216, March.
    18. Bai, Xiaomei & Zhang, Fuli & Liu, Jiaying & Xia, Feng, 2023. "Quantifying the impact of scientific collaboration and papers via motif-based heterogeneous networks," Journal of Informetrics, Elsevier, vol. 17(2).
    19. Johannes König & David I. Stern & Richard S.J. Tol, 2022. "Confidence Intervals for Recursive Journal Impact Factors," Tinbergen Institute Discussion Papers 22-038/III, Tinbergen Institute.
    20. Xu, Shuqi & Mariani, Manuel Sebastian & Lü, Linyuan & Medo, Matúš, 2020. "Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data," Journal of Informetrics, Elsevier, vol. 14(1).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:17:y:2023:i:1:s1751157723000044. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.