IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v10y2016i2p471-484.html
   My bibliography  Save this article

A data analytic approach to quantifying scientific impact

Author

Listed:
  • Cao, Xuanyu
  • Chen, Yan
  • Ray Liu, K.J.

Abstract

Citation is perhaps the mostly used metric to evaluate the scientific impact of papers. Various measures of the scientific impact of researchers and journals rely heavily on the citations of papers. Furthermore, in many practical applications, people may need to know not only the current citations of a paper, but also a prediction of its future citations. However, the complex heterogeneous temporal patterns of the citation dynamics make the predictions of future citations rather difficult. The existing state-of-the-art approaches used parametric methods that require long period of data and have poor performance on some scientific disciplines. In this paper, we present a simple yet effective and robust data analytic method to predict future citations of papers from a variety of disciplines. With rather short-term (e.g., 3 years after the paper is published) citation data, the proposed approach can give accurate estimate of future citations, outperforming state-of-the-art prediction methods significantly. Extensive experiments confirm the robustness of the proposed approach across various journals of different disciplines.

Suggested Citation

  • Cao, Xuanyu & Chen, Yan & Ray Liu, K.J., 2016. "A data analytic approach to quantifying scientific impact," Journal of Informetrics, Elsevier, vol. 10(2), pages 471-484.
  • Handle: RePEc:eee:infome:v:10:y:2016:i:2:p:471-484
    DOI: 10.1016/j.joi.2016.02.006
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157715301346
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.joi.2016.02.006?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    2. Stegehuis, Clara & Litvak, Nelly & Waltman, Ludo, 2015. "Predicting the long-term citation impact of recent publications," Journal of Informetrics, Elsevier, vol. 9(3), pages 642-657.
    3. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
    4. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    5. repec:grz:wpaper:2012-02 is not listed on IDEAS
    6. Michael J Stringer & Marta Sales-Pardo & Luís A Nunes Amaral, 2008. "Effectiveness of Journal Ranking Schemes as a Tool for Locating Information," PLOS ONE, Public Library of Science, vol. 3(2), pages 1-8, February.
    7. Frank Havemann & Birger Larsen, 2015. "Bibliometric indicators of young authors in astrophysics: Can later stars be predicted?," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1413-1434, February.
    8. Daniel E. Acuna & Stefano Allesina & Konrad P. Kording, 2012. "Predicting scientific success," Nature, Nature, vol. 489(7415), pages 201-202, September.
    9. Schreiber, Michael, 2013. "How relevant is the predictive power of the h-index? A case study of the time-dependent Hirsch index," Journal of Informetrics, Elsevier, vol. 7(2), pages 325-329.
    10. Smolinsky, Lawrence, 2016. "Expected number of citations and the crown indicator," Journal of Informetrics, Elsevier, vol. 10(1), pages 43-47.
    11. Ludo Waltman & Rodrigo Costas, 2014. "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 433-445, March.
    12. Thed N. van Leeuwen & Henk F. Moed, 2005. "Characteristics of journal impact factors: The effects of uncitedness and citation distribution on the understanding of journal impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(2), pages 357-371, April.
    13. Anthony Breitzman & Patrick Thomas, 2015. "Inventor team size as a predictor of the future citation impact of patents," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 631-647, May.
    14. Tian Yu & Guang Yu & Peng-Yu Li & Liang Wang, 2014. "Citation impact prediction for scientific papers using stepwise regression analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1233-1252, November.
    15. Young-Ho Eom & Santo Fortunato, 2011. "Characterizing and Modeling Citation Dynamics," PLOS ONE, Public Library of Science, vol. 6(9), pages 1-7, September.
    16. Haunschild, Robin & Bornmann, Lutz, 2016. "Normalization of Mendeley reader counts for impact assessment," Journal of Informetrics, Elsevier, vol. 10(1), pages 62-73.
    17. Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    18. Ajiferuke, Isola & Famoye, Felix, 2015. "Modelling count response variables in informetric studies: Comparison among count, linear, and lognormal regression models," Journal of Informetrics, Elsevier, vol. 9(3), pages 499-513.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kehan Wang & Wenxuan Shi & Junsong Bai & Xiaoping Zhao & Liying Zhang, 2021. "Prediction and application of article potential citations based on nonlinear citation-forecasting combined model," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6533-6550, August.
    2. Fenghua Wang & Ying Fan & An Zeng & Zengru Di, 2019. "Can we predict ESI highly cited publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 109-125, January.
    3. Akella, Akhil Pandey & Alhoori, Hamed & Kondamudi, Pavan Ravikanth & Freeman, Cole & Zhou, Haiming, 2021. "Early indicators of scientific impact: Predicting citations with altmetrics," Journal of Informetrics, Elsevier, vol. 15(2).
    4. Xie, Zheng, 2020. "Predicting publication productivity for researchers: A piecewise Poisson model," Journal of Informetrics, Elsevier, vol. 14(3).
    5. Bai, Xiaomei & Zhang, Fuli & Lee, Ivan, 2019. "Predicting the citations of scholarly paper," Journal of Informetrics, Elsevier, vol. 13(1), pages 407-418.
    6. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    7. Wumei Du & Zheng Xie & Yiqin Lv, 2021. "Predicting publication productivity for authors: Shallow or deep architecture?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5855-5879, July.
    8. Xiaomei Bai & Fuli Zhang & Jinzhou Li & Zhong Xu & Zeeshan Patoli & Ivan Lee, 2021. "Quantifying scientific collaboration impact by exploiting collaboration-citation network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7993-8008, September.
    9. Li, Xin & Ma, Xiaodi & Feng, Ye, 2024. "Early identification of breakthrough research from sleeping beauties using machine learning," Journal of Informetrics, Elsevier, vol. 18(2).
    10. Zhang, Fang & Wu, Shengli, 2020. "Predicting future influence of papers, researchers, and venues in a dynamic academic network," Journal of Informetrics, Elsevier, vol. 14(2).
    11. Fang Zhang & Shengli Wu, 2024. "Predicting citation impact of academic papers across research areas using multiple models and early citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4137-4166, July.
    12. Cui, Haochuan & Zeng, An & Fan, Ying & Di, Zengru, 2021. "Quantifying the impact of a teamwork publication," Journal of Informetrics, Elsevier, vol. 15(4).
    13. Jun Zhang & Yan Hu & Zhaolong Ning & Amr Tolba & Elsayed Elashkar & Feng Xia, 2018. "AIRank: Author Impact Ranking through Positions in Collaboration Networks," Complexity, Hindawi, vol. 2018, pages 1-16, June.
    14. Anqi Ma & Yu Liu & Xiujuan Xu & Tao Dong, 2021. "A deep-learning based citation count prediction model with paper metadata semantic features," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6803-6823, August.
    15. Shaibu Mohammed & Anthony Morgan & Emmanuel Nyantakyi, 2020. "On the influence of uncited publications on a researcher’s h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1791-1799, March.
    16. Yuhao Zhou & Ruijie Wang & An Zeng, 2022. "Predicting the impact and publication date of individual scientists’ future papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1867-1882, April.
    17. Zhiya Zuo & Kang Zhao, 2021. "Understanding and predicting future research impact at different career stages—A social network perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(4), pages 454-472, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    2. Stegehuis, Clara & Litvak, Nelly & Waltman, Ludo, 2015. "Predicting the long-term citation impact of recent publications," Journal of Informetrics, Elsevier, vol. 9(3), pages 642-657.
    3. David I Stern, 2014. "High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
    4. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    5. Yin, Yian & Wang, Dashun, 2017. "The time dimension of science: Connecting the past to the future," Journal of Informetrics, Elsevier, vol. 11(2), pages 608-621.
    6. Pech, Gerson & Delgado, Catarina, 2021. "Screening the most highly cited papers in longitudinal bibliometric studies and systematic literature reviews of a research field or journal: Widespread used metrics vs a percentile citation-based app," Journal of Informetrics, Elsevier, vol. 15(3).
    7. Wang, Xing & Zhang, Zhihui, 2020. "Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact," Journal of Informetrics, Elsevier, vol. 14(2).
    8. Thelwall, Mike, 2016. "The discretised lognormal and hooked power law distributions for complete citation data: Best options for modelling and regression," Journal of Informetrics, Elsevier, vol. 10(2), pages 336-346.
    9. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Felici, Giovanni, 2019. "Predicting publication long-term impact through a combination of early citations and journal impact factor," Journal of Informetrics, Elsevier, vol. 13(1), pages 32-49.
    10. Fang Zhang & Shengli Wu, 2024. "Predicting citation impact of academic papers across research areas using multiple models and early citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4137-4166, July.
    11. Hu, Ya-Han & Tai, Chun-Tien & Liu, Kang Ernest & Cai, Cheng-Fang, 2020. "Identification of highly-cited papers using topic-model-based and bibliometric features: the consideration of keyword popularity," Journal of Informetrics, Elsevier, vol. 14(1).
    12. Schreiber, Michael, 2015. "Restricting the h-index to a publication and citation time window: A case study of a timed Hirsch index," Journal of Informetrics, Elsevier, vol. 9(1), pages 150-155.
    13. Rodrigo Dorantes-Gilardi & Aurora A. Ramírez-Álvarez & Diana Terrazas-Santamaría, 2023. "Is there a differentiated gender effect of collaboration with super-cited authors? Evidence from junior researchers in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2317-2336, April.
    14. Tóth, István & Lázár, Zsolt I. & Varga, Levente & Járai-Szabó, Ferenc & Papp, István & Florian, Răzvan V. & Ercsey-Ravasz, Mária, 2021. "Mitigating ageing bias in article level metrics using citation network analysis," Journal of Informetrics, Elsevier, vol. 15(1).
    15. Gerson Pech & Catarina Delgado, 2020. "Percentile and stochastic-based approach to the comparison of the number of citations of articles indexed in different bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 223-252, April.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. Li, Xin & Ma, Xiaodi & Feng, Ye, 2024. "Early identification of breakthrough research from sleeping beauties using machine learning," Journal of Informetrics, Elsevier, vol. 18(2).
    18. Michal Brzezinski, 2015. "Power laws in citation distributions: evidence from Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 213-228, April.
    19. Bornmann, Lutz, 2014. "Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime," Journal of Informetrics, Elsevier, vol. 8(4), pages 935-950.
    20. Babak Sohrabi & Hamideh Iraj, 2017. "The effect of keyword repetition in abstract and keyword frequency per journal in predicting citation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 243-251, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:10:y:2016:i:2:p:471-484. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.