IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i10d10.1007_s11192-022-04485-5.html
   My bibliography  Save this article

The role of citation networks to explain academic promotions: an empirical analysis of the Italian national scientific qualification

Author

Listed:
  • Maria Cristiana Martini

    (University of Modena and Reggio Emilia)

  • Elvira Pelle

    (University of Modena and Reggio Emilia)

  • Francesco Poggi

    (Italian National Research Council (CNR), Institute of Cognitive Sciences and Technologies (ISTC))

  • Andrea Sciandra

    (University of Modena and Reggio Emilia)

Abstract

The aim of this paper is to study the role of citation network measures in the assessment of scientific maturity. Referring to the case of the Italian national scientific qualification (ASN), we investigate if there is a relationship between citation network indices and the results of the researchers’ evaluation procedures. In particular, we want to understand if network measures can enhance the prediction accuracy of the results of the evaluation procedures beyond basic performance indices. Moreover, we want to highlight which citation network indices prove to be more relevant in explaining the ASN results, and if quantitative indices used in the citation-based disciplines assessment can replace the citation network measures in non-citation-based disciplines. Data concerning Statistics and Computer Science disciplines are collected from different sources (ASN, Italian Ministry of University and Research, and Scopus) and processed in order to calculate the citation-based measures used in this study. Then, we apply logistic regression models to estimate the effects of network variables. We find that network measures are strongly related to the results of the ASN and significantly improve the explanatory power of the models, especially for the research fields of Statistics. Additionally, citation networks in the specific sub-disciplines are far more relevant than those in the general disciplines. Finally, results show that the citation network measures are not a substitute of the citation-based bibliometric indices.

Suggested Citation

  • Maria Cristiana Martini & Elvira Pelle & Francesco Poggi & Andrea Sciandra, 2022. "The role of citation networks to explain academic promotions: an empirical analysis of the Italian national scientific qualification," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5633-5659, October.
    • Handle: RePEc:spr:scient:v:127:y:2022:i:10:d:10.1007_s11192-022-04485-5
    DOI: 10.1007/s11192-022-04485-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04485-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-022-04485-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Andrea Giovanni Nuzzolese & Paolo Ciancarini & Aldo Gangemi & Silvio Peroni & Francesco Poggi & Valentina Presutti, 2019. "Do altmetrics work for assessing research quality?," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 539-562, February.
    2. Saikou Y. Diallo & Christopher J. Lynch & Ross Gore & Jose J. Padilla, 2016. "Identifying key papers within a journal via network centrality measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1005-1020, June.
    3. Marzolla, Moreno, 2016. "Assessing evaluation procedures for individual researchers: The case of the Italian National Scientific Qualification," Journal of Informetrics, Elsevier, vol. 10(2), pages 408-438.
    4. Ying Ding & Erjia Yan & Arthur Frazho & James Caverlee, 2009. "PageRank for ranking authors in co‐citation networks," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(11), pages 2229-2243, November.
    5. Federico Scarpa & Vincenzo Bianco & Luca A. Tagliafico, 2018. "The impact of the national assessment exercises on self-citation rate and publication venue: an empirical investigation on the engineering academic sector in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 997-1022, November.
    6. S. R. Goldberg & H. Anthony & T. S. Evans, 2015. "Modelling citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1577-1604, December.
    7. Vieira, Elizabeth S. & Cabral, José A.S. & Gomes, José A.N.F., 2014. "How good is a model based on bibliometric indicators in predicting the final decisions made by peers?," Journal of Informetrics, Elsevier, vol. 8(2), pages 390-405.
    8. Thomas Hugh Feeley & Katherine Hart LaVail & George A. Barnett, 2011. "Predicting faculty job centrality in communication," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 303-314, May.
    9. Marzolla, Moreno, 2015. "Quantitative analysis of the Italian National Scientific Qualification," Journal of Informetrics, Elsevier, vol. 9(2), pages 285-316.
    10. Butts, Carter T., 2008. "Social Network Analysis with sna," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 24(i06).
    11. Marco Pautasso, 2015. "The Italian University Habilitation and the Challenge of Increasing the Representation of Women in Academia," Challenges, MDPI, vol. 6(1), pages 1-16, March.
    12. Ding, Ying, 2011. "Scientific collaboration and endorsement: Network analysis of coauthorship and citation networks," Journal of Informetrics, Elsevier, vol. 5(1), pages 187-203.
    13. Xuan Liu & Shan Jiang & Hsinchun Chen & Catherine A. Larson & Mihail C. Roco, 2015. "Modeling knowledge diffusion in scientific innovation networks: an institutional comparison between China and US with illustration for nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1953-1984, December.
    14. Silvio Peroni & Paolo Ciancarini & Aldo Gangemi & Andrea Giovanni Nuzzolese & Francesco Poggi & Valentina Presutti, 2020. "The practice of self-citations: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 253-282, April.
    15. Erjia Yan & Ying Ding, 2009. "Applying centrality measures to impact analysis: A coauthorship network analysis," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(10), pages 2107-2118, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Adrian Furnham, 2023. "Peer nominations as scientometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1451-1458, February.
    2. Federica Bologna & Angelo Iorio & Silvio Peroni & Francesco Poggi, 2023. "Do open citations give insights on the qualitative peer-review evaluation in research assessments? An analysis of the Italian National Scientific Qualification," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 19-53, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Camil Demetrescu & Irene Finocchi & Andrea Ribichini & Marco Schaerf, 2020. "On bibliometrics in academic promotions: a case study in computer science and engineering in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2207-2228, September.
    2. Jianhua Hou & Bili Zheng & Yang Zhang & Chaomei Chen, 2021. "How do Price medalists’ scholarly impact change before and after their awards?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5945-5981, July.
    3. Silvio Peroni & Paolo Ciancarini & Aldo Gangemi & Andrea Giovanni Nuzzolese & Francesco Poggi & Valentina Presutti, 2020. "The practice of self-citations: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 253-282, April.
    4. Mehmet Ali Köseoglu & John A. Parnell & Melissa Yan Yee Yick, 2021. "Identifying influential studies and maturity level in intellectual structure of fields: evidence from strategic management," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1271-1309, February.
    5. Jun-Ping Qiu & Ke Dong & Hou-Qiang Yu, 2014. "Comparative study on structure and correlation among author co-occurrence networks in bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1345-1360, November.
    6. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    7. Giovanni Abramo & Ciriaco D’Angelo, 2015. "An assessment of the first “scientific habilitation” for university appointments in Italy," Economia Politica: Journal of Analytical and Institutional Economics, Springer;Fondazione Edison, vol. 32(3), pages 329-357, December.
    8. Li, Eldon Y. & Liao, Chien Hsiang & Yen, Hsiuju Rebecca, 2013. "Co-authorship networks and research impact: A social capital perspective," Research Policy, Elsevier, vol. 42(9), pages 1515-1530.
    9. Saikou Y. Diallo & Christopher J. Lynch & Ross Gore & Jose J. Padilla, 2016. "Identifying key papers within a journal via network centrality measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1005-1020, June.
    10. Zhang, Yi & Wu, Mengjia & Miao, Wen & Huang, Lu & Lu, Jie, 2021. "Bi-layer network analytics: A methodology for characterizing emerging general-purpose technologies," Journal of Informetrics, Elsevier, vol. 15(4).
    11. Celeste Vong & Paulo Rita & Nuno António, 2021. "Health-Related Crises in Tourism Destination Management: A Systematic Review," Sustainability, MDPI, vol. 13(24), pages 1-28, December.
    12. Zhou, Yuhao & Wang, Ruijie & Zeng, An & Zhang, Yi-Cheng, 2020. "Identifying prize-winning scientists by a competition-aware ranking," Journal of Informetrics, Elsevier, vol. 14(3).
    13. Mehmet Ali Koseoglu, 2016. "Mapping the institutional collaboration network of strategic management research: 1980–2014," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(1), pages 203-226, October.
    14. Jianlin Zhou & An Zeng & Ying Fan & Zengru Di, 2018. "Identifying important scholars via directed scientific collaboration networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1327-1343, March.
    15. Kazuki Nakajima & Kazuyuki Shudo & Naoki Masuda, 2023. "Higher-order rich-club phenomenon in collaborative research grant networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2429-2446, April.
    16. Zhang, Xinyuan & Xie, Qing & Song, Min, 2021. "Measuring the impact of novelty, bibliometric, and academic-network factors on citation count using a neural network," Journal of Informetrics, Elsevier, vol. 15(2).
    17. Yong Huang & Yi Bu & Ying Ding & Wei Lu, 2018. "Number versus structure: towards citing cascades," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 2177-2193, December.
    18. Marzolla, Moreno, 2016. "Assessing evaluation procedures for individual researchers: The case of the Italian National Scientific Qualification," Journal of Informetrics, Elsevier, vol. 10(2), pages 408-438.
    19. Federica Bologna & Angelo Iorio & Silvio Peroni & Francesco Poggi, 2023. "Do open citations give insights on the qualitative peer-review evaluation in research assessments? An analysis of the Italian National Scientific Qualification," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 19-53, January.
    20. Erik Boetto & Maria Pia Fantini & Aldo Gangemi & Davide Golinelli & Manfredi Greco & Andrea Giovanni Nuzzolese & Valentina Presutti & Flavia Rallo, 2021. "Using altmetrics for detecting impactful research in quasi-zero-day time-windows: the case of COVID-19," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1189-1215, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:10:d:10.1007_s11192-022-04485-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.