IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v122y2020i1d10.1007_s11192-019-03278-7.html
   My bibliography  Save this article

A flexible approach for measuring author-level publishing performance

Author

Listed:
  • Nadia Simoes

    (Instituto Universitário de Lisboa (ISCTE - IUL))

  • Nuno Crespo

    (Instituto Universitário de Lisboa (ISCTE - IUL))

Abstract

We propose a framework to evaluate, in relative terms, author-level publishing performance. To that end we introduce the publishing performance index (PPI) and the publishing performance box (PPB), and discuss the associated publishing profiles. We illustrate our approach conducting an extensive empirical application covering 472 top economists and developing several robustness tests. Instead of using a pre-designed measure without flexibility to adjust to the circumstances of each specific case, our approach accommodates alternative evaluation criteria, as defined by the evaluators. Beyond this key characteristic, our approach has some other important advantages: (1) it is easy to apply; (2) it is sensitive to the full list of publications and citations; (3) it is able to include additional dimensions of scientific performance beyond papers and citations; (4) it is a high granularity measure, providing a complete ranking of the authors under analysis.

Suggested Citation

  • Nadia Simoes & Nuno Crespo, 2020. "A flexible approach for measuring author-level publishing performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 331-355, January.
    • Handle: RePEc:spr:scient:v:122:y:2020:i:1:d:10.1007_s11192-019-03278-7
    DOI: 10.1007/s11192-019-03278-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03278-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-019-03278-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    2. Frandsen, Tove Faber & Nicolaisen, Jeppe, 2010. "What is in a name? Credit assignment practices in different disciplines," Journal of Informetrics, Elsevier, vol. 4(4), pages 608-617.
    3. Ludo Waltman & Rodrigo Costas, 2014. "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 433-445, March.
    4. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    5. Motty Perry & Philip J. Reny, 2016. "How to Count Citations If You Must," American Economic Review, American Economic Association, vol. 106(9), pages 2722-2741, September.
    6. Björn Hammarfelt & Alexander D. Rushforth, 2017. "Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation," Research Evaluation, Oxford University Press, vol. 26(3), pages 169-180.
    7. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2008. "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 830-837, March.
    8. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    9. António Osório, 2018. "On the impossibility of a perfect counting method to allocate the credits of multi-authored publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2161-2173, September.
    10. Chao Gao & Zhen Wang & Xianghua Li & Zili Zhang & Wei Zeng, 2016. "PR-Index: Using the h-Index and PageRank for Determining True Impact," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-13, September.
    11. Chun-Ting Zhang, 2009. "The e-Index, Complementing the h-Index for Excess Citations," PLOS ONE, Public Library of Science, vol. 4(5), pages 1-4, May.
    12. Lutz Bornmann & Andreas Thor & Werner Marx & Hermann Schier, 2016. "The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(11), pages 2778-2789, November.
    13. Ying Ding, 2011. "Applying weighted PageRank to author citation networks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 236-245, February.
    14. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    15. Kalervo Järvelin & Olle Persson, 2008. "Erratum re: “The DCI‐index: Discounted cumulated impact‐based research evaluation”, Journal of the American Society for Information Science and Technology, 59(9), 1433‐1440," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(14), pages 2350-2352, December.
    16. Kalervo Järvelin & Olle Persson, 2008. "The DCI index: Discounted cumulated impact‐based research evaluation," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(9), pages 1433-1440, July.
    17. Lutz Bornmann & Robin Haunschild, 2018. "Plots for visualizing paper impact and journal impact of single researchers in a single graph," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 385-394, April.
    18. Sabrina J. Mayer & Justus M. K. Rathmann, 2018. "How does research productivity relate to gender? Analyzing gender differences for multiple publication dimensions," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1663-1693, December.
    19. Philip Ball, 2005. "Index aims for fair ranking of scientists," Nature, Nature, vol. 436(7053), pages 900-900, August.
    20. Marek Kosmulski, 2018. "Are you in top 1% (1‰)?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 557-565, February.
    21. Liu, Xuan Zhen & Fang, Hui, 2012. "Modifying h-index by allocating credit of multi-authored papers whose author names rank based on contribution," Journal of Informetrics, Elsevier, vol. 6(4), pages 557-565.
    22. Anthony F. J. Raan, 2006. "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 491-502, June.
    23. Qiang Wu, 2010. "The w-index: A measure to assess scientific impact by focusing on widely cited papers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(3), pages 609-614, March.
    24. Ad A.M. Prins & Rodrigo Costas & Thed N. van Leeuwen & Paul F. Wouters, 2016. "Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data," Research Evaluation, Oxford University Press, vol. 25(3), pages 264-270.
    25. Henk F Moed, 2007. "The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review," Science and Public Policy, Oxford University Press, vol. 34(8), pages 575-583, October.
    26. Dorte Henriksen, 2018. "What factors are associated with increasing co-authorship in the social sciences? A case study of Danish Economics and Political Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1395-1421, March.
    27. S. Alonso & F. J. Cabrerizo & E. Herrera-Viedma & F. Herrera, 2010. "hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 391-400, February.
    28. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    29. Helmut A. Abt, 2012. "A publication index that is independent of age," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 863-868, June.
    30. Hausken, Kjell, 2016. "The Ranking of Researchers by Publications and Citations," UiS Working Papers in Economics and Finance 2016/11, University of Stavanger.
    31. Wolfgang Glänzel & Koenraad Debackere & Bart Thijs & András Schubert, 2006. "A concise review on the role of author self-citations in information science, bibliometrics and science policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(2), pages 263-277, May.
    32. Kadel, Annke & Walter, Andreas, 2015. "Do scholars in Economics and Finance react to alphabetical discrimination?," Finance Research Letters, Elsevier, vol. 14(C), pages 64-68.
    33. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    34. Kjell HAUSKEN, 2016. "The Ranking of Researchers by Publications and Citations: Using RePEc Data," Journal of Economics Bibliography, KSP Journals, vol. 3(4), pages 530-558, December.
    35. Trevor Fenner & Martyn Harris & Mark Levene & Judit Bar-Ilan, 2018. "A novel bibliometric index with a simple geometric interpretation," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-14, July.
    36. Ying Ding, 2011. "Applying weighted PageRank to author citation networks," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 236-245, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rojko, Katarina & Lužar, Borut, 2022. "Scientific performance across research disciplines: Trends and differences in the case of Slovenia," Journal of Informetrics, Elsevier, vol. 16(2).
    2. Marek Kwiek & Wojciech Roszka, 2022. "Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3543-3575, June.
    3. Gustavo Vicentis Oliveira Fernandes & Juliana Campos Hasse Fernandes, 2024. "GFsa (GF “Scientific Age”) Index Application for Assessment of 1020 Highly Cited Researchers in Dentistry: A Pilot Study Comparing GFsa Index and H-Index," Publications, MDPI, vol. 12(2), pages 1-11, June.
    4. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.
    5. Xie, Qing & Zhang, Xinyuan & Song, Min, 2021. "A network embedding-based scholar assessment indicator considering four facets: Research topic, author credit allocation, field-normalized journal impact, and published time," Journal of Informetrics, Elsevier, vol. 15(4).
    6. Yitong Chen & Keye Wu & Yue Li & Jianjun Sun, 2023. "Impacts of inter-institutional mobility on scientific performance from research capital and social capital perspectives," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3473-3506, June.
    7. Ana Teresa Santos & Sandro Mendonça, 2022. "Do papers (really) match journals’ “aims and scope”? A computational assessment of innovation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7449-7470, December.
    8. Orlando Fonseca Guilarte & Simone Diniz Junqueira Barbosa & Sinesio Pesco, 2021. "RelPath: an interactive tool to visualize branches of studies and quantify the expertise of authors by citation paths," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4871-4897, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simoes, Nadia & Crespo, Nuno, 2020. "Self-Citations and scientific evaluation: Leadership, influence, and performance," Journal of Informetrics, Elsevier, vol. 14(1).
    2. Sidiropoulos, A. & Gogoglou, A. & Katsaros, D. & Manolopoulos, Y., 2016. "Gazing at the skyline for star scientists," Journal of Informetrics, Elsevier, vol. 10(3), pages 789-813.
    3. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    4. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    5. Parul Khurana & Kiran Sharma, 2022. "Impact of h-index on author’s rankings: an improvement to the h-index for lower-ranked authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4483-4498, August.
    6. Madiha Ameer & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its qualitative and quantitative variants in Neuroscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 653-673, November.
    7. Muhammad Usman & Ghulam Mustafa & Muhammad Tanvir Afzal, 2021. "Ranking of author assessment parameters using Logistic Regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 335-353, January.
    8. J. E. Hirsch, 2019. "hα: An index to quantify an individual’s scientific leadership," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 673-686, February.
    9. Franceschini, Fiorenzo & Maisano, Domenico, 2010. "The citation triad: An overview of a scientist's publication output based on Ferrers diagrams," Journal of Informetrics, Elsevier, vol. 4(4), pages 503-511.
    10. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    11. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    12. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    13. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    14. Lorna Wildgaard, 2015. "A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 873-906, September.
    15. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    16. Lathabai, Hiran H., 2020. "ψ-index: A new overall productivity index for actors of science and technology," Journal of Informetrics, Elsevier, vol. 14(4).
    17. Yu Liu & Wei Zuo & Ying Gao & Yanhong Qiao, 2013. "Comprehensive geometrical interpretation of h-type indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 605-615, August.
    18. Kuan, Chung-Huei & Huang, Mu-Hsuan & Chen, Dar-Zen, 2011. "Ranking patent assignee performance by h-index and shape descriptors," Journal of Informetrics, Elsevier, vol. 5(2), pages 303-312.
    19. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    20. Wei, Shelia X. & Tong, Tong & Rousseau, Ronald & Wang, Wanru & Ye, Fred Y., 2022. "Relations among the h-, g-, ψ-, and p-index and offset-ability," Journal of Informetrics, Elsevier, vol. 16(4).

    More about this item

    Keywords

    Research evaluation; Scientific outputs; Bibliometrics; Selection criteria; Evaluators;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • M51 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics - - - Firm Employment Decisions; Promotions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:122:y:2020:i:1:d:10.1007_s11192-019-03278-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.