IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v105y2015i3d10.1007_s11192-015-1703-6.html
   My bibliography  Save this article

A flexible bibliometric approach for the assessment of professorial appointments

Author

Listed:
  • Juan Gorraiz

    (University of Vienna)

  • Christian Gumpenberger

    (University of Vienna)

Abstract

Recruitment and professorial appointment procedures are crucial for the administration and management of universities and higher education institutions in order to guarantee a certain level of performance quality and reputation. The complementary use of quantitative and objective bibliometric analyses is meant to be an enhancement for the assessment of candidates and a possible antidote for subjective, discriminatory and corrupt practices. In this paper, we present the Vienna University bibliometric approach, offering a method which relies on a variety of basicindicators and further control parameters in order to address the multidimensionality of the problem and to foster comprehensibility. Our “top counts approach” allows an appointment committee to pick and choose from a portfolio of indicators according to the actual strategic alignment. Furthermore, control and additional data help to understand disciplinary publication habits, to unveil concealed aspects and to identify individual publication strategies of the candidates. Our approach has already been applied to 14 professorial appointment procedures (PAP) in the life sciences, earth and environmental sciences and social sciences, comprising 221 candidates in all. The usefulness of the bibliometric approach was confirmed by all heads of appointment committees in the life sciences. For the earth and environmental sciences as well as the social sciences, the usefulness was less obvious and sometimes questioned due to the low coverage of the candidates’ publication output in the traditional citation data sources. A retrospective assessment of all hitherto performed PAP also showed an overlap between the committees’ designated top candidates and the bibliometric top candidates to a certain degree.

Suggested Citation

  • Juan Gorraiz & Christian Gumpenberger, 2015. "A flexible bibliometric approach for the assessment of professorial appointments," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1699-1719, December.
    • Handle: RePEc:spr:scient:v:105:y:2015:i:3:d:10.1007_s11192-015-1703-6
    DOI: 10.1007/s11192-015-1703-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-015-1703-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-015-1703-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jonathan Adams & Karen Gurney & Stuart Marshall, 2007. "Profiling citation impact: A new methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 325-344, August.
    2. Juan Gorraiz & Ralph Reimann & Christian Gumpenberger, 2012. "Key factors and considerations in the assessment of international collaboration: a case study for Austria and six countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 417-433, May.
    3. repec:cte:idrepe:id-12-02 is not listed on IDEAS
    4. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    5. Natalia Zinovyeva & Manuel Bagues, 2015. "The Role of Connections in Academic Promotions," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 264-292, April.
    6. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems," Journal of Informetrics, Elsevier, vol. 6(2), pages 155-168.
    7. Juan Gorraiz & Philip J. Purnell & Wolfgang Glänzel, 2013. "Opportunities for and limitations of the Book Citation Index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(7), pages 1388-1398, July.
    8. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    9. Fabio Ferlazzo & Stefano Sdoia, 2012. "Measuring Nepotism through Shared Last Names: Are We Really Moving from Opinions to Facts?," PLOS ONE, Public Library of Science, vol. 7(8), pages 1-6, August.
    10. Grit Laudel, 2002. "What do we measure by co-authorships?," Research Evaluation, Oxford University Press, vol. 11(1), pages 3-15, April.
    11. Frederick van der Ploeg, 2008. "Towards Evidence-based Reform of European Universities," CESifo Economic Studies, CESifo Group, vol. 54(2), pages 99-120, June.
    12. Combes, Pierre-Philippe & Linnemer, Laurent & Visser, Michael, 2008. "Publish or peer-rich? The role of skills and networks in hiring economics professors," Labour Economics, Elsevier, vol. 15(3), pages 423-441, June.
    13. Pierre-Philippe Combes & Laurent Linnemer & Michael Visser, 2008. "Publish or Peer-rich," Post-Print hal-00558189, HAL.
    14. Jean-François Bach & J.F. Bach & Denis Jérome & Brigitte d'Artemare, 2011. "On the proper use of bibliometrics to evaluate individual researchers [Du bon usage de la bibliométrie pour l'évaluation individuelle des chercheurs]," Working Papers hal-00604136, HAL.
    15. Stefano Allesina, 2011. "Measuring Nepotism through Shared Last Names: The Case of Italian Academia," PLOS ONE, Public Library of Science, vol. 6(8), pages 1-6, August.
    16. Olle Persson & Wolfgang Glänzel & Rickard Danell, 2004. "Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(3), pages 421-432, August.
    17. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    18. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    19. Wolfgang Glänzel & Koenraad Debackere & Bart Thijs & András Schubert, 2006. "A concise review on the role of author self-citations in information science, bibliometrics and science policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(2), pages 263-277, May.
    20. Judit Bar-Ilan, 2008. "Which h-index? — A comparison of WoS, Scopus and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 257-271, February.
    21. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    22. Juan Gorraiz & Philip J. Purnell & Wolfgang Glänzel, 2013. "Opportunities for and limitations of the Book Citation Index," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(7), pages 1388-1398, July.
    23. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2014. "Career advancement and scientific performance in universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 891-907, February.
    24. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "Revisiting the scaling of citations for research assessment," Journal of Informetrics, Elsevier, vol. 6(4), pages 470-479.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bonaccorsi, Andrea & Haddawy, Peter & Cicero, Tindaro & Hassan, Saeed-Ul, 2017. "The solitude of stars. An analysis of the distributed excellence model of European universities," Journal of Informetrics, Elsevier, vol. 11(2), pages 435-454.
    2. Johannes Sorz & Wolfgang Glänzel & Ursula Ulrych & Christian Gumpenberger & Juan Gorraiz, 2020. "Research strengths identified by esteem and bibliometric indicators: a case study at the University of Vienna," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1095-1116, November.
    3. Belmonte-Ureña, Luis Jesús & Plaza-Úbeda, José Antonio & Vazquez-Brust, Diego & Yakovleva, Natalia, 2021. "Circular economy, degrowth and green growth as pathways for research on sustainable development goals: A global analysis and future agenda," Ecological Economics, Elsevier, vol. 185(C).
    4. Juan Gorraiz & Ursula Ulrych & Wolfgang Glänzel & Wenceslao Arroyo-Machado & Daniel Torres-Salinas, 2022. "Measuring the excellence contribution at the journal level: an alternative to Garfield’s impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7229-7251, December.
    5. Thomas C. Erren & J. Valérie Groß, 2016. "Research metrics: What about weighted citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 315-316, April.
    6. Sánchez-Gil, Susana & Gorraiz, Juan & Melero-Fuentes, David, 2018. "Reference density trends in the major disciplines," Journal of Informetrics, Elsevier, vol. 12(1), pages 42-58.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Bonaccorsi, Andrea & Haddawy, Peter & Cicero, Tindaro & Hassan, Saeed-Ul, 2017. "The solitude of stars. An analysis of the distributed excellence model of European universities," Journal of Informetrics, Elsevier, vol. 11(2), pages 435-454.
    3. Michel Zitt, 2012. "The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 485-503, August.
    4. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Rosati, Francesco, 2016. "A methodology to measure the effectiveness of academic recruitment and turnover," Journal of Informetrics, Elsevier, vol. 10(1), pages 31-42.
    5. Marcel Clermont & Johanna Krolak & Dirk Tunger, 2021. "Does the citation period have any effect on the informative value of selected citation indicators in research evaluations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1019-1047, February.
    6. Bosquet, Clément & Combes, Pierre-Philippe & Garcia-Penalosa, Cecilia, 2013. "Gender and competition: evidence from academic promotions in France," LSE Research Online Documents on Economics 58350, London School of Economics and Political Science, LSE Library.
    7. repec:hal:spmain:info:hdl:2441/37ufknmfv39tppkbjb4dmidnqe is not listed on IDEAS
    8. Hamid Bouabid & Hind Achachi, 2022. "Size of science team at university and internal co-publications: science policy implications," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6993-7013, December.
    9. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    10. repec:spo:wpmain:info:hdl:2441/1seuirq4ak9b9bouu1j29ebui7 is not listed on IDEAS
    11. Anne Boring, 2015. "Gender Biases in Student Evaluations of Teachers," SciencePo Working papers Main hal-03470161, HAL.
    12. Clément Bosquet & Pierre-Philippe Combes, 2013. "Are academics who publish more also more cited? Individual determinants of publication and citation records," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 831-857, December.
    13. Pierre Deschamps, 2024. "Gender Quotas in Hiring Committees: A Boon or a Bane for Women?," Management Science, INFORMS, vol. 70(11), pages 7486-7505, November.
    14. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    15. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2016. "Gender bias in academic recruitment," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 119-141, January.
    16. Juan Gorraiz & Ursula Ulrych & Wolfgang Glänzel & Wenceslao Arroyo-Machado & Daniel Torres-Salinas, 2022. "Measuring the excellence contribution at the journal level: an alternative to Garfield’s impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7229-7251, December.
    17. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2014. "Career advancement and scientific performance in universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 891-907, February.
    18. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.
    19. Truyken L. B. Ossenblok & Tim C. E. Engels, 2015. "Edited books in the Social Sciences and Humanities: Characteristics and collaboration analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 219-237, July.
    20. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2016. "The north–south divide in the Italian higher education system," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2093-2117, December.
    21. Nikolaos A. Kazakis & Anastasios D. Diamantidis & Leonidas L. Fragidis & Miltos K. Lazarides, 2014. "Evaluating the research performance of the Greek medical schools using bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1367-1384, February.
    22. Jianhua Hou & Xiucai Yang & Chaomei Chen, 2018. "Emerging trends and new developments in information science: a document co-citation analysis (2009–2016)," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 869-892, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:105:y:2015:i:3:d:10.1007_s11192-015-1703-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.