IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v86y2011i2d10.1007_s11192-010-0297-2.html
   My bibliography  Save this article

National-scale research performance assessment at the individual level

Author

Listed:
  • Giovanni Abramo

    (Università degli Studi di Roma “Tor Vergata”
    National Research Council of Italy)

  • Ciriaco Andrea D’Angelo

    (Università degli Studi di Roma “Tor Vergata”)

Abstract

There is an evident and rapid trend towards the adoption of evaluation exercises for national research systems for purposes, among others, of improving allocative efficiency in public funding of individual institutions. However the desired macroeconomic aims could be compromised if internal redistribution of government resources within each research institution does not follow a consistent logic: the intended effects of national evaluation systems can result only if a “funds for quality” rule is followed at all levels of decision-making. The objective of this study is to propose a bibliometric methodology for: (i) large-scale comparative evaluation of research performance by individual scientists, research groups and departments within research institution, to inform selective funding allocations; and (ii) assessment of strengths and weaknesses by field of research, to inform strategic planning and control. The proposed methodology has been applied to the hard science disciplines of the Italian university research system for the period 2004–2006.

Suggested Citation

  • Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "National-scale research performance assessment at the individual level," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 347-364, February.
    • Handle: RePEc:spr:scient:v:86:y:2011:i:2:d:10.1007_s11192-010-0297-2
    DOI: 10.1007/s11192-010-0297-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-010-0297-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-010-0297-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Giovanni Abramo & Ciriaco Andrea D'Angelo & Flavia Di Costa, 2008. "Assessment of sectoral aggregation distortion in research productivity measurements," Research Evaluation, Oxford University Press, vol. 17(2), pages 111-121, June.
    2. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2010. "A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(8), pages 1564-1581, August.
    3. Herman Van den Berghe & Josee A. Houben & Renger E. de Bruin & Henk F. Moed & André Kint & Marc Luwel & Eric H. J. Spruyt, 1998. "Bibliometric indicators of university research performance in Flanders," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 49(1), pages 59-67.
    4. Massimo Franceschet, 2009. "A cluster analysis of scholar and journal bibliometric indicators," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(10), pages 1950-1964, October.
    5. A Bhattacharya & H Newhouse, 2010. "Allocative Efficiency and an Incentive Scheme for Research," Discussion Papers 10/02, Department of Economics, University of York.
    6. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2010. "A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(8), pages 1564-1581, August.
    7. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    8. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    9. Thed N. van Leeuwen & Henk F. Moed, 2002. "Development and application of journal impact measures in the Dutch science system," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 249-266, February.
    10. Franz Strehl & Sabine Reisinger & Michael Kalatschan, 2007. "Funding Systems and their Effects on Higher Education Systems," OECD Education Working Papers 6, OECD Publishing.
    11. Ronald Rousseau & Mark Smeyers, 2000. "Output-Financing at LUC," Scientometrics, Springer;Akadémiai Kiadó, vol. 47(2), pages 379-387, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:hum:wpaper:sfb649dp2018-001 is not listed on IDEAS
    2. Giovanni Abramo & Ciriaco Andrea D’Angelo & Tindaro Cicero, 2012. "What is the appropriate length of the publication period over which to assess research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1005-1017, December.
    3. Zharova, Alona & Härdle, Wolfgang Karl & Lessmann, Stefan, 2023. "Data-driven support for policy and decision-making in university research management: A case study from Germany," European Journal of Operational Research, Elsevier, vol. 308(1), pages 353-368.
    4. Aliakbar Akbaritabar & Andrés F. Castro Torres & Vincent Larivière, 2023. "A global perspective on the social structure of science," MPIDR Working Papers WP-2023-029, Max Planck Institute for Demographic Research, Rostock, Germany.
    5. Alona Zharova & Janine Tellinger-Rice & Wolfgang Karl Härdle, 2018. "How to measure the performance of a Collaborative Research Center," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 1023-1040, November.
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2014. "Relatives in the same university faculty: nepotism or merit?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 737-749, October.
    7. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "National research assessment exercises: the effects of changing the rules of the game during the game," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 229-238, July.
    8. Giovanni Abramo & Tindaro Cicero & Ciriaco Andrea D’Angelo, 2011. "The dangers of performance-based research funding in non-competitive higher education systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 641-654, June.
    9. Carusi, Chiara & Bianchi, Giuseppe, 2019. "Scientific community detection via bipartite scholar/journal graph co-clustering," Journal of Informetrics, Elsevier, vol. 13(1), pages 354-386.
    10. Aliakbar Akbaritabar & Niccolò Casnici & Flaminio Squazzoni, 2018. "The conundrum of research productivity: a study on sociologists in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 859-882, March.
    11. Amara, Nabil & Rhaiem, Mehdi & Halilem, Norrin, 2020. "Assessing the research efficiency of Canadian scholars in the management field: Evidence from the DEA and fsQCA," Journal of Business Research, Elsevier, vol. 115(C), pages 296-306.
    12. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    13. Vincent Larivière & Rodrigo Costas, 2016. "How Many Is Too Many? On the Relationship between Research Productivity and Impact," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-10, September.
    14. Xiaozan Lyu & Rodrigo Costas, 2021. "Studying the characteristics of scientific communities using individual-level bibliometrics: the case of Big Data research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6965-6987, August.
    15. Giovanni Abramo & Ciriaco Andrea D’Angelo & Fulvio Viel, 2011. "The field-standardized average impact of national research systems compared to world average: the case of Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 599-615, August.
    16. Alona Zharova & Wolfgang K. Härdle & Stefan Lessmann, 2017. "Is Scientific Performance a Function of Funds?," SFB 649 Discussion Papers SFB649DP2017-028, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    17. Federico Scarpa & Vincenzo Bianco & Luca A. Tagliafico, 2018. "The impact of the national assessment exercises on self-citation rate and publication venue: an empirical investigation on the engineering academic sector in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 997-1022, November.
    18. Zharova, Alona & Härdle, Wolfgang Karl & Lessmann, Stefan, 2017. "Is scientific performance a function of funds?," SFB 649 Discussion Papers 2017-028, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    19. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2011. "Research productivity: Are higher academic ranks more productive than lower ones?," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 915-928, September.
    20. Giovanni Abramo & Ciriaco Andrea D’Angelo & Fulvio Viel, 2013. "The suitability of h and g indexes for measuring the research performance of institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 555-570, December.
    21. Alfonso Ibáñez & Pedro Larrañaga & Concha Bielza, 2013. "Cluster methods for assessing research performance: exploring Spanish computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 571-600, December.
    22. Yu Liu & Wei Zuo & Ying Gao & Yanhong Qiao, 2013. "Comprehensive geometrical interpretation of h-type indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 605-615, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giovanni Abramo & Tindaro Cicero & Ciriaco Andrea D’Angelo, 2011. "The dangers of performance-based research funding in non-competitive higher education systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 641-654, June.
    2. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    3. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems," Journal of Informetrics, Elsevier, vol. 6(2), pages 155-168.
    4. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2014. "Relatives in the same university faculty: nepotism or merit?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 737-749, October.
    5. Abramo, Giovanni & D'Angelo, CiriacoAndrea & Di Costa, Flavia, 2024. "The moderating role of personal characteristics of authors in the publications’ quality for quantity trade-off," Journal of Informetrics, Elsevier, vol. 18(1).
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2011. "Research productivity: Are higher academic ranks more productive than lower ones?," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 915-928, September.
    7. Ioana Alexandra HORODNIC, 2014. "Academic Performance: Measurement Methods Used In Socio - Economic Sciences," THE YEARBOOK OF THE "GH. ZANE" INSTITUTE OF ECONOMIC RESEARCHES, Gheorghe Zane Institute for Economic and Social Research ( from THE ROMANIAN ACADEMY, JASSY BRANCH), vol. 23(1), pages 5-17.
    8. Cinzia Daraio & Giancarlo Ruocco, 2012. "An Empirical Approach to Compare the Performance of Heterogeneous Academic Fields," DIAG Technical Reports 2012-03, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    9. Marek Kwiek & Wojciech Roszka, 2022. "Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3543-3575, June.
    10. Rodrigo Costas & María Bordons, 2011. "Do age and professional rank influence the order of authorship in scientific publications? Some evidence from a micro-level perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 145-161, July.
    11. Claus-Christian Carbon, 2011. "The Carbon_h-Factor: Predicting Individuals' Research Impact at Early Stages of Their Career," PLOS ONE, Public Library of Science, vol. 6(12), pages 1-7, December.
    12. Giancarlo Ruocco & Cinzia Daraio, 2013. "An empirical approach to compare the performance of heterogeneous academic fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 601-625, December.
    13. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    14. Wildgaard, Lorna, 2016. "A critical cluster analysis of 44 indicators of author-level performance," Journal of Informetrics, Elsevier, vol. 10(4), pages 1055-1078.
    15. Shang, Jing & Zeng, Mingbin & Zhang, Gupeng, 2022. "Investigating the mentorship effect on the academic success of young scientists: An empirical study of the 985 project universities of China," Journal of Informetrics, Elsevier, vol. 16(2).
    16. Jonas Lindahl & Cristian Colliander & Rickard Danell, 2020. "Early career performance and its correlation with gender and publication output during doctoral education," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 309-330, January.
    17. Ioana Alexandra Horodnic & Adriana Zaiţ, 2015. "Motivation and research productivity in a university system undergoing transition," Research Evaluation, Oxford University Press, vol. 24(3), pages 282-292.
    18. Xingchen Li & Qiang Wu & Yuanyuan Liu, 2017. "A quantitative analysis of researcher citation personal display considering disciplinary differences and influence factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1093-1112, November.
    19. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "A national-scale cross-time analysis of university research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 399-413, May.
    20. João M. Santos & Hugo Horta, 2015. "The generational gap of science: a dynamic cluster analysis of doctorates in an evolving scientific system," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 381-406, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:86:y:2011:i:2:d:10.1007_s11192-010-0297-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.