IDEAS home Printed from https://ideas.repec.org/a/gam/jscscx/v2y2013i4p284-297d31000.html
   My bibliography  Save this article

Why an Unbiased External R&D Evaluation System is Important for the Progress of Social Sciences—the Case of a Small Social Science Community

Author

Listed:
  • Franc Mali

    (Faculty of Social Sciences, University of Ljubljana, Kardeljeva pl. 5, Ljubljana 1000, Slovenia)

Abstract

This article deals with the impact of external R&D evaluations as one of the institutional factors that can encourage (or discourage) the progress of the social sciences. A critical overview is presented of the increasing use of bibliometric indicators in the external R&D evaluation procedures employed by the Slovenian Research Agency, which is the leading research council for financing the public sector of social sciences in Slovenia. We attempt to establish that, in order to ensure a good external R&D evaluation practice for a small social science community, it is insufficient to only have reliable bibliometric meta-databases. It is argued that it is equally important to formulate very precise criteria to ascertain their validity.

Suggested Citation

  • Franc Mali, 2013. "Why an Unbiased External R&D Evaluation System is Important for the Progress of Social Sciences—the Case of a Small Social Science Community," Social Sciences, MDPI, vol. 2(4), pages 1-14, December.
    • Handle: RePEc:gam:jscscx:v:2:y:2013:i:4:p:284-297:d:31000
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2076-0760/2/4/284/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2076-0760/2/4/284/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hess, David, 2007. "Social Reporting and New Governance Regulation: The Prospects of Achieving Corporate Accountability Through Transparency," Business Ethics Quarterly, Cambridge University Press, vol. 17(3), pages 453-476, July.
    2. Rémi Barré, 2010. "Towards socially robust S&T indicators: indicators as debatable devices, enabling collective learning," Research Evaluation, Oxford University Press, vol. 19(3), pages 227-231, September.
    3. Stefan Hornbostel, 2001. "Third party funding of German universities. An indicator of research activity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 50(3), pages 523-537, March.
    4. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    5. Mu‐hsuan Huang & Yu‐wei Chang, 2008. "Characteristics of research output in social sciences and humanities: From a research evaluation perspective," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1819-1828, September.
    6. Deborah Brautigam & Michael Woolcock, 2001. "Small States in a Global Economy: the Role of Institutions in Managing Vulnerability and Opportunity in Small Developing Countries," WIDER Working Paper Series DP2001-37, World Institute for Development Economic Research (UNU-WIDER).
    7. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    8. Luka Kronegger & Franc Mali & Anuška Ferligoj & Patrick Doreian, 2012. "Collaboration structures in Slovenian scientific communities," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 631-647, February.
    9. David Pontille & Didier Torny, 2010. "The controversial policies of journal ratings: evaluating social sciences and humanities," Research Evaluation, Oxford University Press, vol. 19(5), pages 347-360, December.
    10. Benedetto Lepori & Carole Probst, 2009. "Using curricula vitae for mapping scientific fields: a small-scale experience for Swiss communication sciences," Research Evaluation, Oxford University Press, vol. 18(2), pages 125-134, June.
    11. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    12. Erik Ernø-Kjølhede & Finn Hansson, 2011. "Measuring research performance during a changing relationship between science and society," Research Evaluation, Oxford University Press, vol. 20(2), pages 131-143, June.
    13. Torres-Salinas, Daniel & Moed, Henk F., 2009. "Library Catalog Analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in Economics," Journal of Informetrics, Elsevier, vol. 3(1), pages 9-26.
    14. Ulrich Schmoch & Torben Schubert & Dorothea Jansen & Richard Heidler & Regina von Görtz, 2010. "How to use indicators to measure scientific performance: a balanced approach," Research Evaluation, Oxford University Press, vol. 19(1), pages 2-18, March.
    15. Kayvan Kousha & Mike Thelwall, 2009. "Google book search: Citation analysis for social science and the humanities," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(8), pages 1537-1549, August.
    16. Ed C. M. Noyons & Clara Calero-Medina, 2009. "Applying bibliometric mapping in a high level science policy context," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(2), pages 261-275, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Romina Rodela, 2016. "On the use of databases about research performance: comments on Karlovčec and Mladenić (2015) and others using the SICRIS database," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2151-2157, December.
    2. Franc Mali & Toni Pustovrh & Rok Platinovšek & Luka Kronegger & Anuška Ferligoj, 2017. "The effects of funding and co-authorship on research performance in a small scientific community," Science and Public Policy, Oxford University Press, vol. 44(4), pages 486-496.
    3. Carlos Miguel Ferreira & Sandro Serpa, 2018. "Online Visibility, Social Networks and Glamorous Scientific Publications," International Journal of Social Science Studies, Redfame publishing, vol. 6(10), pages 58-66, October.
    4. Marjan Cugmas & Franc Mali & Aleš Žiberna, 2020. "Scientific collaboration of researchers and organizations: a two-level blockmodeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2471-2489, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ülle Must, 2012. "Alone or together: examples from history research," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 527-537, May.
    2. Daniel Torres-Salinas & Nicolás Robinson-Garcia & Juan Gorraiz, 2017. "Filling the citation gap: measuring the multidimensional impact of the academic book at institutional level with PlumX," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1371-1384, December.
    3. Boyack, Kevin W. & Klavans, Richard, 2014. "Including cited non-source items in a large-scale map of science: What difference does it make?," Journal of Informetrics, Elsevier, vol. 8(3), pages 569-580.
    4. Ashraf Maleki, 2022. "Why does library holding format really matter for book impact assessment?: Modelling the relationship between citations and altmetrics with print and electronic holdings," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 1129-1160, February.
    5. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    6. Maja Jokić & Andrea Mervar & Stjepan Mateljan, 2019. "Comparative analysis of book citations in social science journals by Central and Eastern European authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1005-1029, September.
    7. Yongjun Zhu & Erjia Yan & Silvio Peroni & Chao Che, 2020. "Nine million book items and eleven million citations: a study of book-based scholarly communication using OpenCitations," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1097-1112, February.
    8. Kousha, Kayvan & Thelwall, Mike, 2018. "Can Microsoft Academic help to assess the citation impact of academic books?," Journal of Informetrics, Elsevier, vol. 12(3), pages 972-984.
    9. Andrea Bonaccorsi & Cinzia Daraio & Stefano Fantoni & Viola Folli & Marco Leonetti & Giancarlo Ruocco, 2017. "Do social sciences and humanities behave like life and hard sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 607-653, July.
    10. Guy Madison & Knut Sundell, 2022. "Numbers of publications and citations for researchers in fields pertinent to the social services: a comparison of peer-reviewed journal publications across six disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 6029-6046, October.
    11. Alesia Zuccala & Thed van Leeuwen, 2011. "Book reviews in humanities research evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(10), pages 1979-1991, October.
    12. Mohammadamin Erfanmanesh & A. Noorhidawati & A. Abrizah, 2019. "What can Bookmetrix tell us about the impact of Springer Nature’s books," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 521-536, October.
    13. Siluo Yang & Xin Xing & Fan Qi & Maria Cláudia Cabrini Grácio, 2021. "Comparison of academic book impact from a disciplinary perspective: an analysis of citations and altmetric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1101-1123, February.
    14. Jerome K. Vanclay & Lutz Bornmann, 2012. "Metrics to evaluate research performance in academic institutions: a critique of ERA 2010 as applied in forestry and the indirect H2 index as a possible alternative," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 751-771, June.
    15. Björn Hammarfelt, 2014. "Using altmetrics for assessing research impact in the humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1419-1430, November.
    16. Kousha, Kayvan & Thelwall, Mike & Rezaie, Somayeh, 2010. "Using the Web for research evaluation: The Integrated Online Impact indicator," Journal of Informetrics, Elsevier, vol. 4(1), pages 124-135.
    17. Lars Herberholz & Berthold U. Wigger, 2020. "Efficiency of European Universities: A Comparison of Peers," CESifo Working Paper Series 8044, CESifo.
    18. Ekaterina Dyachenko, 2013. "Internationalization of academic journals: is there still a gap between social and natural sciences?," HSE Working papers WP BRP 28/HUM/2013, National Research University Higher School of Economics.
    19. Anne-Wil Harzing, 2013. "A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1057-1075, March.
    20. Daniel Torres-Salinas & Wenceslao Arroyo-Machado & Mike Thelwall, 2021. "Exploring WorldCat identities as an altmetric information source: a library catalog analysis experiment in the field of Scientometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1725-1743, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jscscx:v:2:y:2013:i:4:p:284-297:d:31000. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.