IDEAS home Printed from https://ideas.repec.org/a/gam/jscscx/v2y2013i4p284-297d31000.html
   My bibliography  Save this article

Why an Unbiased External R&D Evaluation System is Important for the Progress of Social Sciences—the Case of a Small Social Science Community

Author

Listed:
  • Franc Mali

    (Faculty of Social Sciences, University of Ljubljana, Kardeljeva pl. 5, Ljubljana 1000, Slovenia)

Abstract

This article deals with the impact of external R&D evaluations as one of the institutional factors that can encourage (or discourage) the progress of the social sciences. A critical overview is presented of the increasing use of bibliometric indicators in the external R&D evaluation procedures employed by the Slovenian Research Agency, which is the leading research council for financing the public sector of social sciences in Slovenia. We attempt to establish that, in order to ensure a good external R&D evaluation practice for a small social science community, it is insufficient to only have reliable bibliometric meta-databases. It is argued that it is equally important to formulate very precise criteria to ascertain their validity.

Suggested Citation

  • Franc Mali, 2013. "Why an Unbiased External R&D Evaluation System is Important for the Progress of Social Sciences—the Case of a Small Social Science Community," Social Sciences, MDPI, vol. 2(4), pages 1-14, December.
    • Handle: RePEc:gam:jscscx:v:2:y:2013:i:4:p:284-297:d:31000
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2076-0760/2/4/284/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2076-0760/2/4/284/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    2. Hess, David, 2007. "Social Reporting and New Governance Regulation: The Prospects of Achieving Corporate Accountability Through Transparency," Business Ethics Quarterly, Cambridge University Press, vol. 17(3), pages 453-476, July.
    3. Rémi Barré, 2010. "Towards socially robust S&T indicators: indicators as debatable devices, enabling collective learning," Research Evaluation, Oxford University Press, vol. 19(3), pages 227-231, September.
    4. Stefan Hornbostel, 2001. "Third party funding of German universities. An indicator of research activity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 50(3), pages 523-537, March.
    5. Erik Ernø-Kjølhede & Finn Hansson, 2011. "Measuring research performance during a changing relationship between science and society," Research Evaluation, Oxford University Press, vol. 20(2), pages 131-143, June.
    6. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    7. Mu‐hsuan Huang & Yu‐wei Chang, 2008. "Characteristics of research output in social sciences and humanities: From a research evaluation perspective," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1819-1828, September.
    8. Deborah Brautigam & Michael Woolcock, 2001. "Small States in a Global Economy: the Role of Institutions in Managing Vulnerability and Opportunity in Small Developing Countries," WIDER Working Paper Series DP2001-37, World Institute for Development Economic Research (UNU-WIDER).
    9. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    10. Torres-Salinas, Daniel & Moed, Henk F., 2009. "Library Catalog Analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in Economics," Journal of Informetrics, Elsevier, vol. 3(1), pages 9-26.
    11. Ulrich Schmoch & Torben Schubert & Dorothea Jansen & Richard Heidler & Regina von Görtz, 2010. "How to use indicators to measure scientific performance: a balanced approach," Research Evaluation, Oxford University Press, vol. 19(1), pages 2-18, March.
    12. Luka Kronegger & Franc Mali & Anuška Ferligoj & Patrick Doreian, 2012. "Collaboration structures in Slovenian scientific communities," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 631-647, February.
    13. David Pontille & Didier Torny, 2010. "The controversial policies of journal ratings: evaluating social sciences and humanities," Research Evaluation, Oxford University Press, vol. 19(5), pages 347-360, December.
    14. Kayvan Kousha & Mike Thelwall, 2009. "Google book search: Citation analysis for social science and the humanities," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(8), pages 1537-1549, August.
    15. Ed C. M. Noyons & Clara Calero-Medina, 2009. "Applying bibliometric mapping in a high level science policy context," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(2), pages 261-275, May.
    16. Benedetto Lepori & Carole Probst, 2009. "Using curricula vitae for mapping scientific fields: a small-scale experience for Swiss communication sciences," Research Evaluation, Oxford University Press, vol. 18(2), pages 125-134, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Romina Rodela, 2016. "On the use of databases about research performance: comments on Karlovčec and Mladenić (2015) and others using the SICRIS database," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2151-2157, December.
    2. Carlos Miguel Ferreira & Sandro Serpa, 2018. "Online Visibility, Social Networks and Glamorous Scientific Publications," International Journal of Social Science Studies, Redfame publishing, vol. 6(10), pages 58-66, October.
    3. Franc Mali & Toni Pustovrh & Rok Platinovšek & Luka Kronegger & Anuška Ferligoj, 2017. "The effects of funding and co-authorship on research performance in a small scientific community," Science and Public Policy, Oxford University Press, vol. 44(4), pages 486-496.
    4. Marjan Cugmas & Franc Mali & Aleš Žiberna, 2020. "Scientific collaboration of researchers and organizations: a two-level blockmodeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2471-2489, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ülle Must, 2012. "Alone or together: examples from history research," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 527-537, May.
    2. Daniel Torres-Salinas & Nicolás Robinson-Garcia & Juan Gorraiz, 2017. "Filling the citation gap: measuring the multidimensional impact of the academic book at institutional level with PlumX," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1371-1384, December.
    3. Boyack, Kevin W. & Klavans, Richard, 2014. "Including cited non-source items in a large-scale map of science: What difference does it make?," Journal of Informetrics, Elsevier, vol. 8(3), pages 569-580.
    4. Andrea Bonaccorsi & Cinzia Daraio & Stefano Fantoni & Viola Folli & Marco Leonetti & Giancarlo Ruocco, 2017. "Do social sciences and humanities behave like life and hard sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 607-653, July.
    5. Ashraf Maleki, 2022. "Why does library holding format really matter for book impact assessment?: Modelling the relationship between citations and altmetrics with print and electronic holdings," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 1129-1160, February.
    6. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    7. Maja Jokić & Andrea Mervar & Stjepan Mateljan, 2019. "Comparative analysis of book citations in social science journals by Central and Eastern European authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1005-1029, September.
    8. Guy Madison & Knut Sundell, 2022. "Numbers of publications and citations for researchers in fields pertinent to the social services: a comparison of peer-reviewed journal publications across six disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 6029-6046, October.
    9. Yongjun Zhu & Erjia Yan & Silvio Peroni & Chao Che, 2020. "Nine million book items and eleven million citations: a study of book-based scholarly communication using OpenCitations," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1097-1112, February.
    10. Kousha, Kayvan & Thelwall, Mike, 2018. "Can Microsoft Academic help to assess the citation impact of academic books?," Journal of Informetrics, Elsevier, vol. 12(3), pages 972-984.
    11. Alesia Zuccala & Thed van Leeuwen, 2011. "Book reviews in humanities research evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(10), pages 1979-1991, October.
    12. Alesia Zuccala & Roberto Cornacchia, 2016. "Data matching, integration, and interoperability for a metric assessment of monographs," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 465-484, July.
    13. Paul Benneworth, 2015. "Between certainty and comprehensiveness in evaluating the societal impact of humanities research," CHEPS Working Papers 201502, University of Twente, Center for Higher Education Policy Studies (CHEPS).
    14. Mohammadamin Erfanmanesh & A. Noorhidawati & A. Abrizah, 2019. "What can Bookmetrix tell us about the impact of Springer Nature’s books," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 521-536, October.
    15. Andrea Mervar & Maja Jokić, 2022. "Core-periphery nexus in the EU social sciences: bibliometric perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5793-5817, October.
    16. Siluo Yang & Xin Xing & Fan Qi & Maria Cláudia Cabrini Grácio, 2021. "Comparison of academic book impact from a disciplinary perspective: an analysis of citations and altmetric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1101-1123, February.
    17. Ekaterina L. Dyachenko, 2014. "Internationalization of academic journals: Is there still a gap between social and natural sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 241-255, October.
    18. Katarina Rojko & Brankica Bratić & Borut Lužar, 2020. "The Bologna reform’s impacts on the scientific publication performance of Ph.D. graduates—the case of Slovenia," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 329-356, July.
    19. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.
    20. Thelwall, Mike & Sud, Pardeep, 2014. "No citation advantage for monograph-based collaborations?," Journal of Informetrics, Elsevier, vol. 8(1), pages 276-283.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jscscx:v:2:y:2013:i:4:p:284-297:d:31000. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.