IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i4d10.1007_s11192-021-03881-7.html
   My bibliography  Save this article

Quantitative quality: a study on how performance-based measures may change the publication patterns of Danish researchers

Author

Listed:
  • Daniella Bayle Deutz

    (University Library, University of Southern Denmark)

  • Thea Marie Drachen

    (University Library, University of Southern Denmark)

  • Dorte Drongstrup

    (University Library, University of Southern Denmark)

  • Niels Opstrup

    (University of Southern Denmark)

  • Charlotte Wien

    (University Library, University of Southern Denmark)

Abstract

Nations the world over are increasingly turning to quantitative performance-based metrics to evaluate the quality of research outputs, as these metrics are abundant and provide an easy measure of ranking research. In 2010, the Danish Ministry of Science and Higher Education followed this trend and began portioning out a percentage of the available research funding according to how many research outputs each Danish university produces. Not all research outputs are eligible: only those published in a curated list of academic journals and publishers, the so-called BFI list, are included. The BFI list is ranked, which may create incentives for academic authors to target certain publication outlets or publication types over others. In this study we examine the potential effect these relatively new research evaluation methods have had on the publication patterns of researchers in Denmark. The study finds that publication behaviors in the Natural Sciences & Technology, Social Sciences and Humanities (SSH) have changed, while the Health Sciences appear unaffected. Researchers in Natural Sciences & Technology appear to focus on high impact journals that reap more BFI points. While researchers in SSH have also increased their focus on the impact of the publication outlet, they also appear to have altered their preferred publication types, publishing more journal articles in the Social Sciences and more anthologies in the Humanities.

Suggested Citation

  • Daniella Bayle Deutz & Thea Marie Drachen & Dorte Drongstrup & Niels Opstrup & Charlotte Wien, 2021. "Quantitative quality: a study on how performance-based measures may change the publication patterns of Danish researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3303-3320, April.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:4:d:10.1007_s11192-021-03881-7
    DOI: 10.1007/s11192-021-03881-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-03881-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-03881-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Björn Hammarfelt & Gaby Haddow, 2018. "Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 924-935, July.
    2. Kaare Aagaard & Carter Bloch & Jesper W. Schneider, 2015. "Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator," Research Evaluation, Oxford University Press, vol. 24(2), pages 106-117.
    3. Lanteri,Alessandro & Vromen,Jack (ed.), 2014. "The Economics of Economists," Cambridge Books, Cambridge University Press, number 9781107015708, September.
    4. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    5. Daniele Fanelli & Vincent Larivière, 2016. "Researchers’ Individual Publication Rate Has Not Increased in a Century," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-12, March.
    6. Rómulo Pinheiro & Bjørn Stensaker, 2014. "Designing the Entrepreneurial University: The Interpretation of a Global Idea," Public Organization Review, Springer, vol. 14(4), pages 497-516, December.
    7. Jonas Krog Lind, 2019. "The missing link: How university managers mediate the impact of a performance-based research funding system," Research Evaluation, Oxford University Press, vol. 28(1), pages 84-93.
    8. Charlotte Wien & Bertil F. Dorch & Asger Væring Larsen, 2017. "Contradicting incentives for research collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 903-915, August.
    9. Kaare Aagaard, 2015. "How incentives trickle down: Local use of a national bibliometric indicator system," Science and Public Policy, Oxford University Press, vol. 42(5), pages 725-737.
    10. Daniella B Deutz & Evgenios Vlachos & Dorte Drongstrup & Bertil F Dorch & Charlotte Wien, 2020. "Effective publication strategies in clinical research," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-12, January.
    11. Katharine Barker, 2007. "The UK Research Assessment Exercise: the evolution of a national research evaluation system," Research Evaluation, Oxford University Press, vol. 16(1), pages 3-12, March.
    12. Truyken L. B. Ossenblok & Tim C. E. Engels & Gunnar Sivertsen, 2012. "The representation of the social sciences and humanities in the Web of Science--a comparison of publication patterns and incentive structures in Flanders and Norway (2005--9)," Research Evaluation, Oxford University Press, vol. 21(4), pages 280-290, September.
    13. Koen Jonkers & Thomas Zacharewicz, 2016. "Research Performance Based Funding Systems: a Comparative Assessment," JRC Research Reports JRC101043, Joint Research Centre.
    14. Gaby Haddow & Björn Hammarfelt, 2019. "Quality, impact, and quantification: Indicators and metrics use by social scientists," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(1), pages 16-26, January.
    15. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    16. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laura Himanen & Hanna-Mari Puuska, 2022. "Does monitoring performance act as an incentive for improving research performance? National and organizational level analysis of Finnish universities," Research Evaluation, Oxford University Press, vol. 31(2), pages 236-248.
    2. Linda Sīle & Raf Vanderstraeten, 2019. "Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 71-91, January.
    3. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    4. Christian Schneijderberg & Nicolai Götze & Lars Müller, 2022. "A study of 25 years of publication outputs in the German academic profession," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 1-28, January.
    5. Robert A. Buckle & John Creedy, 2022. "Methods to evaluate institutional responses to performance‐based research funding systems," Australian Economic Papers, Wiley Blackwell, vol. 61(3), pages 615-634, September.
    6. Kulczycki, Emanuel & Korzeń, Marcin & Korytkowski, Przemysław, 2017. "Toward an excellence-based research funding system: Evidence from Poland," Journal of Informetrics, Elsevier, vol. 11(1), pages 282-298.
    7. Renata Kudaibergenova & Sandugash Uzakbay & Asselya Makanova & Kymbat Ramadinkyzy & Erlan Kistaubayev & Ruslan Dussekeev & Kadyrzhan Smagulov, 2022. "Managing publication change at Al-Farabi Kazakh National University: a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 453-479, January.
    8. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    9. Yaşar Tonta & Müge Akbulut, 2020. "Does monetary support increase citation impact of scholarly papers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1617-1641, November.
    10. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Di Costa, Flavia, 2019. "When research assessment exercises leave room for opportunistic behavior by the subjects under evaluation," Journal of Informetrics, Elsevier, vol. 13(3), pages 830-840.
    11. Marques, Marcelo, 2021. "How do policy instruments generate new ones? Analysing policy instruments feedback and interaction in educational research in England, 1986-2014," Research Policy, Elsevier, vol. 50(10).
    12. Elea Giménez-Toledo & Jorge Mañana-Rodríguez & Tim C. E. Engels & Peter Ingwersen & Janne Pölönen & Gunnar Sivertsen & Frederik T. Verleysen & Alesia A. Zuccala, 2016. "Taking scholarly books into account: current developments in five European countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 685-699, May.
    13. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    14. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.
    15. Yuret, Tolga, 2017. "Do researchers pay attention to publication subsidies?," Journal of Informetrics, Elsevier, vol. 11(2), pages 423-434.
    16. Dzieżyc, Maciej & Kazienko, Przemysław, 2022. "Effectiveness of research grants funded by European Research Council and Polish National Science Centre," Journal of Informetrics, Elsevier, vol. 16(1).
    17. Shibayama, Sotaro & Baba, Yasunori, 2015. "Impact-oriented science policies and scientific publication practices: The case of life sciences in Japan," Research Policy, Elsevier, vol. 44(4), pages 936-950.
    18. Buehling, Kilian, 2021. "Changing research topic trends as an effect of publication rankings – The case of German economists and the Handelsblatt Ranking," Journal of Informetrics, Elsevier, vol. 15(3).
    19. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 21104, Victoria University of Wellington, Chair in Public Finance.
    20. Banal-Estañol, Albert & Jofre-Bonet, Mireia & Iori, Giulia & Maynou, Laia & Tumminello, Michele & Vassallo, Pietro, 2023. "Performance-based research funding: Evidence from the largest natural experiment worldwide," Research Policy, Elsevier, vol. 52(6).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:4:d:10.1007_s11192-021-03881-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.