IDEAS home Printed from https://ideas.repec.org/a/oup/scippl/v51y2024i3p337-351..html
   My bibliography  Save this article

Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure

Author

Listed:
  • Tony Ross-Hellauer
  • Thomas Klebel
  • Petr Knoth
  • Nancy Pontika

Abstract

There are currently broad moves to reform research assessment, especially to better incentivize open and responsible research and avoid problematic use of inappropriate quantitative indicators. This study adds to the evidence base for such decision-making by investigating researcher perceptions of current processes of research assessment in institutional review, promotion, and tenure processes. Analysis of an international survey of 198 respondents reveals a disjunct between personal beliefs and perceived institutional priorities (‘value dissonance’), with practices of open and responsible research, as well as ‘research citizenship’ comparatively poorly valued by institutions at present. Our findings hence support current moves to reform research assessment. But we also add crucial nuance to the debate by discussing the relative weighting of open and responsible practices and suggesting that fostering research citizenship activities like collegiality and mentorship may be an important way to rebalance criteria towards environments, which better foster quality, openness, and responsibility.

Suggested Citation

  • Tony Ross-Hellauer & Thomas Klebel & Petr Knoth & Nancy Pontika, 2024. "Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure," Science and Public Policy, Oxford University Press, vol. 51(3), pages 337-351.
    • Handle: RePEc:oup:scippl:v:51:y:2024:i:3:p:337-351.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/scipol/scad073
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Farrow, Katherine & Grolleau, Gilles & Ibanez, Lisette, 2017. "Social Norms and Pro-environmental Behavior: A Review of the Evidence," Ecological Economics, Elsevier, vol. 140(C), pages 1-13.
    2. Meredith T Niles & Lesley A Schimanski & Erin C McKiernan & Juan Pablo Alperin, 2020. "Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-15, March.
    3. Hendrik P. van Dalen & Kène Henkens, 2012. "Intended and unintended consequences of a publish-or-perish culture: A worldwide survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1282-1293, July.
    4. Jelke Bethlehem, 2010. "Selection Bias in Web Surveys," International Statistical Review, International Statistical Institute, vol. 78(2), pages 161-188, August.
    5. Alexander Cotte Poveda & Clara Pardo Martinez, 2018. "Knowledge and Perceptions of Open Science among Researchers—A Case Study for Colombia," Serie de Documentos en Economía y Violencia 16971, Centro de Investigaciones en Violencia, Instituciones y Desarrollo Económico (VIDE).
    6. Björn Hammarfelt & Gaby Haddow, 2018. "Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 924-935, July.
    7. repec:bla:jamist:v:63:y:2012:i:7:p:1282-1293 is not listed on IDEAS
    8. Ted I. K. Youn & Tanya M. Price, 2009. "Learning from the Experience of Others: The Evolution of Faculty Tenure and Promotion Rules in Comprehensive Institutions," The Journal of Higher Education, Taylor & Francis Journals, vol. 80(2), pages 204-237, March.
    9. Ruth Müller & Sarah de Rijcke, 2017. "Thinking with Indicators. Exploring the Epistemic Impacts of Academic Performance Indicators in the Life Sciences," Research Evaluation, Oxford University Press, vol. 26(4), pages 361-361.
    10. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    11. Malsch, Bertrand & Tessier, Sophie, 2015. "Journal ranking effects on junior academics: Identity fragmentation and politicization," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 26(C), pages 84-98.
    12. David Moher & Lex Bouter & Sabine Kleinert & Paul Glasziou & Mai Har Sham & Virginia Barbour & Anne-Marie Coriat & Nicole Foeger & Ulrich Dirnagl, 2020. "The Hong Kong Principles for assessing researchers: Fostering research integrity," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-14, July.
    13. Joeri K Tijdink & Anton C M Vergouwen & Yvo M Smulders, 2013. "Publication Pressure and Burn Out among Dutch Medical Professors: A Nationwide Survey," PLOS ONE, Public Library of Science, vol. 8(9), pages 1-6, September.
    14. Lucas Brunet & Ruth Müller, 2022. "Making the cut: How panel reviewers use evaluation devices to select applications at the European Research Council," Research Evaluation, Oxford University Press, vol. 31(4), pages 486-497.
    15. Gaby Haddow & Björn Hammarfelt, 2019. "Quality, impact, and quantification: Indicators and metrics use by social scientists," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(1), pages 16-26, January.
    16. Don Haviland & Nathan F. Alleman & Cara Cliburn Allen, 2017. "‘Separate but Not Quite Equal’: Collegiality Experiences of Full-Time Non-Tenure-Track Faculty Members," The Journal of Higher Education, Taylor & Francis Journals, vol. 88(4), pages 505-528, July.
    17. Diane (DeDe) Dawson & Esteban Morales & Erin C McKiernan & Lesley A Schimanski & Meredith T Niles & Juan Pablo Alperin, 2022. "The role of collegiality in academic review, promotion, and tenure," PLOS ONE, Public Library of Science, vol. 17(4), pages 1-17, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    2. Daniele Fanelli & Vincent Larivière, 2016. "Researchers’ Individual Publication Rate Has Not Increased in a Century," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-12, March.
    3. van Dalen, Hendrik Peter, 2020. "How the Publish-or-Perish Principle Divides a Science : The Case of Academic Economists," Discussion Paper 2020-020, Tilburg University, Center for Economic Research.
    4. Diane (DeDe) Dawson & Esteban Morales & Erin C McKiernan & Lesley A Schimanski & Meredith T Niles & Juan Pablo Alperin, 2022. "The role of collegiality in academic review, promotion, and tenure," PLOS ONE, Public Library of Science, vol. 17(4), pages 1-17, April.
    5. Michaela Strinzel & Josh Brown & Wolfgang Kaltenbrunner & Sarah Rijcke & Michael Hill, 2021. "Ten ways to improve academic CVs for fairer research assessment," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-4, December.
    6. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.
    7. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    8. Cruz-Castro, Laura & Sanz-Menendez, Luis, 2021. "What should be rewarded? Gender and evaluation criteria for tenure and promotion," Journal of Informetrics, Elsevier, vol. 15(3).
    9. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    10. Nicky Agate & Rebecca Kennison & Stacy Konkiel & Christopher P. Long & Jason Rhody & Simone Sacchi & Penelope Weber, 2020. "The transformative power of values-enacted scholarship," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-12, December.
    11. Ginevra Peruginelli & Janne Pölönen, 2024. "The legal foundation of responsible research assessment: An overview on European Union and Italy," Research Evaluation, Oxford University Press, vol. 32(4), pages 670-682.
    12. Bornmann, Lutz & Ganser, Christian & Tekles, Alexander, 2022. "Simulation of the h index use at university departments within the bibliometrics-based heuristics framework: Can the indicator be used to compare individual researchers?," Journal of Informetrics, Elsevier, vol. 16(1).
    13. Meredith T Niles & Lesley A Schimanski & Erin C McKiernan & Juan Pablo Alperin, 2020. "Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-15, March.
    14. Daniella Bayle Deutz & Thea Marie Drachen & Dorte Drongstrup & Niels Opstrup & Charlotte Wien, 2021. "Quantitative quality: a study on how performance-based measures may change the publication patterns of Danish researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3303-3320, April.
    15. Alberto Baccini & Giuseppe De Nicolao & Eugenio Petrovich, 2019. "Citation gaming induced by bibliometric evaluation: A country-level comparative analysis," PLOS ONE, Public Library of Science, vol. 14(9), pages 1-16, September.
    16. Alexander Schniedermann, 2021. "A comparison of systematic reviews and guideline-based systematic reviews in medical studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9829-9846, December.
    17. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
    18. Julia Heuritsch, 2021. "Reflexive Behaviour: How Publication Pressure Affects Research Quality in Astronomy," Publications, MDPI, vol. 9(4), pages 1-23, November.
    19. Julian Hamann & Wolfgang Kaltenbrunner, 2022. "Biographical representation, from narrative to list: The evolution of curricula vitae in the humanities, 1950 to 2010," Research Evaluation, Oxford University Press, vol. 31(4), pages 438-451.
    20. Christian Schneijderberg & Nicolai Götze & Lars Müller, 2022. "A study of 25 years of publication outputs in the German academic profession," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 1-28, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:scippl:v:51:y:2024:i:3:p:337-351.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/spp .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.