IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v22y2013i5p369-383.html
   My bibliography  Save this article

Criteria for assessing research quality in the humanities: a Delphi study among scholars of English literature, German literature and art history

Author

Listed:
  • Sven E. Hug
  • Michael Ochsner
  • Hans-Dieter Daniel

Abstract

How to assess research quality in the humanities is an intricate question. Despite the recent efforts of many initiatives, the measurement and assessment of research quality still faces strong opposition from humanities scholars, indicating that currently used evaluation schemes and tools are not tailored well enough to humanities disciplines. We have collected quality criteria from scholars in Switzerland and at League of European Research Universities (LERU) in the three disciplines, German literature studies, English literature studies and art history with a multi-round Delphi survey. The first Delphi round resulted in a comprehensive list of 19 criteria of good research specified by 70 aspects. Although 10 of these criteria are well known and commonly used in various evaluation schemes, nine criteria are not, or at least not frequently, employed in evaluation schemes. In the second Delphi round, consensual criteria and aspects (i.e. items that were clearly approved by a majority and disapproved by very few scholars) were identified in each discipline. Specifically, 11 criteria reached consensus in all three disciplines (shared criteria), six criteria were consensual in only one or two disciplines (discipline-specific criteria) and two criteria were not consensual in any discipline (i.e. 'productivity' and 'relation to and impact on society'). The results of this study corroborate previous findings that researchers have not yet adopted Mode 2-related assessment criteria. Implications for research assessment are being discussed. The focus lies in particular on the mismatch of criteria between evaluators and scholars as well as on an approach to bridge such a mismatch. Copyright The Author 2013. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com, Oxford University Press.

Suggested Citation

  • Sven E. Hug & Michael Ochsner & Hans-Dieter Daniel, 2013. "Criteria for assessing research quality in the humanities: a Delphi study among scholars of English literature, German literature and art history," Research Evaluation, Oxford University Press, vol. 22(5), pages 369-383, August.
    • Handle: RePEc:oup:rseval:v:22:y:2013:i:5:p:369-383
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvt008
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tausch, Arno, 2015. "Die Buchpublikationen der Nobelpreis-Ökonomen und die führenden Buchverlage der Disziplin. Eine bibliometrische Analyse [The book publications of the Nobel-Prize economists and the leading book pub," MPRA Paper 67224, University Library of Munich, Germany.
    2. Song Jing & Qingzhao Ma & Siyi Wang & Hanliang Xu & Tian Xu & Xia Guo & Zhuolin Wu, 2024. "Research on developmental evaluation based on the "four abilities" model: evidence from early career researchers in China," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(1), pages 681-704, February.
    3. Sven E. Hug & Mirjam Aeschbach, 2020. "Criteria for assessing grant applications: a systematic review," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-15, December.
    4. Alessandro Margherita & Gianluca Elia & Claudio Petti, 2022. "What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives," Sustainability, MDPI, vol. 14(5), pages 1-18, March.
    5. Tausch, Arno, 2018. "The Market Power of Global Scientific Publishing Companies in the Age of Globalization. An Analysis Based on the OCLC Worldcat," MPRA Paper 87442, University Library of Munich, Germany.
    6. Peter van den Besselaar & Ulf Sandström, 2019. "Measuring researcher independence using bibliometric data: A proposal for a new performance indicator," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-20, March.
    7. Verleysen, Frederik T. & Weeren, Arie, 2016. "Clustering by publication patterns of senior authors in the social sciences and humanities," Journal of Informetrics, Elsevier, vol. 10(1), pages 254-272.
    8. Koch, Susanne & Tetley, Camilla, 2023. "What ‘counts’ in international forest policy research? A conference ethnography of valuation practice and habitus in an interdisciplinary social science field," Forest Policy and Economics, Elsevier, vol. 154(C).
    9. Panagopoulos, George & Tsatsaronis, George & Varlamis, Iraklis, 2017. "Detecting rising stars in dynamic collaborative networks," Journal of Informetrics, Elsevier, vol. 11(1), pages 198-222.
    10. Zoe Bulaitis, 2017. "Measuring impact in the humanities: Learning from accountability and economics in a contemporary history of cultural value," Palgrave Communications, Palgrave Macmillan, vol. 3(1), pages 1-11, December.
    11. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    12. Sven Helmer & David B. Blumenthal & Kathrin Paschen, 2020. "What is meaningful research and how should we measure it?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 153-169, October.
    13. Kamila Lewandowska & Emanuel Kulczycki & Michael Ochsner, 2024. "Evaluation of the arts in performance-based research funding systems: An international perspective," Research Evaluation, Oxford University Press, vol. 32(1), pages 19-31.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:22:y:2013:i:5:p:369-383. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.