IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v45y2016i3p593-603.html
   My bibliography  Save this article

Evaluating research: A multidisciplinary approach to assessing research practice and quality

Author

Listed:
  • Mårtensson, Pär
  • Fors, Uno
  • Wallin, Sven-Bertil
  • Zander, Udo
  • Nilsson, Gunnar H

Abstract

There are few widely acknowledged quality standards for research practice, and few definitions of what constitutes good research. The overall aim was therefore to describe what constitutes research, and then to use this description to develop a model of research practice and to define concepts related to its quality. The primary objective was to explore such a model and to create a multidisciplinary understanding of the generic dimensions of the quality of research practice. Eight concept modelling working seminars were conducted. A graphic representation of concepts and their relationships was developed to bridge the gap between different disciplines. A concept model of research as a phenomenon was created, which included a total of 18 defined concepts and their relationships. In a second phase four main areas were distilled, describing research practice in a multidisciplinary context: Credible, Contributory, Communicable, and Conforming. Each of these was further specified in a concept hierarchy together with a defined terminology. A comprehensive quality model including 32 concepts, based on the four main areas, was developed for describing quality issues of research practice, where the model of research as a phenomenon was used to define the quality concepts. The quality model may be used for further development of elements, weights and operationalizations related to the quality of research practice in different academic fields.

Suggested Citation

  • Mårtensson, Pär & Fors, Uno & Wallin, Sven-Bertil & Zander, Udo & Nilsson, Gunnar H, 2016. "Evaluating research: A multidisciplinary approach to assessing research practice and quality," Research Policy, Elsevier, vol. 45(3), pages 593-603.
    • Handle: RePEc:eee:respol:v:45:y:2016:i:3:p:593-603
    DOI: 10.1016/j.respol.2015.11.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733315001845
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2015.11.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hessels, Laurens K. & van Lente, Harro, 2008. "Re-thinking new knowledge production: A literature review and a research agenda," Research Policy, Elsevier, vol. 37(4), pages 740-760, May.
    2. Allen S. Lee & Richard L. Baskerville, 2003. "Generalizing Generalizability in Information Systems Research," Information Systems Research, INFORMS, vol. 14(3), pages 221-243, September.
    3. Laursen, Keld & Salter, Ammon, 2004. "Searching high and low: what types of firms use universities as a source of innovation?," Research Policy, Elsevier, vol. 33(8), pages 1201-1215, October.
    4. Sutherland, H. J. & Meslin, E. M. & da Cunha, R. & Till, J. E., 1993. "Judging clinical research questions: What criteria are used?," Social Science & Medicine, Elsevier, vol. 37(12), pages 1427-1430, December.
    5. Amin, Ash & Roberts, Joanne, 2008. "Knowing in action: Beyond communities of practice," Research Policy, Elsevier, vol. 37(2), pages 353-369, March.
    6. Laurens K. Hessels & Harro van Lente, 2008. "Re-thinking knowledge production: a literature review and a research agenda," Innovation Studies Utrecht (ISU) working paper series 08-03, Utrecht University, Department of Innovation Studies, revised Feb 2008.
    7. Daniel Robey & M. Lynne Markus, 1998. "Beyond Rigor and Relevance: Producing Consumable Research about Information Systems," Information Resources Management Journal (IRMJ), IGI Global, vol. 11(1), pages 7-16, January.
    8. Pierre Berthon & Leyland Pitt & Michael Ewing & Christopher L. Carr, 2002. "Potential Research Space in MIS: A Framework for Envisioning and Evaluating Research Replication, Extension, and Generation," Information Systems Research, INFORMS, vol. 13(4), pages 416-427, December.
    9. Tsao, J.Y. & Boyack, K.W. & Coltrin, M.E. & Turnley, J.G. & Gauster, W.B., 2008. "Galileo's stream: A framework for understanding knowledge production," Research Policy, Elsevier, vol. 37(2), pages 330-352, March.
    10. Shewchuk, Richard M. & O'Connor, Stephen J. & Williams, Eric S. & Savage, Grant T., 2006. "Beyond rankings: Using cognitive mapping to understand what health care journals represent," Social Science & Medicine, Elsevier, vol. 62(5), pages 1192-1204, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
    2. Sven E. Hug & Mirjam Aeschbach, 2020. "Criteria for assessing grant applications: a systematic review," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-15, December.
    3. Rau, Henrike & Goggins, Gary & Fahy, Frances, 2018. "From invisibility to impact: Recognising the scientific and societal relevance of interdisciplinary sustainability research," Research Policy, Elsevier, vol. 47(1), pages 266-276.
    4. Alessandro Margherita & Gianluca Elia & Claudio Petti, 2022. "What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives," Sustainability, MDPI, vol. 14(5), pages 1-18, March.
    5. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    6. Sofie Sandin & Mats Benner, 2022. "Research evaluations for an energy transition? Insights from a review of Swedish research evaluation reports," Research Evaluation, Oxford University Press, vol. 31(1), pages 80-92.
    7. Sven Helmer & David B. Blumenthal & Kathrin Paschen, 2020. "What is meaningful research and how should we measure it?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 153-169, October.
    8. Pär Mårtensson & Uno Fors & Emelie Fröberg & Udo Zander & Gunnar H Nilsson, 2019. "Quality of Research Practice – An interdisciplinary face validity evaluation of a quality model," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-19, February.
    9. Jefferson Seide Molléri & Kai Petersen & Emilia Mendes, 2018. "Towards understanding the relation between citations and research quality in software engineering studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1453-1478, December.
    10. Jennifer Leigh & Nicole Brown, 2021. "Researcher experiences in practice-based interdisciplinary research [Imagining Autism: Feasibility and Impact of a Drama-Based Intervention on the Social Communicative and Imaginative Behaviour of ," Research Evaluation, Oxford University Press, vol. 30(4), pages 421-430.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pär Mårtensson & Uno Fors & Emelie Fröberg & Udo Zander & Gunnar H Nilsson, 2019. "Quality of Research Practice – An interdisciplinary face validity evaluation of a quality model," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-19, February.
    2. van Rijnsoever, Frank J. & Hessels, Laurens K. & Vandeberg, Rens L.J., 2008. "A resource-based view on the interactions of university researchers," Research Policy, Elsevier, vol. 37(8), pages 1255-1266, September.
    3. Julia Olmos‐Peñuela & Paul Benneworth & Elena Castro‐Martínez, 2015. "Exploring the factors related with scientists’ willingness to incorporating external knowledge," CHEPS Working Papers 201504, University of Twente, Center for Higher Education Policy Studies (CHEPS).
    4. Andreas Bjurström & Merritt Polk, 2011. "Climate change and interdisciplinarity: a co-citation analysis of IPCC Third Assessment Report," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 525-550, June.
    5. van Rijnsoever, Frank J. & Hessels, Laurens K., 2011. "Factors associated with disciplinary and interdisciplinary research collaboration," Research Policy, Elsevier, vol. 40(3), pages 463-472, April.
    6. Llopis, Oscar & D'Este, Pablo & McKelvey, Maureen & Yegros, Alfredo, 2022. "Navigating multiple logics: Legitimacy and the quest for societal impact in science," Technovation, Elsevier, vol. 110(C).
    7. Froese, Anna & Woiwode, Hendrik & Suckow, Silvio, 2019. "Mission Impossible? Neue Wege zu Interdisziplinarität: Empfehlungen für Wissenschaft, Wissenschaftspolitik und Praxis," Discussion Papers, Research Group Science Policy Studies SP III 2019-601, WZB Berlin Social Science Center.
    8. Niels Stijn & Frank J. Rijnsoever & Martine Veelen, 2018. "Exploring the motives and practices of university–start-up interaction: evidence from Route 128," The Journal of Technology Transfer, Springer, vol. 43(3), pages 674-713, June.
    9. Dorothea Jansen & Regina Görtz & Richard Heidler, 2010. "Knowledge production and the structure of collaboration networks in two scientific fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 219-241, April.
    10. Hubeau, Marianne & Marchand, Fleur & Coteur, Ine & Mondelaers, Koen & Debruyne, Lies & Van Huylenbroeck, Guido, 2017. "A new agri-food systems sustainability approach to identify shared transformation pathways towards sustainability," Ecological Economics, Elsevier, vol. 131(C), pages 52-63.
    11. Mari Jose Aranguren & James Karlsen & Miren Larrea & James R. Wilson, 2013. "The development of action research processes and their impacts on socio-economic development in the Basque Country," Chapters, in: Roger Sugden & Marcela Valania & James R. Wilson (ed.), Leadership and Cooperation in Academia, chapter 14, pages 216-233, Edward Elgar Publishing.
    12. Deborah Compeau & Barbara Marcolin & Helen Kelley & Chris Higgins, 2012. "Research Commentary ---Generalizability of Information Systems Research Using Student Subjects---A Reflection on Our Practices and Recommendations for Future Research," Information Systems Research, INFORMS, vol. 23(4), pages 1093-1109, December.
    13. Yoguel, Gabriel & Brixner, Cristián & Lerena, Octavio & Minervini, Mariana, 2021. "The relationship between universities and business: identification of thematic communities," Revista CEPAL, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL), December.
    14. Michaela Trippl & Tanja Sinozic & Helen Lawton Smith, 2015. "The Role of Universities in Regional Development: Conceptual Models and Policy Institutions in the UK, Sweden and Austria," European Planning Studies, Taylor & Francis Journals, vol. 23(9), pages 1722-1740, September.
    15. James Cunningham & Paul O’Reilly & Conor O’Kane & Vincent Mangematin, 2014. "The inhibiting factors that principal investigators experience in leading publicly funded research," The Journal of Technology Transfer, Springer, vol. 39(1), pages 93-110, February.
    16. Atiase, Victor Yawo & Kolade, Oluwaseun & Liedong, Tahiru Azaaviele, 2020. "The emergence and strategy of tech hubs in Africa: Implications for knowledge production and value creation," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    17. Fisher, Erik, 2019. "Governing with ambivalence: The tentative origins of socio-technical integration," Research Policy, Elsevier, vol. 48(5), pages 1138-1149.
    18. Srinivasan, M.S. & Jongmans, C. & Bewsell, D. & Elley, G., 2019. "Research idea to science for impact: Tracing the significant moments in an innovation based irrigation study," Agricultural Water Management, Elsevier, vol. 212(C), pages 181-192.
    19. Ismael Rafols & Alan Porter & Loet Leydesdorff, 2009. "Overlay Maps of Science: a New Tool for Research Policy," SPRU Working Paper Series 179, SPRU - Science Policy Research Unit, University of Sussex Business School.
    20. Rodríguez, Hannot & Fisher, Erik & Schuurbiers, Daan, 2013. "Integrating science and society in European Framework Programmes: Trends in project-level solicitations," Research Policy, Elsevier, vol. 42(5), pages 1126-1137.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:45:y:2016:i:3:p:593-603. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.