IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v45y2016i3p593-603.html
   My bibliography  Save this article

Evaluating research: A multidisciplinary approach to assessing research practice and quality

Author

Listed:
  • Mårtensson, Pär
  • Fors, Uno
  • Wallin, Sven-Bertil
  • Zander, Udo
  • Nilsson, Gunnar H

Abstract

There are few widely acknowledged quality standards for research practice, and few definitions of what constitutes good research. The overall aim was therefore to describe what constitutes research, and then to use this description to develop a model of research practice and to define concepts related to its quality. The primary objective was to explore such a model and to create a multidisciplinary understanding of the generic dimensions of the quality of research practice. Eight concept modelling working seminars were conducted. A graphic representation of concepts and their relationships was developed to bridge the gap between different disciplines. A concept model of research as a phenomenon was created, which included a total of 18 defined concepts and their relationships. In a second phase four main areas were distilled, describing research practice in a multidisciplinary context: Credible, Contributory, Communicable, and Conforming. Each of these was further specified in a concept hierarchy together with a defined terminology. A comprehensive quality model including 32 concepts, based on the four main areas, was developed for describing quality issues of research practice, where the model of research as a phenomenon was used to define the quality concepts. The quality model may be used for further development of elements, weights and operationalizations related to the quality of research practice in different academic fields.

Suggested Citation

  • Mårtensson, Pär & Fors, Uno & Wallin, Sven-Bertil & Zander, Udo & Nilsson, Gunnar H, 2016. "Evaluating research: A multidisciplinary approach to assessing research practice and quality," Research Policy, Elsevier, vol. 45(3), pages 593-603.
    • Handle: RePEc:eee:respol:v:45:y:2016:i:3:p:593-603
    DOI: 10.1016/j.respol.2015.11.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733315001845
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2015.11.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hessels, Laurens K. & van Lente, Harro, 2008. "Re-thinking new knowledge production: A literature review and a research agenda," Research Policy, Elsevier, vol. 37(4), pages 740-760, May.
    2. Allen S. Lee & Richard L. Baskerville, 2003. "Generalizing Generalizability in Information Systems Research," Information Systems Research, INFORMS, vol. 14(3), pages 221-243, September.
    3. Daniel Robey & M. Lynne Markus, 1998. "Beyond Rigor and Relevance: Producing Consumable Research about Information Systems," Information Resources Management Journal (IRMJ), IGI Global, vol. 11(1), pages 7-16, January.
    4. Pierre Berthon & Leyland Pitt & Michael Ewing & Christopher L. Carr, 2002. "Potential Research Space in MIS: A Framework for Envisioning and Evaluating Research Replication, Extension, and Generation," Information Systems Research, INFORMS, vol. 13(4), pages 416-427, December.
    5. Laursen, Keld & Salter, Ammon, 2004. "Searching high and low: what types of firms use universities as a source of innovation?," Research Policy, Elsevier, vol. 33(8), pages 1201-1215, October.
    6. Tsao, J.Y. & Boyack, K.W. & Coltrin, M.E. & Turnley, J.G. & Gauster, W.B., 2008. "Galileo's stream: A framework for understanding knowledge production," Research Policy, Elsevier, vol. 37(2), pages 330-352, March.
    7. Sutherland, H. J. & Meslin, E. M. & da Cunha, R. & Till, J. E., 1993. "Judging clinical research questions: What criteria are used?," Social Science & Medicine, Elsevier, vol. 37(12), pages 1427-1430, December.
    8. Amin, Ash & Roberts, Joanne, 2008. "Knowing in action: Beyond communities of practice," Research Policy, Elsevier, vol. 37(2), pages 353-369, March.
    9. Laurens K. Hessels & Harro van Lente, 2008. "Re-thinking knowledge production: a literature review and a research agenda," Innovation Studies Utrecht (ISU) working paper series 08-03, Utrecht University, Department of Innovation Studies, revised Feb 2008.
    10. Shewchuk, Richard M. & O'Connor, Stephen J. & Williams, Eric S. & Savage, Grant T., 2006. "Beyond rankings: Using cognitive mapping to understand what health care journals represent," Social Science & Medicine, Elsevier, vol. 62(5), pages 1192-1204, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
    2. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    3. Sofie Sandin & Mats Benner, 2022. "Research evaluations for an energy transition? Insights from a review of Swedish research evaluation reports," Research Evaluation, Oxford University Press, vol. 31(1), pages 80-92.
    4. Sven Helmer & David B. Blumenthal & Kathrin Paschen, 2020. "What is meaningful research and how should we measure it?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 153-169, October.
    5. Pär Mårtensson & Uno Fors & Emelie Fröberg & Udo Zander & Gunnar H Nilsson, 2019. "Quality of Research Practice – An interdisciplinary face validity evaluation of a quality model," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-19, February.
    6. Jefferson Seide Molléri & Kai Petersen & Emilia Mendes, 2018. "Towards understanding the relation between citations and research quality in software engineering studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1453-1478, December.
    7. Sven E. Hug & Mirjam Aeschbach, 2020. "Criteria for assessing grant applications: a systematic review," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-15, December.
    8. Rau, Henrike & Goggins, Gary & Fahy, Frances, 2018. "From invisibility to impact: Recognising the scientific and societal relevance of interdisciplinary sustainability research," Research Policy, Elsevier, vol. 47(1), pages 266-276.
    9. Alessandro Margherita & Gianluca Elia & Claudio Petti, 2022. "What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives," Sustainability, MDPI, vol. 14(5), pages 1-18, March.
    10. Jennifer Leigh & Nicole Brown, 2021. "Researcher experiences in practice-based interdisciplinary research [Imagining Autism: Feasibility and Impact of a Drama-Based Intervention on the Social Communicative and Imaginative Behaviour of ," Research Evaluation, Oxford University Press, vol. 30(4), pages 421-430.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pär Mårtensson & Uno Fors & Emelie Fröberg & Udo Zander & Gunnar H Nilsson, 2019. "Quality of Research Practice – An interdisciplinary face validity evaluation of a quality model," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-19, February.
    2. van Rijnsoever, Frank J. & Hessels, Laurens K. & Vandeberg, Rens L.J., 2008. "A resource-based view on the interactions of university researchers," Research Policy, Elsevier, vol. 37(8), pages 1255-1266, September.
    3. Julia Olmos‐Peñuela & Paul Benneworth & Elena Castro‐Martínez, 2015. "Exploring the factors related with scientists’ willingness to incorporating external knowledge," CHEPS Working Papers 201504, University of Twente, Center for Higher Education Policy Studies (CHEPS).
    4. A. E. Rodríguez Salazar & M. A. Domínguez-Crespo & A. M. Torres-Huerta & A. I. Licona-Aguilar & A. Nivón-Pellón & V. N. Orta-Guzmán, 2021. "Analysis of the Dynamical Capabilities into the Public Research Institutes to Their Strategic Decision-Making," Sustainability, MDPI, vol. 13(12), pages 1-22, June.
    5. Andreas Bjurström & Merritt Polk, 2011. "Climate change and interdisciplinarity: a co-citation analysis of IPCC Third Assessment Report," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 525-550, June.
    6. David B. Audretsch & Albert N. Link & Martijn Hasselt, 2019. "Knowledge begets knowledge: university knowledge spillovers and the output of scientific papers from U.S. Small Business Innovation Research (SBIR) projects," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1367-1383, December.
    7. König, Jonas & Suwala, Lech & Delargy, Colin, 2020. "Helix Models of Innovation and Sustainable Development Goals," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 1-15.
    8. van Rijnsoever, Frank J. & Hessels, Laurens K., 2011. "Factors associated with disciplinary and interdisciplinary research collaboration," Research Policy, Elsevier, vol. 40(3), pages 463-472, April.
    9. Llopis, Oscar & D'Este, Pablo & McKelvey, Maureen & Yegros, Alfredo, 2022. "Navigating multiple logics: Legitimacy and the quest for societal impact in science," Technovation, Elsevier, vol. 110(C).
    10. Stefano Denicolai & Antonella Zucchella & Federico Moretti, 2018. "Not So Similar After All: Exploring The Diversity Of Strategic Orientations For Innovation," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 22(04), pages 1-33, May.
    11. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    12. Bianca Vienni-Baptista & Isabel Fletcher & Catherine Lyall & Christian Pohl, 2022. "Embracing heterogeneity: Why plural understandings strengthen interdisciplinarity and transdisciplinarity [Defining Interdisciplinary Research: Conclusions from a Critical Review of the Literature]," Science and Public Policy, Oxford University Press, vol. 49(6), pages 865-877.
    13. Froese, Anna & Woiwode, Hendrik & Suckow, Silvio, 2019. "Mission Impossible? Neue Wege zu Interdisziplinarität: Empfehlungen für Wissenschaft, Wissenschaftspolitik und Praxis," Discussion Papers, Research Group Science Policy Studies SP III 2019-601, WZB Berlin Social Science Center.
    14. Cristiano Codagnone & Federico Biagi & Fabienne Abadie, 2016. "The Passions and the Interests: Unpacking the ‘Sharing Economy’," JRC Research Reports JRC101279, Joint Research Centre.
    15. Hokey Min & Yohannes Haile, 2021. "Examining the Role of Disruptive Innovation in Renewable Energy Businesses from a Cross National Perspective," Energies, MDPI, vol. 14(15), pages 1-19, July.
    16. Anna Jonsson & Maria Grafström & Mikael Klintman, 2022. "Unboxing knowledge in collaboration between academia and society: A story about conceptions and epistemic uncertainty [De-essentializing the Knowledge Intensive Firm: Reflections on Skeptical Resea," Science and Public Policy, Oxford University Press, vol. 49(4), pages 583-597.
    17. Niels Stijn & Frank J. Rijnsoever & Martine Veelen, 2018. "Exploring the motives and practices of university–start-up interaction: evidence from Route 128," The Journal of Technology Transfer, Springer, vol. 43(3), pages 674-713, June.
    18. Dorothea Jansen & Regina Görtz & Richard Heidler, 2010. "Knowledge production and the structure of collaboration networks in two scientific fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 219-241, April.
    19. Loet Leydesdorff & Martin Meyer, 2010. "The decline of university patenting and the end of the Bayh–Dole effect," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 355-362, May.
    20. Hubeau, Marianne & Marchand, Fleur & Coteur, Ine & Mondelaers, Koen & Debruyne, Lies & Van Huylenbroeck, Guido, 2017. "A new agri-food systems sustainability approach to identify shared transformation pathways towards sustainability," Ecological Economics, Elsevier, vol. 131(C), pages 52-63.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:45:y:2016:i:3:p:593-603. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.