IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v8y2021i1d10.1057_s41599-021-00854-2.html
   My bibliography  Save this article

Conceptualizing the elements of research impact: towards semantic standards

Author

Listed:
  • Brian Belcher

    (Royal Roads University
    Center for International Forestry Research, P.O. Box 0113, BOCBD)

  • Janet Halliwell

    (J. E. Halliwell Associates Inc)

Abstract

Any effort to understand, evaluate, and improve the impact of research must begin with clear concepts and definitions. Currently, key terms to describe research results are used ambiguously, and the most common definitions for these terms are fundamentally flawed. This hinders research design, evaluation, learning, and accountability. Specifically, the terms outcome and impact are often defined and distinguished from one another using relative characteristics, such as the degree, directness, scale, or duration of change. It is proposed instead to define these terms by the kind of change rather than by the degree or temporal nature of change. Research contributions to a change process are modeled as a series of causally inter-related steps in a results chain or results web with three main kinds of results: (i) the direct products of research, referred to as outputs; (ii) changes in the agency and actions of system actors when they are informed/influenced by research outputs, referred to as outcomes; and (iii) tangible changes in the social, economic, environmental, or other physical condition, referred to as realized benefits. Complete definitions for these terms are provided, along with examples. This classification aims to help focus research evaluation appropriately and enhance appreciation of the multiple pathways and mechanisms by which scholarship contributes to change.

Suggested Citation

  • Brian Belcher & Janet Halliwell, 2021. "Conceptualizing the elements of research impact: towards semantic standards," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-6, December.
  • Handle: RePEc:pal:palcom:v:8:y:2021:i:1:d:10.1057_s41599-021-00854-2
    DOI: 10.1057/s41599-021-00854-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-021-00854-2
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1057/s41599-021-00854-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alis Oancea, 2019. "Research governance and the future(s) of research assessment," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-12, December.
    2. Kate Williams, 2020. "Playing the fields: Theorizing research impact and its assessment," Research Evaluation, Oxford University Press, vol. 29(2), pages 191-202.
    3. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    4. J. Edler & L. Georghiou & K. Blind & E. Uyarra, 2012. "Evaluating the demand side: New challenges for evaluation," Research Evaluation, Oxford University Press, vol. 21(1), pages 33-47, February.
    5. Brian M. Belcher & Katherine E. Rasmussen & Matthew R. Kemshaw & Deborah A. Zornes, 2016. "Defining and assessing research quality in a transdisciplinary context," Research Evaluation, Oxford University Press, vol. 25(1), pages 1-17.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jordi Ardanuy & Llorenç Arguimbau & Ángel Borrego, 2022. "Social sciences and humanities research funded under the European Union Sixth Framework Programme (2002–2006): a long-term assessment of projects, acknowledgements and publications," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-13, December.
    2. repec:oup:rseval:v:32:y:2024:i:2:p:371-383. is not listed on IDEAS
    3. Rosa Kuipers-Dirven & Matthijs Janssen & Jarno Hoekman, 2023. "Assessing university policies for enhancing societal impact of academic research: A multicriteria mapping approach," Research Evaluation, Oxford University Press, vol. 32(2), pages 371-383.
    4. Murphy, Robert P. & Taaffe, Carol & Byrne, Molly & Delaney, Liam & Lunn, Peter D. & Robertson, Deirdre A. & Ryan, Helen & Wood, Alex M., 2024. "Improving the management of hospital waiting lists by using nudges in letters: A Randomised controlled trial," Social Science & Medicine, Elsevier, vol. 361(C).
    5. Gang Chen & Wen-Wen Yan & Xi-Yu Wang & Qingshan Ni & Yang Xiang & Xuhu Mao & Juan-Juan Yue, 2024. "The relationship between coauthorship and the research impact of medical doctoral students: A social capital perspective," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-13, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Morgan-Thomas, Anna & Tsoukas, Serafeim & Dudau, Adina & Gąska, Paweł, 2024. "Beyond declarations: Metrics, rankings and responsible assessment," Research Policy, Elsevier, vol. 53(10).
    2. Jonna Brenninkmeijer, 2022. "Achieving societal and academic impacts of research: A comparison of networks, values, and strategies [University Research Funding and Publication Performance - an International Comparison]," Science and Public Policy, Oxford University Press, vol. 49(5), pages 728-738.
    3. Reed, M.S. & Ferré, M. & Martin-Ortega, J. & Blanche, R. & Lawford-Rolfe, R. & Dallimer, M. & Holden, J., 2021. "Evaluating impact from research: A methodological framework," Research Policy, Elsevier, vol. 50(4).
    4. Jorrit P Smit & Laurens K Hessels, 2021. "The production of scientific and societal value in research evaluation: a review of societal impact assessment methods [Systems Thinking, Knowledge and Action: Towards Better Models and Methods]," Research Evaluation, Oxford University Press, vol. 30(3), pages 323-335.
    5. Bührer, Susanne & Feidenheimer, Alexander & Walz, Rainer & Lindner, Ralf & Beckert, Bernd & Wallwaey, Elisa, 2022. "Concepts and methods to measure societal impacts: An overview," Discussion Papers "Innovation Systems and Policy Analysis" 74, Fraunhofer Institute for Systems and Innovation Research (ISI).
    6. Vučković Dijana & Pekovic Sanja & Popović Stevo & Janinovic Jovana, 2023. "Assessing the Appraisal of Research Quality in Social Sciences and Humanities: A Case Study of the University of Montenegro," Business Systems Research, Sciendo, vol. 14(1), pages 131-152, September.
    7. Bryce, Cormac & Dowling, Michael & Lucey, Brian, 2020. "The journal quality perception gap," Research Policy, Elsevier, vol. 49(5).
    8. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    9. Michaela Strinzel & Josh Brown & Wolfgang Kaltenbrunner & Sarah Rijcke & Michael Hill, 2021. "Ten ways to improve academic CVs for fairer research assessment," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-4, December.
    10. Sten F Odenwald, 2020. "A citation study of earth science projects in citizen science," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-26, July.
    11. Dima Jamali & Georges Samara & Lokman I. Meho, 2024. "Determinants of research productivity and efficiency among the Arab world’s accredited business schools," Management Review Quarterly, Springer, vol. 74(4), pages 2511-2543, December.
    12. Corsini, Alberto & Pezzoni, Michele, 2023. "Does grant funding foster research impact? Evidence from France," Journal of Informetrics, Elsevier, vol. 17(4).
    13. Ghisetti, Claudia, 2017. "Demand-pull and environmental innovations: Estimating the effects of innovative public procurement," Technological Forecasting and Social Change, Elsevier, vol. 125(C), pages 178-187.
    14. Alexander Kalgin & Olga Kalgina & Anna Lebedeva, 2019. "Publication Metrics as a Tool for Measuring Research Productivity and Their Relation to Motivation," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 44-86.
    15. Gregorio González-Alcaide, 2021. "Bibliometric studies outside the information science and library science field: uncontainable or uncontrollable?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6837-6870, August.
    16. Martin Ricker, 2017. "Letter to the Editor: About the quality and impact of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1851-1855, June.
    17. Melinda Craike & Bojana Klepac & Amy Mowle & Therese Riley, 2023. "Theory of systems change: An initial, middle-range theory of public health research impact," Research Evaluation, Oxford University Press, vol. 32(3), pages 603-621.
    18. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    19. Joost Kosten, 2016. "A classification of the use of research indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 457-464, July.
    20. Daniela De Filippo & Fernanda Morillo & Borja González-Albo, 2023. "Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences," Publications, MDPI, vol. 11(2), pages 1-17, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:8:y:2021:i:1:d:10.1057_s41599-021-00854-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.