IDEAS home Printed from https://ideas.repec.org/a/eee/forpol/v114y2020ics138993411830220x.html
   My bibliography  Save this article

A framework to evaluate the impacts of research on policy and practice: A forestry pilot study

Author

Listed:
  • Edwards, David M.
  • Meagher, Laura R.

Abstract

Increasingly, research funders are seeking evidence of the non-academic impacts of research, yet there is confusion as to what is meant by impact generation, and how it can be facilitated, captured and shared. This paper presents a framework to evaluate research impact, which we have piloted on 12 case studies led by Forest Research, a government research agency in UK. The framework was found to be user-friendly and fit for purpose. The breadth of impact types identified in the framework stimulated researchers to think beyond ‘instrumental’ impact and identify other changes generated by their work. By providing a generic list of ‘reasons for impact’, it encouraged researchers to think about the relationships between processes that had led, or could lead, towards impacts, including less tangible reasons that might otherwise go unmentioned. Asking for ‘lessons learned’ opened the way for researchers to step back and reflect critically on what had happened and what could be improved in the future. The framework was seen to function as a launch pad for research teams and stakeholders to translate isolated examples of impact and reasons for impact into narratives for both learning and dissemination. It could prove operationally useful to researchers and organisations beyond forestry, particularly those in applied fields and/or institutes.

Suggested Citation

  • Edwards, David M. & Meagher, Laura R., 2020. "A framework to evaluate the impacts of research on policy and practice: A forestry pilot study," Forest Policy and Economics, Elsevier, vol. 114(C).
    • Handle: RePEc:eee:forpol:v:114:y:2020:i:c:s138993411830220x
    DOI: 10.1016/j.forpol.2019.101975
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S138993411830220X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.forpol.2019.101975?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Christian Pohl & Stephan Rist & Anne Zimmermann & Patricia Fry & Ghana S Gurung & Flurina Schneider & Chinwe Ifejika Speranza & Boniface Kiteme & Sébastian Boillat & Elvira Serrano & Gertrude Hirsch H, 2010. "Researchers' roles in knowledge co-production: experience from sustainability research in Kenya, Switzerland, Bolivia and Nepal," Science and Public Policy, Oxford University Press, vol. 37(4), pages 267-281, May.
    2. Laura Meagher & Catherine Lyall & Sandra Nutley, 2008. "Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research," Research Evaluation, Oxford University Press, vol. 17(3), pages 163-173, September.
    3. Michael Woolcock, 2013. "Using Case Studies to Explore the External Validity of ‘Complex’ Development Interventions," CID Working Papers 270, Center for International Development at Harvard University.
    4. Stina Hansson & Merritt Polk, 2018. "Assessing the impact of transdisciplinary research: The usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact," Research Evaluation, Oxford University Press, vol. 27(2), pages 132-144.
    5. Steve Martin, 2010. "Co-production of social research: strategies for engaged scholarship," Public Money & Management, Taylor & Francis Journals, vol. 30(4), pages 211-218, July.
    6. Hulme,Mike, 2009. "Why We Disagree about Climate Change," Cambridge Books, Cambridge University Press, number 9780521727327, September.
    7. Branwen Morgan, 2014. "Research impact: Income for outcome," Nature, Nature, vol. 511(7510), pages 72-75, July.
    8. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    9. Jon Bannister & Irene Hardill, 2013. "Knowledge mobilisation and the social sciences: dancing with new partners in an age of austerity," Contemporary Social Science, Taylor & Francis Journals, vol. 8(3), pages 167-175, November.
    10. Krott, Max, 2012. "Value and risks of the use of analytical theory in science for forest policy," Forest Policy and Economics, Elsevier, vol. 16(C), pages 35-42.
    11. Stevanov, Mirjana & Böcher, Michael & Krott, Max & Krajter, Silvija & Vuletic, Dijana & Orlovic, Sasa, 2013. "The Research, Integration and Utilization (RIU) model as an analytical framework for the professionalization of departmental research organizations: Case studies of publicly funded forest research ins," Forest Policy and Economics, Elsevier, vol. 37(C), pages 20-28.
    12. Hulme,Mike, 2009. "Why We Disagree about Climate Change," Cambridge Books, Cambridge University Press, number 9780521898690, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elizabeth Troncoso & Francisco Ganga-Contreras & Margarita Briceño, 2022. "Incentive Policies for Scientific Publications in the State Universities of Chile," Publications, MDPI, vol. 10(2), pages 1-19, June.
    2. Yitbarek, Tibebe Weldesemaet & Wilson, John R.U. & Dehnen-Schmutz, Katharina, 2023. "A governance framework for the design and evaluation of tree planting schemes," Forest Policy and Economics, Elsevier, vol. 152(C).
    3. Marcelo de Souza Bispo, 2022. "In Defense of Theory and Original Theoretical Contributions in Administration," RAC - Revista de Administração Contemporânea (Journal of Contemporary Administration), ANPAD - Associação Nacional de Pós-Graduação e Pesquisa em Administração, vol. 26(6), pages 220158-2201.
    4. Maria Nijnik & Tatiana Kluvánková & Albert Nijnik & Serhiy Kopiy & Mariana Melnykovych & Simo Sarkki & Carla Barlagne & Stanislava Brnkaláková & Leonid Kopiy & Igor Fizyk & David Miller, 2020. "Is There a Scope for Social Innovation in Ukrainian Forestry?," Sustainability, MDPI, vol. 12(22), pages 1-19, November.
    5. Gerlak, Andrea K. & Guido, Zack & Owen, Gigi & McGoffin, Mariana Sofia Rodriguez & Louder, Elena & Davies, Julia & Smith, Kelly Jay & Zimmer, Andy & Murveit, Anna M. & Meadow, Alison & Shrestha, Padme, 2023. "Stakeholder engagement in the co-production of knowledge for environmental decision-making," World Development, Elsevier, vol. 170(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Do, Thi Huong & Krott, Max & Böcher, Michael, 2020. "Multiple traps of scientific knowledge transfer: Comparative case studies based on the RIU model from Vietnam, Germany, Indonesia, Japan, and Sweden," Forest Policy and Economics, Elsevier, vol. 114(C).
    2. Böcher, Michael, 2016. "How does science-based policy advice matter in policy making? The RIU model as a framework for analyzing and explaining processes of scientific knowledge transfer," Forest Policy and Economics, Elsevier, vol. 68(C), pages 65-72.
    3. Stevanov, Mirjana & Böcher, Michael & Krott, Max & Krajter, Silvija & Vuletic, Dijana & Orlovic, Sasa, 2013. "The Research, Integration and Utilization (RIU) model as an analytical framework for the professionalization of departmental research organizations: Case studies of publicly funded forest research ins," Forest Policy and Economics, Elsevier, vol. 37(C), pages 20-28.
    4. Andreas Bjurström & Merritt Polk, 2011. "Climate change and interdisciplinarity: a co-citation analysis of IPCC Third Assessment Report," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 525-550, June.
    5. Katherine Casey & Rachel Glennerster & Edward Miguel & Maarten Voors, 2023. "Skill Versus Voice in Local Development," The Review of Economics and Statistics, MIT Press, vol. 105(2), pages 311-326, March.
    6. Quentin Stoeffler & Michael Carter & Catherine Guirkinger & Wouter Gelade, 2022. "The Spillover Impact of Index Insurance on Agricultural Investment by Cotton Farmers in Burkina Faso," The World Bank Economic Review, World Bank, vol. 36(1), pages 114-140.
    7. Janet Judy McIntyre‐Mills, 2013. "Anthropocentrism and Well‐being: A Way Out of the Lobster Pot?," Systems Research and Behavioral Science, Wiley Blackwell, vol. 30(2), pages 136-155, March.
    8. Friederike Hartz, 2024. "“We are not droids”– IPCC participants’ senses of responsibility and affective experiences across the production, assessment, communication and enactment of climate science," Climatic Change, Springer, vol. 177(6), pages 1-21, June.
    9. Hall, C. Michael & Amelung, Bas & Cohen, Scott & Eijgelaar, Eke & Gössling, Stefan & Higham, James & Leemans, Rik & Peeters, Paul & Ram, Yael & Scott, Daniel & Aall, Carlo & Abegg, Bruno & Araña, Jorg, 2015. "No time for smokescreen skepticism: A rejoinder to Shani and Arad," Tourism Management, Elsevier, vol. 47(C), pages 341-347.
    10. Nancy Menning, 2018. "Narrating climate change as a rite of passage," Climatic Change, Springer, vol. 147(1), pages 343-353, March.
    11. Arnaud Vaganay, 2016. "Cluster Sampling Bias in Government-Sponsored Evaluations: A Correlational Study of Employment and Welfare Pilots in England," PLOS ONE, Public Library of Science, vol. 11(8), pages 1-21, August.
    12. Mercedes Bleda & Elisabeth Krull & Jonatan Pinkse & Eleni Christodoulou, 2023. "Organizational heuristics and firms' sensemaking for climate change adaptation," Business Strategy and the Environment, Wiley Blackwell, vol. 32(8), pages 6124-6137, December.
    13. Chhetri, Netra & Ghimire, Rajiv & Wagner, Melissa & Wang, Meng, 2020. "Global citizen deliberation: Case of world-wide views on climate and energy," Energy Policy, Elsevier, vol. 147(C).
    14. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    15. Hochachka, Gail, 2021. "Integrating the four faces of climate change adaptation: Towards transformative change in Guatemalan coffee communities," World Development, Elsevier, vol. 140(C).
    16. Terje Aven & Ortwin Renn, 2015. "An Evaluation of the Treatment of Risk and Uncertainties in the IPCC Reports on Climate Change," Risk Analysis, John Wiley & Sons, vol. 35(4), pages 701-712, April.
    17. Hakiman, Kamran & Sheely, Ryan, 2023. "Unlocking the Potential of Participatory Planning: How Flexible and Adaptive Governance Interventions Can Work in Practice," OSF Preprints kucjs, Center for Open Science.
    18. George Ferns & Kenneth Amaeshi & Aliette Lambert, 2019. "Drilling their Own Graves: How the European Oil and Gas Supermajors Avoid Sustainability Tensions Through Mythmaking," Journal of Business Ethics, Springer, vol. 158(1), pages 201-231, August.
    19. Shaw, Christopher & Nerlich, Brigitte, 2015. "Metaphor as a mechanism of global climate change governance: A study of international policies, 1992–2012," Ecological Economics, Elsevier, vol. 109(C), pages 34-40.
    20. Kattirtzi, Michael & Winskel, Mark, 2020. "When experts disagree: Using the Policy Delphi method to analyse divergent expert expectations and preferences on UK energy futures," Technological Forecasting and Social Change, Elsevier, vol. 153(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:forpol:v:114:y:2020:i:c:s138993411830220x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/forpol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.