IDEAS home Printed from https://ideas.repec.org/a/kap/jtecht/v47y2022i3d10.1007_s10961-017-9601-0.html
   My bibliography  Save this article

Towards a new generation of research impact assessment approaches

Author

Listed:
  • Pierre-Benoit Joly

    (Université Paris Est)

  • Mireille Matt

    (Université Grenoble-Alpes)

Abstract

In the literature, most of the authors agree that addressing “grand challenges” implies systems transformation and requires social, economic and technological changes. It involves new policy rationales and design and consequently new approaches of research impact assessment (RIA). The challenge is to develop methods able to improve our understanding of the impact generating mechanisms, to assess a broader set of impacts, and to incorporate a mix of complementary approaches (combining qualitative and quantitative evaluation, achieving a multi-objective evaluation, and evaluating impacts at various level of aggregation). The six papers in this Special Issue present, discuss and even assess a new generation of RIA approaches. None of the papers covers all the RIA challenges highlighted above. At this stage, it would be illusive to develop a common evaluation approach, with standardized methods and metrics. We contend that the availability of a diversity of tools is a precious asset if impact assessment allows a conscientious attention to the transformative effects of research.

Suggested Citation

  • Pierre-Benoit Joly & Mireille Matt, 2022. "Towards a new generation of research impact assessment approaches," The Journal of Technology Transfer, Springer, vol. 47(3), pages 621-631, June.
  • Handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9601-0
    DOI: 10.1007/s10961-017-9601-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10961-017-9601-0
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s10961-017-9601-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    2. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    3. Zvi Griliches, 1958. "Research Costs and Social Returns: Hybrid Corn and Related Innovations," Journal of Political Economy, University of Chicago Press, vol. 66(5), pages 419-419.
    4. Julian M. Alston & Philip G. Pardey & Jennifer S. James & Matthew A. Anderson, 2009. "The Economics of Agricultural R&D," Annual Review of Resource Economics, Annual Reviews, vol. 1(1), pages 537-566, September.
    5. Douthwaite, Boru & Kuby, Thomas & van de Fliert, Elske & Schulz, Steffen, 2003. "Impact pathway evaluation: an approach for achieving and attributing impact in complex systems," Agricultural Systems, Elsevier, vol. 78(2), pages 243-265, November.
    6. Foray, D. & Mowery, D.C. & Nelson, R.R., 2012. "Public R&D and social challenges: What lessons from mission R&D programs?," Research Policy, Elsevier, vol. 41(10), pages 1697-1702.
    7. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    8. Jakob Edler & Martin Berger & Michael Dinges & Abdullah Gök, 2012. "The practice of evaluation in innovation policy in Europe," Research Evaluation, Oxford University Press, vol. 21(3), pages 167-182, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Paget, Nicolas & Le Gal, Pierre-Yves & Goulet, Frédéric, 2024. "Motivations and challenges of intrapreneurship in research organizations. The case of decision support systems in agricultural research for development," Technovation, Elsevier, vol. 130(C).
    2. Nugent, Annita & Chan, Ho Fai, 2023. "Outsourcing university research commercialization to a sophisticated technology transfer office: Evidence from Australian universities," Technovation, Elsevier, vol. 125(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pierre Benoit Joly & Laurence Colinet & Ariane Gaunand & Stephane Lemarié & Mireille Matt, 2016. "Agricultural research impact assessment: issues, methods and challenges," Working Papers hal-01431457, HAL.
    2. Matt, M. & Gaunand, A. & Joly, P-B. & Colinet, L., 2017. "Opening the black box of impact – Ideal-type impact pathways in a public agricultural research organization," Research Policy, Elsevier, vol. 46(1), pages 207-218.
    3. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    4. Faure, Guy & Barret, Danielle & Blundo-Canto, Genowefa & Dabat, Marie-Hélène & Devaux-Spatarakis, Agathe & Le Guerroué, Jean Louis & Marquié, Catherine & Mathé, Syndhia & Temple, Ludovic & Toillier, A, 2018. "How different agricultural research models contribute to impacts: Evidence from 13 case studies in developing countries," Agricultural Systems, Elsevier, vol. 165(C), pages 128-136.
    5. Paget, Nicolas & Le Gal, Pierre-Yves & Goulet, Frédéric, 2024. "Motivations and challenges of intrapreneurship in research organizations. The case of decision support systems in agricultural research for development," Technovation, Elsevier, vol. 130(C).
    6. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    7. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    8. Bührer, Susanne & Feidenheimer, Alexander & Walz, Rainer & Lindner, Ralf & Beckert, Bernd & Wallwaey, Elisa, 2022. "Concepts and methods to measure societal impacts: An overview," Discussion Papers "Innovation Systems and Policy Analysis" 74, Fraunhofer Institute for Systems and Innovation Research (ISI).
    9. A. Gaunand & L. Colinet & P.-B. Joly & M. Matt, 2022. "Counting what really counts? Assessing the political impact of science," The Journal of Technology Transfer, Springer, vol. 47(3), pages 699-721, June.
    10. Irwin Feller, 2022. "Assessing the societal impact of publicly funded research," The Journal of Technology Transfer, Springer, vol. 47(3), pages 632-650, June.
    11. Andrea Bonaccorsi & Filippo Chiarello & Gualtiero Fantoni, 2021. "Impact for whom? Mapping the users of public research with lexicon-based text mining," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1745-1774, February.
    12. Gaunand, A. & Hocdé, A. & Lemarié, S. & Matt, M. & Turckheim, E.de, 2015. "How does public agricultural research impact society? A characterization of various patterns," Research Policy, Elsevier, vol. 44(4), pages 849-861.
    13. Genowefa Blundo-Canto & Bernard Triomphe & Guy Faure & Danielle Barret & Aurelle de Romemont & Etienne Hainzelin, 2019. "Building a culture of impact in an international agricultural research organization: Process and reflective learning," Research Evaluation, Oxford University Press, vol. 28(2), pages 136-144.
    14. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    15. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    16. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    17. Guesmi, B. & Gil, J.M., 2018. "Monitoring and Evaluation of Public Agricultural Research Organization: Impact oriented monitoring approach," 2018 Conference, July 28-August 2, 2018, Vancouver, British Columbia 277554, International Association of Agricultural Economists.
    18. Magen, Benjamin B., 2012. "An Ex Post Economic Impact Assessment of Bean/Cowpea Crsp’s Investment on Varietal Development in Senegal," Graduate Research Master's Degree Plan B Papers 142739, Michigan State University, Department of Agricultural, Food, and Resource Economics.
    19. Mason-D'Croz, Daniel & Sulser, Timothy B. & Wiebe, Keith & Rosegrant, Mark W. & Lowder, Sarah K. & Nin-Pratt, Alejandro & Willenbockel, Dirk & Robinson, Sherman & Zhu, Tingju & Cenacchi, Nicola & Duns, 2019. "Agricultural investments and hunger in Africa modeling potential contributions to SDG2 – Zero Hunger," World Development, Elsevier, vol. 116(C), pages 38-53.
    20. Alessandro Magrini & Fabio Bartolini & Alessandra Coli & Barbara Pacini, 2019. "A structural equation model to assess the impact of agricultural research expenditure on multiple dimensions," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(4), pages 2063-2080, July.

    More about this item

    Keywords

    Grand challenges; Research impact assessment; Evaluation methods; Broader impacts; Metrology;
    All these keywords.

    JEL classification:

    • O30 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - General
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy
    • A13 - General Economics and Teaching - - General Economics - - - Relation of Economics to Social Values
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9601-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.