IDEAS home Printed from https://ideas.repec.org/a/kap/jtecht/v47y2022i3d10.1007_s10961-017-9603-y.html
   My bibliography  Save this article

The BETA-EvaRIO impact evaluation method: towards a bridging approach?

Author

Listed:
  • Laurent Bach

    (University of Strasbourg/CNRS UMR7522)

  • Sandrine Wolff

    (University of Strasbourg/CNRS UMR7522)

Abstract

This paper presents a specific approach in the field of evaluating the effects of publicly supported R&D activities: the BETA approach. Initially developed for application to large technology procurement/agency driven public R&D programmes, it has been adapted to a larger range of programmes supporting science and technology. The BETA approach consists of identifying and retrospectively measuring, at the micro-level via direct interviews, the impact of different types of learning processes triggered by participation in R&D. A further adaptation to the specific case of research infrastructures was conducted in the frame of the EvaRIO project. It also dealt with learning processes from an ex post and micro-level perspective, but covered a wider variety of activities and actors over time, leading to a broader view of effects. Based on 30 years of impact evaluation experience, we argue that the revised BETA approach, despite its focus on the economic dimension of impacts, is at the intersection between different strands of evaluation research. The dual nature of the BETA approach is examined along several dimensions including an output versus process perspective, project versus organization scope, qualitative versus quantitative investigation, contribution versus attribution issues, private profitability versus public value perspective, etc. We show that the BETA-EvaRIO method can be used to complement other evaluation methodologies and can be considered as bridging among them, or can be seen as a single, specific approach.

Suggested Citation

  • Laurent Bach & Sandrine Wolff, 2022. "The BETA-EvaRIO impact evaluation method: towards a bridging approach?," The Journal of Technology Transfer, Springer, vol. 47(3), pages 651-672, June.
  • Handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9603-y
    DOI: 10.1007/s10961-017-9603-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10961-017-9603-y
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s10961-017-9603-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chiara PANCOTTI & Julie PELLEGRIN & Silvia VIGNETTI, 2014. "Appraisal of Research Infrastructures: Approaches, methods and practical implications," Departmental Working Papers 2014-13, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    2. Unknown, 2016. "Department Publications 2014," Publications Lists 239845, University of Minnesota, Department of Applied Economics.
    3. Abdullah Gök & Jakob Edler, 2012. "The use of behavioural additionality evaluation in innovation policy making," Research Evaluation, Oxford University Press, vol. 21(4), pages 306-318, September.
    4. Gaunand, A. & Hocdé, A. & Lemarié, S. & Matt, M. & Turckheim, E.de, 2015. "How does public agricultural research impact society? A characterization of various patterns," Research Policy, Elsevier, vol. 44(4), pages 849-861.
    5. Autio, Erkko & Hameri, Ari-Pekka & Vuola, Olli, 2004. "A framework of industrial knowledge spillovers in big-science centers," Research Policy, Elsevier, vol. 33(1), pages 107-126, January.
    6. Albert N. Link & Nicholas S. Vonortas (ed.), 2013. "Handbook on the Theory and Practice of Program Evaluation," Books, Edward Elgar Publishing, number 14384.
    7. David J. Teece & Gary Pisano & Amy Shuen, 1997. "Dynamic capabilities and strategic management," Strategic Management Journal, Wiley Blackwell, vol. 18(7), pages 509-533, August.
    8. Toole, Andrew A., 2012. "The impact of public basic research on industrial innovation: Evidence from the pharmaceutical industry," Research Policy, Elsevier, vol. 41(1), pages 1-12.
    9. Olof Hallonsten, 2013. "Introducing ‘facilitymetrics’: a first review and analysis of commonly used measures of scientific leadership among synchrotron radiation facilities worldwide," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 497-513, August.
    10. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    11. Jakob Edler & Martin Berger & Michael Dinges & Abdullah Gök, 2012. "The practice of evaluation in innovation policy in Europe," Research Evaluation, Oxford University Press, vol. 21(3), pages 167-182, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laurent Bach & Sandrine Wolff, 2017. "The BETA-EvaRIO impact evaluation method: towards a bridging approach?," Post-Print hal-02167827, HAL.
    2. Irwin Feller, 2022. "Assessing the societal impact of publicly funded research," The Journal of Technology Transfer, Springer, vol. 47(3), pages 632-650, June.
    3. Andrea, Bastianin & Paolo, Castelnuovo & Massimo, Florio & Anna, Giunta, 2019. "Technological Learning and Innovation Gestation Lags at the Frontier of Science: from CERN Procurement to Patents," Working Papers 405, University of Milano-Bicocca, Department of Economics, revised Apr 2019.
    4. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    5. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    6. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    7. Peter Weißhuhn & Katharina Helming & Johanna Ferretti, 2018. "Research impact assessment in agriculture—A review of approaches and impact areas," Research Evaluation, Oxford University Press, vol. 27(1), pages 36-42.
    8. Kergroach, Sandrine, 2019. "National innovation policies for technology upgrading through GVCs: A cross-country comparison," Technological Forecasting and Social Change, Elsevier, vol. 145(C), pages 258-272.
    9. Purkus, Alexandra & Lüdtke, Jan, 2020. "A systemic evaluation framework for a multi-actor, forest-based bioeconomy governance process: The German Charter for Wood 2.0 as a case study," Forest Policy and Economics, Elsevier, vol. 113(C).
    10. Attila Havas, 2014. "Trapped by the high-tech myth: the need and chances for a new policy rationale," Chapters, in: Hartmut Hirsch-Kreinsen & Isabel Schwinge (ed.), Knowledge-Intensive Entrepreneurship in Low-Tech Industries, chapter 9, pages 193-217, Edward Elgar Publishing.
    11. D’Ippolito, Beatrice & Rüling, Charles-Clemens, 2019. "Research collaboration in Large Scale Research Infrastructures: Collaboration types and policy implications," Research Policy, Elsevier, vol. 48(5), pages 1282-1296.
    12. Matt, M. & Gaunand, A. & Joly, P-B. & Colinet, L., 2017. "Opening the black box of impact – Ideal-type impact pathways in a public agricultural research organization," Research Policy, Elsevier, vol. 46(1), pages 207-218.
    13. Attila Havas, 2015. "The persistent high-tech myth in the EC policy circles - Implications for the EU10 countries," CERS-IE WORKING PAPERS 1517, Institute of Economics, Centre for Economic and Regional Studies.
    14. Robbiano, Simone, 2022. "The innovative impact of public research institutes: Evidence from Italy," Research Policy, Elsevier, vol. 51(10).
    15. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    16. Haegeman, Karel & Spiesberger, Manfred & Könnölä, Totti, 2017. "Evaluating foresight in transnational research programming," Technological Forecasting and Social Change, Elsevier, vol. 115(C), pages 313-326.
    17. Havas, Attila, 2014. "Mit mér(j)ünk?. Az innováció értelmezései - szakpolitikai következmények [The theory and measurement of innovation and its mutual effect on policy]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(9), pages 1022-1059.
    18. Andrea Bastianin & Paolo Castelnovo & Massimo Florio & Anna Giunta, 2022. "Big science and innovation: gestation lag from procurement to patents for CERN suppliers," The Journal of Technology Transfer, Springer, vol. 47(2), pages 531-555, April.
    19. Pierre-Benoit Joly & Mireille Matt, 2022. "Towards a new generation of research impact assessment approaches," The Journal of Technology Transfer, Springer, vol. 47(3), pages 621-631, June.
    20. Genowefa Blundo-Canto & Bernard Triomphe & Guy Faure & Danielle Barret & Aurelle de Romemont & Etienne Hainzelin, 2019. "Building a culture of impact in an international agricultural research organization: Process and reflective learning," Research Evaluation, Oxford University Press, vol. 28(2), pages 136-144.

    More about this item

    Keywords

    Evaluation; Research infrastructures; Science; R&D; Socio-economic impact; Methodological development;
    All these keywords.

    JEL classification:

    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
    • H54 - Public Economics - - National Government Expenditures and Related Policies - - - Infrastructures

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9603-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.