IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v24y2015i2p131-145..html
   My bibliography  Save this article

Impact oriented monitoring: A new methodology for monitoring and evaluation of international public health research projects

Author

Listed:
  • J. Guinea
  • E. Sela
  • A. J. Gómez-Núñez
  • T. Mangwende
  • A. Ambali
  • N. Ngum
  • H. Jaramillo
  • J. M. Gallego
  • A. Patiño
  • C. Latorre
  • S. Srivanichakorn
  • B. Thepthien

Abstract

Impact oriented monitoring (IOM) is a novel methodology for identifying and assessing the impacts of EU-funded research projects in the area of International Public Health. A framework based on the logic and payback categories is used to categorize, into multi-dimensions, the impacts produced by public health research projects. A set of tools, including: 1) a project results framework; 2) a coordinators’ survey; 3) an end users’ survey, and 4) an assessment tool (scoring matrix), have been developed for both collecting data on results and activities and helping in assessing impacts. The need to know the impact of the projects based on robust data but, at the same time, to minimize the time and resources required of both the EC officers implementing the methodology and the project coordinators providing data on results and impacts, has led to the development of the IOM methodology. The methodology is expected to provide the DG Research and Innovation of the European Commission (EC) with usable information on how the research projects in the area of international public health are producing impacts. More importantly, this information could improve the performance of existing programmes and also form the basis for supporting research policy planning. The IOM methodology can be easily adapted to other EC-funded research areas and also implemented in other countries by institutions responsible for funding research activities.

Suggested Citation

  • J. Guinea & E. Sela & A. J. Gómez-Núñez & T. Mangwende & A. Ambali & N. Ngum & H. Jaramillo & J. M. Gallego & A. Patiño & C. Latorre & S. Srivanichakorn & B. Thepthien, 2015. "Impact oriented monitoring: A new methodology for monitoring and evaluation of international public health research projects," Research Evaluation, Oxford University Press, vol. 24(2), pages 131-145.
    • Handle: RePEc:oup:rseval:v:24:y:2015:i:2:p:131-145.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvu034
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Czarnitzki, Dirk & Lopes-Bento, Cindy, 2013. "Value for money? New microeconometric evidence on public R&D grants in Flanders," Research Policy, Elsevier, vol. 42(1), pages 76-89.
    2. Teresa Penfield & Matthew J. Baker & Rosa Scoble & Michael C. Wykes, 2013. "Assessment, evaluations, and definitions of research impact: A review," Research Evaluation, Oxford University Press, vol. 23(1), pages 21-32, October.
    3. Albert N. Link & Nicholas S. Vonortas (ed.), 2013. "Handbook on the Theory and Practice of Program Evaluation," Books, Edward Elgar Publishing, number 14384.
    4. William E. Souder, 1972. "A Scoring Methodology for Assessing the Suitability of Management Science Models," Management Science, INFORMS, vol. 18(10), pages 526-543, June.
    5. Gilbert Fayl & Yves Dumont & Luc Durieux & Isidoros Karatzas & Liam O'Sullivan, 1998. "Evaluation of research and technological development programmes: a tool for policy design," Research Evaluation, Oxford University Press, vol. 7(2), pages 93-97, August.
    6. Steve Hanney & Andrew Davies & Martin Buxton, 1999. "Assessing benefits from health research projects: can we use questionnaires instead of case studies?," Research Evaluation, Oxford University Press, vol. 8(3), pages 189-199, December.
    7. Luukkonen, Terttu, 1998. "The difficulties in assessing the impact of EU framework programmes," Research Policy, Elsevier, vol. 27(6), pages 599-610, September.
    8. Erik Millstone & Patrick Van Zwanenberg & Fiona Marshall, 2010. "Monitoring and Evaluating Agricultural Science and Technology Projects: Theories, Practices and Problems," IDS Bulletin, Blackwell Publishing, vol. 41(6), pages 75-87, November.
    9. Georghiou, Luke & Roessner, David, 2000. "Evaluating technology programs: tools and methods," Research Policy, Elsevier, vol. 29(4-5), pages 657-678, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Parreiras, R.O. & Kokshenev, I. & Carvalho, M.O.M. & Willer, A.C.M. & Dellezzopolles, C.F. & Nacif, D.B. & Santana, J.A., 2019. "A flexible multicriteria decision-making methodology to support the strategic management of Science, Technology and Innovation research funding programs," European Journal of Operational Research, Elsevier, vol. 272(2), pages 725-739.
    2. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huergo, Elena & Trenado, Mayte & Ubierna, Andrés, 2016. "The impact of public support on firm propensity to engage in R&D: Spanish experience," Technological Forecasting and Social Change, Elsevier, vol. 113(PB), pages 206-219.
    2. Bayona-Sáez, Cristina & García-Marco, Teresa, 2010. "Assessing the effectiveness of the Eureka Program," Research Policy, Elsevier, vol. 39(10), pages 1375-1386, December.
    3. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.
    4. Sergio Afcha & Jose García-Quevedo, 2016. "The impact of R&D subsidies on R&D employment composition," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 25(6), pages 955-975.
    5. Catharina Sikow-Magny & Marcel Rommerts, 2005. "Policy Transfer through Research Networks – the Example of Urban Road Pricing," ifo DICE Report, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 3(03), pages 22-27, November.
    6. K. Poehlmann & R. Helm & O. Mauroner & J. Auburger, 2021. "Corporate spin-offs’ success factors: management lessons from a comparative empirical analysis with research-based spin-offs," Review of Managerial Science, Springer, vol. 15(6), pages 1767-1796, August.
    7. Stuart D. Allen & Stephen K. Layson & Albert N. Link, 2013. "Public gains from entrepreneurial research: Inferences about the economic value of public support of the Small Business Innovation Research program," Chapters, in: Public Support of Innovation in Entrepreneurial Firms, chapter 6, pages 105-112, Edward Elgar Publishing.
    8. Simen G. Enger & Fulvio Castellacci, 2016. "Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1611-1638, December.
    9. Ebersberger, Bernd & Edler, Jakob & Lo, Vivien, 2006. "Improving policy understanding by means of secondary analyses of policy evaluation: a concept development," Discussion Papers "Innovation Systems and Policy Analysis" 12, Fraunhofer Institute for Systems and Innovation Research (ISI).
    10. Hans Pohl, 2021. "Internationalisation, innovation, and academic–corporate co-publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1329-1358, February.
    11. Enrico Vanino & Stephen Roper & Bettina Becker, 2020. "Knowledge to Money: Assessing the Business Performance Effects of Publicly Funded R&D Grants," ifo DICE Report, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 17(04), pages 20-24, January.
    12. Dragana Radicic & Geoffrey Pugh & Hugo Hollanders & René Wintjes & Jon Fairburn, 2016. "The impact of innovation support programs on small and medium enterprises innovation in traditional manufacturing industries: An evaluation for seven European Union regions," Environment and Planning C, , vol. 34(8), pages 1425-1452, December.
    13. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    14. Munari, Federico & Toschi, Laura, 2021. "The impact of public funding on science valorisation: an analysis of the ERC Proof-of-Concept Programme," Research Policy, Elsevier, vol. 50(6).
    15. Di Guo & Yan Guo & Kun Jiang, 2017. "Funding Forms, Market Conditions, And Dynamic Effects Of Government R&D Subsidies: Evidence From China," Economic Inquiry, Western Economic Association International, vol. 55(2), pages 825-842, April.
    16. Stjepan Srhoj & Bruno Škrinjarić & Sonja Radas & Janette Walde, 2022. "Small matching grants for women entrepreneurs: lessons from the past recession," Small Business Economics, Springer, vol. 59(1), pages 117-142, June.
    17. Ugur, Mehmet & Trushin, Eshref, 2018. "Asymmetric information and heterogeneous effects of R&D subsidies: evidence on R&D investment and employment of R&D personel," Greenwich Papers in Political Economy 21943, University of Greenwich, Greenwich Political Economy Research Centre.
    18. Tea Petrin & Dragana Radicic, 2023. "Instrument policy mix and firm size: is there complementarity between R&D subsidies and R&D tax credits?," The Journal of Technology Transfer, Springer, vol. 48(1), pages 181-215, February.
    19. Catalina Martínez & Laura Cruz-Castro & Luis Sanz-Menéndez, 2016. "Innovation capabilities in the private sector: evaluating subsidies for hiring S&T workers in Spain," Research Evaluation, Oxford University Press, vol. 25(2), pages 196-208.
    20. Paul Hünermund & Dirk Czarnitzki, 2016. "Estimating the local average treatment effect of R&D subsidies in a pan-European program," Working Papers of Department of Management, Strategy and Innovation, Leuven 541177, KU Leuven, Faculty of Economics and Business (FEB), Department of Management, Strategy and Innovation, Leuven.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:24:y:2015:i:2:p:131-145.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.