IDEAS home Printed from https://ideas.repec.org/a/kap/jtecht/v47y2022i3d10.1007_s10961-017-9606-8.html
   My bibliography  Save this article

The evaluation of the Austrian START programme: an impact analysis of a research funding programme using a multi-method approach

Author

Listed:
  • Sarah Seus

    (Fraunhofer ISI)

  • Susanne Bührer

    (Fraunhofer ISI
    Eva Heckl KMU Forschung Austria)

Abstract

The following article presents and discusses the approach and findings of a recently conducted evaluation study of the Austrian START programme. The START programme is one of Austria’s most prestigious research grants for individual researchers at post-doctoral level and provides the grantee with up to 1.2 million Euro for up to 5 years. The programme’s aims are twofold: supporting excellent research and qualifying the grantee for a (permanent) senior research position in the research system. The article discusses the effects of the programme and focuses especially on the impacts on the grantees as main beneficiaries. In particular, the scientific output of the grantees and their career development is investigated. Furthermore, the analysis of the indirect beneficiary groups and the analysis of the system in which the START programme is placed, aims at answering the questions whether and how the START programme has contributed to strengthening the capabilities of the Austrian science system. The evaluation uses a control group approach to quantify the effects on the grantees. In order to counterbalance the weaknesses of traditional quantitative impact analysis and to obtain a deeper understanding of the mechanisms of the effects of the funding, the evaluation was complemented by further evidence of a qualitative and quantitative nature.

Suggested Citation

  • Sarah Seus & Susanne Bührer, 2022. "The evaluation of the Austrian START programme: an impact analysis of a research funding programme using a multi-method approach," The Journal of Technology Transfer, Springer, vol. 47(3), pages 673-698, June.
    • Handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9606-8
    DOI: 10.1007/s10961-017-9606-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10961-017-9606-8
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10961-017-9606-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Éric Archambault & Étienne Vignola-Gagné & Grégoire Côté & Vincent Larivière & Yves Gingrasb, 2006. "Benchmarking scientific output in the social sciences and humanities: The limits of existing databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 329-342, September.
    2. Stefan Hornbostel & Susan Böhmer & Bernd Klingsporn & Jörg Neufeld & Markus Ins, 2009. "Funding of young scientist and scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(1), pages 171-190, April.
    3. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    4. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    5. Carolina Cañibano & Barry Bozeman, 2009. "Curriculum vitae method in science policy and research evaluation: the state-of-the-art," Research Evaluation, Oxford University Press, vol. 18(2), pages 86-94, June.
    6. David Roessner, 2000. "Quantitative and qualitative methods and measures in the evaluation of research," Research Evaluation, Oxford University Press, vol. 9(2), pages 125-132, August.
    7. Sander Gerritsen & Karen van der Wiel & Erik Plug, 2013. "Up or out? How individual research grants affect academic careers in the Netherlands," CPB Discussion Paper 249, CPB Netherlands Bureau for Economic Policy Analysis.
    8. Göran Melin & Rickard Danell, 2006. "The top eight percent: Development of approved and rejected applicants for a prestigious grant in Sweden," Science and Public Policy, Oxford University Press, vol. 33(10), pages 702-712, December.
    9. Conchi, Sonia & Michels, Carolin, 2014. "Scientific mobility: An analysis of Germany, Austria, France and Great Britain," Discussion Papers "Innovation Systems and Policy Analysis" 41, Fraunhofer Institute for Systems and Innovation Research (ISI).
    10. Sander Gerritsen & Karen van der Wiel & Erik Plug, 2013. "Up or out? How individual research grants affect academic careers in the Netherlands," CPB Discussion Paper 249.rdf, CPB Netherlands Bureau for Economic Policy Analysis.
    11. Barbara Befani & John Mayne, 2014. "Process Tracing and Contribution Analysis: A Combined Approach to Generative Causal Inference for Impact Evaluation," IDS Bulletin, Blackwell Publishing, vol. 45(6), pages 17-36, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aboal, Diego & Tacsir, Ezequiel, 2016. "The impact of ex-ante subsidies to researchers on researcher's productivity: Evidence from a developing country," MERIT Working Papers 2016-019, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    2. Youtie, Jan & Rogers, Juan & Heinze, Thomas & Shapira, Philip & Tang, Li, 2013. "Career-based influences on scientific recognition in the United States and Europe: Longitudinal evidence from curriculum vitae data," Research Policy, Elsevier, vol. 42(8), pages 1341-1355.
    3. Mladen Djuric & Marina Dobrota & Jovan Filipovic, 2020. "Complexity-based quality indicators for human and social capital in science and research: the case of Serbian Homeland versus Diaspora," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 303-328, July.
    4. Baruffaldi, Stefano H. & Marino, Marianna & Visentin, Fabiana, 2020. "Money to move: The effect on researchers of an international mobility grant," Research Policy, Elsevier, vol. 49(8).
    5. Ekaterina L. Dyachenko, 2014. "Internationalization of academic journals: Is there still a gap between social and natural sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 241-255, October.
    6. Tóth, Tamás & Demeter, Márton & Csuhai, Sándor & Major, Zsolt Balázs, 2024. "When career-boosting is on the line: Equity and inequality in grant evaluation, productivity, and the educational backgrounds of Marie Skłodowska-Curie Actions individual fellows in social sciences an," Journal of Informetrics, Elsevier, vol. 18(2).
    7. Andrea Bonaccorsi & Cinzia Daraio & Stefano Fantoni & Viola Folli & Marco Leonetti & Giancarlo Ruocco, 2017. "Do social sciences and humanities behave like life and hard sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 607-653, July.
    8. Star X. Zhao & Wen Lou & Alice M. Tan & Shuang Yu, 2018. "Do funded papers attract more usage?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 153-168, April.
    9. James Cunningham & Paul O'Reilly, 2019. "Roles and Responsibilities of Project Coordinators: A Contingency Model for Project Coordinator Effectiveness," JRC Research Reports JRC117576, Joint Research Centre.
    10. Kevin W. Boyack & Caleb Smith & Richard Klavans, 2018. "Toward predicting research proposal success," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 449-461, February.
    11. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    12. Marielle Non & Jeroen van Honk & Vince van Houten & Inge van der Weijden & Thed van Leeuwen, 2022. "Getting off to a flying start? The effects of an early-career international mobility grant on scientific performance," CPB Discussion Paper 443, CPB Netherlands Bureau for Economic Policy Analysis.
    13. Saïd Echchakoui, 2020. "Why and how to merge Scopus and Web of Science during bibliometric analysis: the case of sales force literature from 1912 to 2019," Journal of Marketing Analytics, Palgrave Macmillan, vol. 8(3), pages 165-184, September.
    14. Saïd Echchakoui, 0. "Why and how to merge Scopus and Web of Science during bibliometric analysis: the case of sales force literature from 1912 to 2019," Journal of Marketing Analytics, Palgrave Macmillan, vol. 0, pages 1-20.
    15. Danielle Lee, 2023. "Bibliometric analysis of Asian ‘language and linguistics’ research: A case of 13 countries," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-23, December.
    16. Pei-Shan Chi, 2014. "Which role do non-source items play in the social sciences? A case study in political science in Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1195-1213, November.
    17. Adriana Bin & Sergio Salles-Filho & Ana Carolina Spatti & Jesús Pascual Mena-Chalco & Fernando Antonio Basile Colugnati, 2022. "How much does a Ph.D. scholarship program impact an emerging economy research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6935-6960, December.
    18. Ekaterina Dyachenko, 2013. "Internationalization of academic journals: is there still a gap between social and natural sciences?," HSE Working papers WP BRP 28/HUM/2013, National Research University Higher School of Economics.
    19. Richard Woolley & Jordi Molas-Gallart, 2023. "Research impact seen from the user side," Research Evaluation, Oxford University Press, vol. 32(3), pages 591-602.
    20. Genowefa Blundo-Canto & Bernard Triomphe & Guy Faure & Danielle Barret & Aurelle de Romemont & Etienne Hainzelin, 2019. "Building a culture of impact in an international agricultural research organization: Process and reflective learning," Research Evaluation, Oxford University Press, vol. 28(2), pages 136-144.

    More about this item

    Keywords

    Programme evaluation; Impact analysis; Science policy; Research funding; Mixed methods approach; Bibliometric analysis;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9606-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.