IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v104y2024ics0149718924000314.html
   My bibliography  Save this article

Beyond the average effect of the innovation subsidies: Using case selection via matching to break impasse in delivering useful advice to policy makers

Author

Listed:
  • Koniewski, Maciej
  • Krupnik, Seweryn
  • Skórska, Paulina

Abstract

Experts and stakeholders promote the combined use of counterfactual and theory-based approaches in program evaluation. We illustrated combined application of these two approaches in a single evaluation study of innovation subsidies, using “case selection via matching” and follow-up in-depth interviews. We conducted interviews in contrasting pairs of companies—one successful and one unsuccessful—which were otherwise similar on a defined set of covariates. Our procedure helped to reveal factors, which might be overlooked or simply not available in data at hand and hence not accounted for in analyses of the intervention effects. As such it extends beyond the average effect estimate to highlight causes of an intervention success or failure.

Suggested Citation

  • Koniewski, Maciej & Krupnik, Seweryn & Skórska, Paulina, 2024. "Beyond the average effect of the innovation subsidies: Using case selection via matching to break impasse in delivering useful advice to policy makers," Evaluation and Program Planning, Elsevier, vol. 104(C).
    • Handle: RePEc:eee:epplan:v:104:y:2024:i:c:s0149718924000314
    DOI: 10.1016/j.evalprogplan.2024.102429
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718924000314
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2024.102429?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Howard White, 2009. "Theory-based impact evaluation: principles and practice," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 271-284.
    2. David Bartle & Michele Morris, 2010. "Evaluating the impacts of government business assistance programmes: approaches to testing additionality," Research Evaluation, Oxford University Press, vol. 19(4), pages 275-280, October.
    3. José Ángel Zúñiga-Vicente & César Alonso-Borrego & Francisco J. Forcadell & José I. Galán, 2014. "Assessing The Effect Of Public Subsidies On Firm R&D Investment: A Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 28(1), pages 36-67, February.
    4. Lieberman, Evan S., 2005. "Nested Analysis as a Mixed-Method Strategy for Comparative Research," American Political Science Review, Cambridge University Press, vol. 99(3), pages 435-452, August.
    5. Chen, Huey-Tsyh & Rossi, Peter H., 1989. "Issues in the theory-driven perspective," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 299-306, January.
    6. Dominique Guellec & Bruno Van Pottelsberghe De La Potterie, 2003. "The impact of public R&D expenditure on business R&D," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 12(3), pages 225-243.
    7. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    8. King, Gary & Nielsen, Richard, 2019. "Why Propensity Scores Should Not Be Used for Matching," Political Analysis, Cambridge University Press, vol. 27(4), pages 435-454, October.
    9. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    10. Spyros Arvanitis, 2013. "Micro-econometric approaches to the evaluation of technology-oriented public programmes: a non-technical review of the state of the art," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 3, pages 56-88, Edward Elgar Publishing.
    11. Lijphart, Arend, 1971. "Comparative Politics and the Comparative Method," American Political Science Review, Cambridge University Press, vol. 65(3), pages 682-693, September.
    12. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    13. Marino, Marianna & Lhuillery, Stephane & Parrotta, Pierpaolo & Sala, Davide, 2016. "Additionality or crowding-out? An overall evaluation of public R&D subsidy on private R&D expenditure," Research Policy, Elsevier, vol. 45(9), pages 1715-1730.
    14. Nanny Bressers & Lasse Gerrits, 2015. "A Complexity-Informed Approach to Evaluating National Knowledge and Innovation Programmes," Systems Research and Behavioral Science, Wiley Blackwell, vol. 32(1), pages 50-63, January.
    15. James J. Heckman, 1989. "Causal Inference and Nonrandom Samples," Journal of Educational and Behavioral Statistics, , vol. 14(2), pages 159-168, June.
    16. Falck, Oliver & Koenen, Johannes & Lohse, Tobias, 2019. "Evaluating a place-based innovation policy: Evidence from the innovative Regional Growth Cores Program in East Germany," Regional Science and Urban Economics, Elsevier, vol. 79(C).
    17. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    18. Michele Morris & Oliver J. Herrmann, 2013. "Beyond surveys: The research frontier moves to the use of administrative data to evaluate R&D grants," Research Evaluation, Oxford University Press, vol. 22(5), pages 298-306, October.
    19. Bracic, Ana, 2016. "Reaching the Individual: EU Accession, NGOs, and Human Rights," American Political Science Review, Cambridge University Press, vol. 110(3), pages 530-546, August.
    20. Madrigal, Róger & Alpízar, Francisco & Schlüter, Achim, 2011. "Determinants of Performance of Community-Based Drinking Water Organizations," World Development, Elsevier, vol. 39(9), pages 1663-1675, September.
    21. Wanzenböck, Iris & Scherngell, Thomas & Fischer, Manfred M., 2013. "How do firm characteristics affect behavioural additionalities of public R&D subsidies? Evidence for the Austrian transport sector," MPRA Paper 77552, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ugur, Mehmet & Trushin, Eshref, 2018. "Asymmetric information and heterogeneous effects of R&D subsidies: evidence on R&D investment and employment of R&D personel," Greenwich Papers in Political Economy 21943, University of Greenwich, Greenwich Political Economy Research Centre.
    2. Thomas H. W. Ziesemer, 2021. "The Effects of R&D Subsidies and Publicly Performed R&D on Business R&D: A Survey," Hacienda Pública Española / Review of Public Economics, IEF, vol. 236(1), pages 171-205, March.
    3. Michel Dumont, 2019. "Working Paper 04-19 - Tax incentives for business R&D in Belgium - Third evaluation," Working Papers 1904, Federal Planning Bureau, Belgium.
    4. Dumont, Michel, 2017. "Assessing the policy mix of public support to business R&D," Research Policy, Elsevier, vol. 46(10), pages 1851-1862.
    5. Lee, Jeongwon & Hwang, Junseok & Kim, Hana, 2022. "Different government support effects on emerging and mature ICT sectors," Technological Forecasting and Social Change, Elsevier, vol. 174(C).
    6. Cristiano Antonelli, 2020. "Knowledge exhaustibility public support to business R&D and the additionality constraint," The Journal of Technology Transfer, Springer, vol. 45(3), pages 649-663, June.
    7. Szücs, Florian, 2020. "Do research subsidies crowd out private R&D of large firms? Evidence from European Framework Programmes," Research Policy, Elsevier, vol. 49(3).
    8. Wanshu Wu & Kai Zhao & Lei Li, 2021. "Can government subsidy strategies and strategy combinations effectively stimulate enterprise innovation? Theory and evidence," Economia Politica: Journal of Analytical and Institutional Economics, Springer;Fondazione Edison, vol. 38(2), pages 423-446, July.
    9. Puggioni Daniela & Calderón Mariana & Cebreros Alfonso & Fernández León & Inguanzo José A. & Jaume David, 2022. "Inequality, Income Dynamics, and Transitions of Mexican Workers," Working Papers 2022-14, Banco de México.
    10. Liliana Gelabert & Martín Pereyra & Flavia Roldán, 2021. "Public support prevalence and innovation behavior. Uruguay 2007-2015," Documentos de Investigación 127, Universidad ORT Uruguay. Facultad de Administración y Ciencias Sociales.
    11. Mehmet Ugur & Eshref Trushin, 2023. "Information asymmetry, risk aversion and R&D subsidies: effect-size heterogeneity and policy conundrums," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 32(8), pages 1190-1215, November.
    12. Dosi, Giovanni & Lamperti, Francesco & Mazzucato, Mariana & Napoletano, Mauro & Roventini, Andrea, 2023. "Mission-oriented policies and the “Entrepreneurial State” at work: An agent-based exploration," Journal of Economic Dynamics and Control, Elsevier, vol. 151(C).
    13. Aalto, Eero & Gustafsson, Robin, 2020. "Innovation Promotion Rationales and Impacts – A Review," ETLA Reports 99, The Research Institute of the Finnish Economy.
    14. Peter Mayerhofer & Michael Klien, 2016. "Unternehmensinvestitionen in den österreichischen Bundesländern. Entwicklung – Struktur – Funktion regionaler Förderung," WIFO Studies, WIFO, number 61950.
    15. Huergo, Elena & Trenado, Mayte & Ubierna, Andrés, 2016. "The impact of public support on firm propensity to engage in R&D: Spanish experience," Technological Forecasting and Social Change, Elsevier, vol. 113(PB), pages 206-219.
    16. Blázquez, Maite & Herrarte, Ainhoa & Moro-Egido, Ana I., 2024. "Has the COVID-19 pandemic widened the gender gap in paid work hours in Spain?," Journal of Demographic Economics, Cambridge University Press, vol. 90(2), pages 313-348, June.
    17. Maarten Goos & Anna Salomons, 2017. "Measuring teaching quality in higher education: assessing selection bias in course evaluations," Research in Higher Education, Springer;Association for Institutional Research, vol. 58(4), pages 341-364, June.
    18. Carolina Caetano & Gregorio Caetano & Eric R. Nielsen, 2020. "Should Children Do More Enrichment Activities? Leveraging Bunching to Correct for Endogeneity," Finance and Economics Discussion Series 2020-036, Board of Governors of the Federal Reserve System (U.S.).
    19. Yidi Liu & Xin Li & Zhiqiang (Eric) Zheng, 2024. "Smart Natural Disaster Relief: Assisting Victims with Artificial Intelligence in Lending," Information Systems Research, INFORMS, vol. 35(2), pages 489-504, June.
    20. Emma Lappi, 2023. "Help from the past—coworker ties and entry wages after self-employment," Small Business Economics, Springer, vol. 60(3), pages 1171-1196, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:104:y:2024:i:c:s0149718924000314. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.