IDEAS home Printed from https://ideas.repec.org/p/pbs/ecofin/2023-08.html
   My bibliography  Save this paper

Does Performance-based Public Funding Pay off? UK’s Research Excellence Framework (REF) and Research Productivity

Author

Listed:
  • Ajab Khan

    (University of Portsmouth)

  • Ali Sina Önder

    (University of Portsmouth)

  • Sercan Özcan

    (University of Portsmouth)

Abstract

It is important to understand whether and in what ways performance-based public funds enhance universities’ research output. Using a synthetic difference-in-differences analysis and propensity score methods to compare research productivity differences between UK universities’ Economics and Business fields and their synthetic counterparts in the US during the REF period in contrast to the pre-REF period, we find a significant and consistent increase in peer-reviewed journal publications of UK universities since the introduction of REF in 2009. We further show that publication output per author declined as a result of increased collaboration while overall citations increased. Our findings further suggest that REF may have played a pivotal role in elevating research excellence in elite institutions.

Suggested Citation

  • Ajab Khan & Ali Sina Önder & Sercan Özcan, 2023. "Does Performance-based Public Funding Pay off? UK’s Research Excellence Framework (REF) and Research Productivity," Working Papers in Economics & Finance 2023-08, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    • Handle: RePEc:pbs:ecofin:2023-08
    as

    Download full text from publisher

    File URL: http://repec.port.ac.uk/EconFinance/PBSEconFin_2023_08.pdf
    File Function: Full text
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Thomas Zacharewicz & Benedetto Lepori & Emanuela Reale & Koen Jonkers, 2019. "Performance-based research funding in EU Member States—a comparative assessment," Science and Public Policy, Oxford University Press, vol. 46(1), pages 105-115.
    2. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    3. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    4. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    5. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    6. Uri Gneezy & Stephan Meier & Pedro Rey-Biel, 2011. "When and Why Incentives (Don't) Work to Modify Behavior," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 191-210, Fall.
    7. repec:clg:wpaper:2009-04 is not listed on IDEAS
    8. Alberto Abadie & Guido W. Imbens, 2016. "Matching on the Estimated Propensity Score," Econometrica, Econometric Society, vol. 84, pages 781-807, March.
    9. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    10. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    11. Geuna, Aldo & Piolatto, Matteo, 2016. "Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)," Research Policy, Elsevier, vol. 45(1), pages 260-271.
    12. Jacob, Brian A. & Lefgren, Lars, 2011. "The impact of research grant funding on scientific productivity," Journal of Public Economics, Elsevier, vol. 95(9), pages 1168-1177.
    13. A. Abigail Payne & Joanne Roberts, 2010. "Government Oversight of Public Universities: Are Centralized Performance Schemes Related to Increased Quantity or Quality?," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 207-212, February.
    14. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    15. Damian Clarke & Daniel Paila~nir & Susan Athey & Guido Imbens, 2023. "Synthetic Difference In Differences Estimation," Papers 2301.11859, arXiv.org, revised Feb 2023.
    16. Qin Zhang & Juneman Abraham & Hui-Zhen Fu, 2020. "Collaboration and its influence on retraction based on retracted publications during 1978–2017," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 213-232, October.
    17. Ginger Zhe Jin & Benjamin Jones & Susan Feng Lu & Brian Uzzi, 2019. "The Reverse Matthew Effect: Consequences of Retraction in Scientific Teams," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 492-506, July.
    18. Mingers, John & White, Leroy, 2010. "A review of the recent contribution of systems thinking to operational research and management science," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1147-1161, December.
    19. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    20. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Banal-Estañol, Albert & Jofre-Bonet, Mireia & Iori, Giulia & Maynou, Laia & Tumminello, Michele & Vassallo, Pietro, 2023. "Performance-based research funding: Evidence from the largest natural experiment worldwide," Research Policy, Elsevier, vol. 52(6).
    2. Ferman, Bruno, 2021. "Matching estimators with few treated and many control observations," Journal of Econometrics, Elsevier, vol. 225(2), pages 295-307.
    3. Cantner, Uwe & Grashof, Nils & Grebel, Thomas & Zhang, Xijie, 2023. "When Excellence is not Excellent: The Impact of the Excellence Initiative on the Relative Productivity of German Universities," MPRA Paper 118139, University Library of Munich, Germany.
    4. Zeqin Liu & Zongwu Cai & Ying Fang & Ming Lin, 2019. "Statistical Analysis and Evaluation of Macroeconomic Policies: A Selective Review," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 201904, University of Kansas, Department of Economics, revised Mar 2019.
    5. Kodjo Adandohoin & Vigninou Gammadigbe, 2022. "The revenue efficiency consequences of the announcement of a tax transition reform: The case of WAEMU countries," African Development Review, African Development Bank, vol. 34(S1), pages 195-218, July.
    6. Di Matteo, Dante & Mariotti, Ilaria & Rossi, Federica, 2023. "Transport infrastructure and economic performance: An evaluation of the Milan-Bologna high-speed rail corridor," Socio-Economic Planning Sciences, Elsevier, vol. 85(C).
    7. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    8. Aydemir, Abdurrahman B. & Kırdar, Murat G., 2017. "Quasi-experimental impact estimates of immigrant labor supply shocks: The role of treatment and comparison group matching and relative skill composition," European Economic Review, Elsevier, vol. 98(C), pages 282-315.
    9. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    10. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    11. Zhang, Cheng & Zhao, Ziwei & Wang, Qunwei & Xu, Bing, 2022. "Title: Holistic governance strategy to reduce carbon intensity," Technological Forecasting and Social Change, Elsevier, vol. 179(C).
    12. Yuret, Tolga, 2017. "Do researchers pay attention to publication subsidies?," Journal of Informetrics, Elsevier, vol. 11(2), pages 423-434.
    13. Gobillon, Laurent & Magnac, Thierry & Selod, Haris, 2012. "L’effet des Zones Franches Urbaines sur le Retour à l’Emploi," CEPREMAP Working Papers (Docweb) 1209, CEPREMAP.
    14. Fredriksen, Kaja & Runst, Petrik, 2018. "Are estimates of the "natural experiment" in the German crafts sector causal?," ifh Working Papers 16/2018, Volkswirtschaftliches Institut für Mittelstand und Handwerk an der Universität Göttingen (ifh).
    15. Ando, Michihito, 2013. "Estimating the effects of nuclear power facilities on local income levels: A quasi-experimental approach," Working Paper Series 2013:3, Uppsala University, Department of Economics.
    16. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 21104, Victoria University of Wellington, Chair in Public Finance.
    17. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    18. Selcuk Besir Demir, 2018. "Pros and cons of the new financial support policy for Turkish researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2053-2068, September.
    19. Jason Poulos & Andrea Albanese & Andrea Mercatanti & Fan Li, 2021. "Retrospective causal inference via matrix completion, with an evaluation of the effect of European integration on cross-border employment," Papers 2106.00788, arXiv.org.
    20. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.

    More about this item

    Keywords

    Public funding; Research Productivity; Research Excellence Frameworks (REFs);
    All these keywords.

    JEL classification:

    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • H52 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Education

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pbs:ecofin:2023-08. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Shuonan Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/depbsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.