IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v128y2023i10d10.1007_s11192-023-04802-6.html
   My bibliography  Save this article

Do research performances of universities and disciplines in England converge or diverge? An assessment of the progress between research excellence frameworks in 2014 and 2021

Author

Listed:
  • Mehmet Pinar

    (Edge Hill University
    Universidad de Sevilla)

Abstract

Performance-based research funding systems (PBRFSs) have been used in selectively distributing research funding, increasing public money accountability and efficiency. Two recent such evaluations in England were called the Research Excellence Framework (REF), which took place in 2014 and 2021, and the research environment, outputs and impact of the research were evaluated. Even though various aspects of the REF were examined, there has been limited research on how the performance of the universities and disciplines changed between the two evaluation periods. This paper assesses whether there has been convergence or divergence in research quality across universities and subject areas between 2014 and 2021 and found that there was an absolute convergence between universities in all three research elements evaluated, and universities that performed relatively worse in REF in 2014 experienced higher growth in their performance between 2014 and 2021. There was also an absolute convergence in the research environment and impact across different subject areas, but there is no significant convergence in the quality of research outputs across disciplines. Our findings also highlight that there has been an absolute convergence in research quality within the universities (between different disciplines in a given university) and within disciplines (between universities in a given subject).

Suggested Citation

  • Mehmet Pinar, 2023. "Do research performances of universities and disciplines in England converge or diverge? An assessment of the progress between research excellence frameworks in 2014 and 2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5731-5766, October.
    • Handle: RePEc:spr:scient:v:128:y:2023:i:10:d:10.1007_s11192-023-04802-6
    DOI: 10.1007/s11192-023-04802-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04802-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04802-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lessmann, Christian & Seidel, André, 2017. "Regional inequality, convergence, and its determinants – A view from outer space," European Economic Review, Elsevier, vol. 92(C), pages 110-132.
    2. Erik E. Lehmann & Matthias Menter, 2016. "University–industry collaboration and regional wealth," The Journal of Technology Transfer, Springer, vol. 41(6), pages 1284-1307, December.
    3. Gibbons, Stephen & Neumayer, Eric & Perkins, Richard, 2015. "Student satisfaction, league tables and university applications: Evidence from Britain," Economics of Education Review, Elsevier, vol. 48(C), pages 148-164.
    4. Thomas Zacharewicz & Benedetto Lepori & Emanuela Reale & Koen Jonkers, 2019. "Performance-based research funding in EU Member States—a comparative assessment," Science and Public Policy, Oxford University Press, vol. 46(1), pages 105-115.
    5. Mehmet Pinar & Emre Unlu, 2020. "Determinants of quality of research environment: An assessment of the environment submissions in the UK’s Research Excellence Framework in 2014," Research Evaluation, Oxford University Press, vol. 29(3), pages 231-244.
    6. Kounetas, Konstantinos Elias, 2018. "Energy consumption and CO2 emissions convergence in European Union member countries. A tonneau des Danaides?," Energy Economics, Elsevier, vol. 69(C), pages 111-127.
    7. Robert A. Buckle & John Creedy & Norman Gemmell, 2020. "Is external research assessment associated with convergence or divergence of research quality across universities and disciplines? Evidence from the PBRF process in New Zealand," Applied Economics, Taylor & Francis Journals, vol. 52(36), pages 3919-3932, July.
    8. Ganong, Peter & Shoag, Daniel, 2017. "Why has regional income convergence in the U.S. declined?," Journal of Urban Economics, Elsevier, vol. 102(C), pages 76-90.
    9. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    10. Stijn Broecke, 2015. "University rankings: do they matter in the UK?," Education Economics, Taylor & Francis Journals, vol. 23(2), pages 137-161, April.
    11. Quah, D., 1990. "Galton'S Fallacy And The Tests Of The Convergence Hypothesis," Working papers 552, Massachusetts Institute of Technology (MIT), Department of Economics.
    12. Alonso Rodríguez-Navarro & Ricardo Brito, 2022. "The link between countries’ economic and scientific wealth has a complex dependence on technological activity and research policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2871-2896, May.
    13. Robert M. Solow, 1956. "A Contribution to the Theory of Economic Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 70(1), pages 65-94.
    14. Joonha Jeon & So Young Kim, 2018. "Is the gap widening among universities? On research output inequality and its measurement in the Korean higher education system," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(2), pages 589-606, March.
    15. Elham Erfanian & Amir B. Ferreira Neto, 2017. "Scientific output: labor or capital intensive? An analysis for selected countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 461-482, July.
    16. Slavo Radosevic & Esin Yoruk, 2014. "Are there global shifts in the world science base? Analysing the catching up and falling behind of world regions," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1897-1924, December.
    17. Štěpán Jurajda & Stanislav Kozubek & Daniel Münich & Samuel Škoda, 2017. "Scientific publication performance in post-communist countries: still lagging far behind," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 315-328, July.
    18. Domingo Docampo & Jean-Jacques Bessoule, 2019. "A new approach to the analysis and evaluation of the research output of countries and institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1207-1225, May.
    19. Bárbara S. Lancho-Barrantes & Francisco J. Cantú-Ortiz, 2019. "Science in Mexico: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 499-517, February.
    20. R. D. Shelton, 2020. "Scientometric laws connecting publication counts to national research funding," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 181-206, April.
    21. Claire Callender & Geoff Mason, 2017. "Does Student Loan Debt Deter Higher Education Participation? New Evidence from England," The ANNALS of the American Academy of Political and Social Science, , vol. 671(1), pages 20-48, May.
    22. Xu, Xu & Reed, Markum, 2021. "The impact of internet access on research output - a cross-country study," Information Economics and Policy, Elsevier, vol. 56(C).
    23. Tuan V. Nguyen & Thao P. Ho-Le & Ut V. Le, 2017. "International collaboration in scientific research in Vietnam: an analysis of patterns and impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 1035-1051, February.
    24. Christoph Emanuel Mueller, 2016. "Accurate forecast of countries’ research output by macro-level indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1307-1328, November.
    25. N. Gregory Mankiw & David Romer & David N. Weil, 1992. "A Contribution to the Empirics of Economic Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(2), pages 407-437.
    26. Bishop, Kate & D'Este, Pablo & Neely, Andy, 2011. "Gaining from interactions with universities: Multiple methods for nurturing absorptive capacity," Research Policy, Elsevier, vol. 40(1), pages 30-40, February.
    27. Pablo D'Este & Frederick Guy & Simona Iammarino, 2013. "Shaping the formation of university--industry research collaborations: what type of proximity does really matter?," Journal of Economic Geography, Oxford University Press, vol. 13(4), pages 537-558, July.
    28. Mehmet Pinar, 2020. "It is not all about performance: Importance of the funding formula in the allocation of performance-based research funding in England," Research Evaluation, Oxford University Press, vol. 29(1), pages 100-119.
    29. Daniele Checchi & Irene Mazzotta & Sandro Momigliano & Francesco Olivanti, 2020. "Convergence or polarisation? The impact of research assessment exercises in the Italian case," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1439-1455, August.
    30. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2021. "The different responses of universities to introduction of performance-based research funding [How Incentives Trickle down: Local Use of a National Bibliometric Indicator System]," Research Evaluation, Oxford University Press, vol. 30(4), pages 514-528.
    31. Sarah de Rijcke & Paul F. Wouters & Alex D. Rushforth & Thomas P. Franssen & Björn Hammarfelt, 2016. "Evaluation practices and effects of indicator use—a literature review," Research Evaluation, Oxford University Press, vol. 25(2), pages 161-169.
    32. Hewitt-Dundas, Nola, 2012. "Research intensity and knowledge transfer activity in UK universities," Research Policy, Elsevier, vol. 41(2), pages 262-275.
    33. Przemysław Korytkowski & Emanuel Kulczycki, 2019. "Examining how country-level science policy shapes publication patterns: the case of Poland," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1519-1543, June.
    34. Guerrero, Maribel & Cunningham, James A. & Urbano, David, 2015. "Economic impact of entrepreneurial universities’ activities: An exploratory study of the United Kingdom," Research Policy, Elsevier, vol. 44(3), pages 748-764.
    35. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    36. Civera, Alice & Lehmann, Erik E. & Paleari, Stefano & Stockinger, Sarah A.E., 2020. "Higher education policy: Why hope for quality when rewarding quantity?," Research Policy, Elsevier, vol. 49(8).
    37. Mehmet Pinar & Timothy J Horne, 2022. "Assessing research excellence: Evaluating the Research Excellence Framework," Research Evaluation, Oxford University Press, vol. 31(2), pages 173-187.
    38. Jesús Crespo Cuaresma & Stephan Klasen & Konstantin M. Wacker, 2022. "When Do We See Poverty Convergence?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(6), pages 1283-1301, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Banal-Estañol, Albert & Jofre-Bonet, Mireia & Iori, Giulia & Maynou, Laia & Tumminello, Michele & Vassallo, Pietro, 2023. "Performance-based research funding: Evidence from the largest natural experiment worldwide," Research Policy, Elsevier, vol. 52(6).
    2. Giliberto Capano & Benedetto Lepori, 2024. "Designing policies that could work: understanding the interaction between policy design spaces and organizational responses in public sector," Policy Sciences, Springer;Society of Policy Sciences, vol. 57(1), pages 53-82, March.
    3. Robert A. Buckle & John Creedy, 2022. "Methods to evaluate institutional responses to performance‐based research funding systems," Australian Economic Papers, Wiley Blackwell, vol. 61(3), pages 615-634, September.
    4. Liu, Tie-Ying & Lee, Chien-Chiang, 2020. "Convergence of the world’s energy use," Resource and Energy Economics, Elsevier, vol. 62(C).
    5. Sarah J. Carrington & Pablo Jiménez‐Ayora, 2021. "Shedding light on the convergence debate: Using luminosity data to investigate economic convergence in Ecuador," Review of Development Economics, Wiley Blackwell, vol. 25(1), pages 200-227, February.
    6. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    7. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    8. Kutuk, Yasin, 2022. "Inequality convergence: A world-systems theory approach," Structural Change and Economic Dynamics, Elsevier, vol. 63(C), pages 150-165.
    9. Vos, Rob & Frenkel, Roberto & Ocampo, José Antonio & Palma, José Gabriel & Marfán, Manuel & Ros, Jaime & Taylor, Lance & Correa, Nelson & Cimoli, Mario, 2005. "Beyond Reforms: Structural Dynamics and Macroeconomic Vulnerability," IDB Publications (Books), Inter-American Development Bank, number 347, November.
    10. Miketa, Asami & Mulder, Peter, 2005. "Energy productivity across developed and developing countries in 10 manufacturing sectors: Patterns of growth and convergence," Energy Economics, Elsevier, vol. 27(3), pages 429-453, May.
    11. Leone Leonida & Leone Leonida & Daniel Montolio, 2003. "Public Capital, Growth and Convergence in Spain. A Counterfactual Density Estimation Approach," Working Papers 2003/3, Institut d'Economia de Barcelona (IEB).
    12. William Easterly & Ross Levine, 2002. "It´s Not Factor Accumulation: Stylized Facts and Growth Models," Central Banking, Analysis, and Economic Policies Book Series, in: Norman Loayza & Raimundo Soto & Norman Loayza (Series Editor) & Klaus Schmidt-Hebbel (Series Editor) (ed.),Economic Growth: Sources, Trends, and Cycles, edition 1, volume 6, chapter 3, pages 061-114, Central Bank of Chile.
    13. Edy Yusuf Agung Gunanto & Tri Wahyu & Jaka Aminata & Banatul Hayati, 2021. "Convergence CO2 Emission in ASEAN Countries: Augmented Green Solow Model Approach," International Journal of Energy Economics and Policy, Econjournals, vol. 11(5), pages 572-578.
    14. Sulekha Hembram & Souparna Maji & Sushil Kr. Haldar, 2019. "Club Convergence among the Major Indian States During 1982–2014: Does Investment in Human Capital Matter?," South Asia Economic Journal, Institute of Policy Studies of Sri Lanka, vol. 20(2), pages 184-204, September.
    15. Cutrini, Eleonora & Mendez, Carlos, 2023. "Convergence clubs and spatial structural change in the European Union," Structural Change and Economic Dynamics, Elsevier, vol. 67(C), pages 167-181.
    16. William Brock & M. Taylor, 2010. "The Green Solow model," Journal of Economic Growth, Springer, vol. 15(2), pages 127-153, June.
    17. Mariusz Próchniak & Bartosz Witkowski, 2006. "Modelowanie realnej konwergencji w skali międzynarodowej," Gospodarka Narodowa. The Polish Journal of Economics, Warsaw School of Economics, issue 10, pages 1-31.
    18. Nicholas Apergis & Christina Christou & Stephen Miller, 2012. "Convergence patterns in financial development: evidence from club convergence," Empirical Economics, Springer, vol. 43(3), pages 1011-1040, December.
    19. Carmela Martin & Francisco J. Velazquez & Bernard Funck, 2001. "European Integration and Income Convergence : Lessons for Central and Eastern European Countries," World Bank Publications - Books, The World Bank Group, number 13968.
    20. Bruno, Giuseppe & De Bonis, Riccardo & Silvestrini, Andrea, 2012. "Do financial systems converge? New evidence from financial assets in OECD countries," Journal of Comparative Economics, Elsevier, vol. 40(1), pages 141-155.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:128:y:2023:i:10:d:10.1007_s11192-023-04802-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.