IDEAS home Printed from https://ideas.repec.org/a/spr/jbecon/v92y2022i8d10.1007_s11573-022-01099-4.html
   My bibliography  Save this article

The impact of selected input and output factors on measuring research efficiency of university research fields: insights from a purpose-, field-, and method-specific perspective

Author

Listed:
  • Heinz Ahn

    (Technische Universität Braunschweig)

  • Marcel Clermont

    (Cooperative University Gera-Eisenach)

  • Julia Langner

    (Technische Universität Braunschweig)

Abstract

The measurement of universities’ efficiency is gaining importance due to the increasing competitive pressure. As the results of such a measurement depend especially on the included input and output factors, it is essential to select them carefully to generate valid results. Against this background, we analyze the impact of including selected input and output factors on measuring the research efficiency of university research fields by applying best-practice frontier benchmarking methods. Based on a standard scenario, we consider the additional inclusion of expenditures, research grants, and bibliometric indicators. In doing so, we take account of different research fields as well as methods and purposes of measuring efficiency. Regarding the latter aspect, we distinguish between the computed efficiency degrees and the resulting ranking. We observe that the additional input and output factors have a quite heterogeneous impact on the respective evaluation criteria. The corresponding findings are relatively robust with respect to the research fields and methods considered. Our results provide guidance for evaluators as well as university and political decision-makers.

Suggested Citation

  • Heinz Ahn & Marcel Clermont & Julia Langner, 2022. "The impact of selected input and output factors on measuring research efficiency of university research fields: insights from a purpose-, field-, and method-specific perspective," Journal of Business Economics, Springer, vol. 92(8), pages 1303-1335, October.
    • Handle: RePEc:spr:jbecon:v:92:y:2022:i:8:d:10.1007_s11573-022-01099-4
    DOI: 10.1007/s11573-022-01099-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11573-022-01099-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11573-022-01099-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Johnes, Geraint & Johnes, Jill, 1993. "Measuring the Research Performance of UK Economics Departments: An Application of Data Envelopment Analysis," Oxford Economic Papers, Oxford University Press, vol. 45(2), pages 332-347, April.
    2. Rafael Ball & Bernhard Mittermaier & Dirk Tunger, 2009. "Creation of journal-based publication profiles of scientific institutions — A methodology for the interdisciplinary comparison of scientific research based on the J-factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 381-392, November.
    3. Marcel Clermont & Alexander Dirksen & Barbara Scheidt & Dirk Tunger, 2017. "Citation metrics as an additional indicator for evaluating research performance? An analysis of their correlations and validity," Business Research, Springer;German Academic Association for Business Research, vol. 10(2), pages 249-279, October.
    4. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    5. Ghimire, Sarad & Amin, Saman Hassanzadeh & Wardley, Leslie J., 2021. "Developing new data envelopment analysis models to evaluate the efficiency in Ontario Universities," Journal of Informetrics, Elsevier, vol. 15(3).
    6. Marcel Clermont, 2016. "Effectiveness and efficiency of research in Germany over time: an analysis of German business schools between 2001 and 2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1347-1381, September.
    7. Rolf Ketzler & Klaus F. Zimmermann, 2013. "A citation-analysis of economic research institutes," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1095-1112, June.
    8. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    9. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    10. Marcel Clermont & Johanna Krolak & Dirk Tunger, 2021. "Does the citation period have any effect on the informative value of selected citation indicators in research evaluations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1019-1047, February.
    11. Olson, Jerome A. & Schmidt, Peter & Waldman, Donald M., 1980. "A Monte Carlo study of estimators of stochastic frontier production functions," Journal of Econometrics, Elsevier, vol. 13(1), pages 67-82, May.
    12. Lee, Boon L. & Worthington, Andrew C., 2016. "A network DEA quantity and quality-orientated production model: An application to Australian university research services," Omega, Elsevier, vol. 60(C), pages 26-33.
    13. Gralka, Sabine, 2018. "Stochastic frontier analysis in higher education: A systematic review," CEPIE Working Papers 05/18, Technische Universität Dresden, Center of Public and International Economics (CEPIE).
    14. Meeusen, Wim & van den Broeck, Julien, 1977. "Efficiency Estimation from Cobb-Douglas Production Functions with Composed Error," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 18(2), pages 435-444, June.
    15. Klaus Wohlrabe & Elisabeth Friedrich, 2017. "The efficiency of economics departments reconsidered," Economics Bulletin, AccessEcon, vol. 37(3), pages 1602-1611.
    16. Beasley, J. E., 1990. "Comparing university departments," Omega, Elsevier, vol. 18(2), pages 171-183.
    17. Melville L. McMillan & Debasish Datta, 1998. "The Relative Efficiencies of Canadian Universities: A DEA Perspective," Canadian Public Policy, University of Toronto Press, vol. 24(4), pages 485-511, December.
    18. Bielecki, Andre & Albers, Sönke, 2012. "Eine Analyse der Forschungseffizienz deutscher betriebswirtschaftlicher Fachbereiche basierend auf den Daten des Centrums für Hochschulentwicklung (CHE)," EconStor Preprints 57429, ZBW - Leibniz Information Centre for Economics.
    19. Mark Andor & Frederik Hesse, 2014. "The StoNED age: the departure into a new era of efficiency analysis? A monte carlo comparison of StoNED and the “oldies” (SFA and DEA)," Journal of Productivity Analysis, Springer, vol. 41(1), pages 85-109, February.
    20. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2013. "Multilevel‐statistical reformulation of citation‐based university rankings: The Leiden ranking 2011/2012," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(8), pages 1649-1658, August.
    21. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    22. Eva M. de la Torre & Tommaso Agasisti & Carmen Perez-Esparrells, 2017. "The relevance of knowledge transfer for universities’ efficiency scores: an empirical approximation on the Spanish public higher education system," Research Evaluation, Oxford University Press, vol. 26(3), pages 211-229.
    23. Harald Dyckhoff, 2018. "Multi-criteria production theory: foundation of non-financial and sustainability performance evaluation," Journal of Business Economics, Springer, vol. 88(7), pages 851-882, September.
    24. Henningsen, Arne & Bělín, Matěj & Henningsen, Géraldine, 2017. "New insights into the stochastic ray production frontier," Economics Letters, Elsevier, vol. 156(C), pages 18-21.
    25. Andrea Bonaccorsi & Cinzia Daraio & Léopold Simar, 2006. "Advanced indicators of productivity of universitiesAn application of robust nonparametric methods to Italian data," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(2), pages 389-410, February.
    26. Jondrow, James & Knox Lovell, C. A. & Materov, Ivan S. & Schmidt, Peter, 1982. "On the estimation of technical inefficiency in the stochastic frontier production function model," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 233-238, August.
    27. Giovanni Abramo & Ciriaco Andrea D'Angelo, 2009. "Assessing technical and cost efficiency of research activities: a case study of the Italian university system," Research Evaluation, Oxford University Press, vol. 18(1), pages 61-70, March.
    28. Battese, George E. & Coelli, Tim J., 1988. "Prediction of firm-level technical efficiencies with a generalized frontier production function and panel data," Journal of Econometrics, Elsevier, vol. 38(3), pages 387-399, July.
    29. Liang Zhang & Wei Bao & Liang Sun, 2016. "Resources and Research Production in Higher Education: A Longitudinal Analysis of Chinese Universities, 2000–2010," Research in Higher Education, Springer;Association for Institutional Research, vol. 57(7), pages 869-891, November.
    30. Sunil Kumar Jauhar & Millie Pant & Richa Dutt, 2018. "Performance measurement of an Indian higher education institute: a sustainable educational supply chain management perspective," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 9(1), pages 180-193, February.
    31. Julia Schaefer & Marcel Clermont, 2018. "Stochastic non-smooth envelopment of data for multi-dimensional output," Journal of Productivity Analysis, Springer, vol. 50(3), pages 139-154, December.
    32. Johnes, Jill & Johnes, Geraint, 1995. "Research funding and performance in U.K. University Departments of Economics: A frontier analysis," Economics of Education Review, Elsevier, vol. 14(3), pages 301-314, September.
    33. Johnes, Geraint, 1997. "Costs and Industrial Structure in Contemporary British Higher Education," Economic Journal, Royal Economic Society, vol. 107(442), pages 727-737, May.
    34. Gianni De Fraja & Giovanni Facchini & John Gathergood, 2019. "Academic salaries and public evaluation of university research: Evidence from the UK Research Excellence Framework," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 34(99), pages 523-583.
    35. Wang, Derek D., 2019. "Performance-based resource allocation for higher education institutions in China," Socio-Economic Planning Sciences, Elsevier, vol. 65(C), pages 66-75.
    36. Tommaso Agasisti & Giuseppe Catalano & Paolo Landoni & Roberto Verganti, 2012. "Evaluating the performance of academic departments: an analysis of research-related output efficiency," Research Evaluation, Oxford University Press, vol. 21(1), pages 2-14, February.
    37. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    38. Peter Bogetoft & Lars Otto, 2011. "Benchmarking with DEA, SFA, and R," International Series in Operations Research and Management Science, Springer, number 978-1-4419-7961-2, April.
    39. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    40. Geraint Johnes, 1995. "Scale and technical efficiency in the production of economic research," Applied Economics Letters, Taylor & Francis Journals, vol. 2(1), pages 7-11.
    41. Mammadov, Rza & Aypay, Ahmet, 2020. "Efficiency analysis of research universities in Turkey," International Journal of Educational Development, Elsevier, vol. 75(C).
    42. Timo Kuosmanen & Mika Kortelainen, 2012. "Stochastic non-smooth envelopment of data: semi-parametric frontier estimation subject to shape constraints," Journal of Productivity Analysis, Springer, vol. 38(1), pages 11-28, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ahn, Heinz & Clermont, Marcel & Langner, Julia, 2023. "Comparative performance analysis of frontier-based efficiency measurement methods – A Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 307(1), pages 294-312.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ahn, Heinz & Clermont, Marcel & Langner, Julia, 2023. "Comparative performance analysis of frontier-based efficiency measurement methods – A Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 307(1), pages 294-312.
    2. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    3. Julia Schaefer & Marcel Clermont, 2018. "Stochastic non-smooth envelopment of data for multi-dimensional output," Journal of Productivity Analysis, Springer, vol. 50(3), pages 139-154, December.
    4. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    5. Calogero Guccio & Marco Ferdinando Martorana & Luisa Monaco, 2016. "Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: a non-parametric frontier approach," Journal of Productivity Analysis, Springer, vol. 45(3), pages 275-298, June.
    6. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    7. Renato A. Villano & Carolyn‐Dung T. T. Tran, 2021. "Survey on technical efficiency in higher education: A meta‐fractional regression analysis," Pacific Economic Review, Wiley Blackwell, vol. 26(1), pages 110-135, February.
    8. Andor, Mark A. & Parmeter, Christopher & Sommer, Stephan, 2019. "Combining uncertainty with uncertainty to get certainty? Efficiency analysis for regulation purposes," European Journal of Operational Research, Elsevier, vol. 274(1), pages 240-252.
    9. Amara, Nabil & Rhaiem, Mehdi & Halilem, Norrin, 2020. "Assessing the research efficiency of Canadian scholars in the management field: Evidence from the DEA and fsQCA," Journal of Business Research, Elsevier, vol. 115(C), pages 296-306.
    10. Mark Andor & Frederik Hesse, 2014. "The StoNED age: the departure into a new era of efficiency analysis? A monte carlo comparison of StoNED and the “oldies” (SFA and DEA)," Journal of Productivity Analysis, Springer, vol. 41(1), pages 85-109, February.
    11. Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2018. "Does econometric methodology matter to rank universities? An analysis of Italian higher education system," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 104-120.
    12. Agasisti, Tommaso & Barra, Cristian & Zotti, Roberto, 2016. "Evaluating the efficiency of Italian public universities (2008–2011) in presence of (unobserved) heterogeneity," Socio-Economic Planning Sciences, Elsevier, vol. 55(C), pages 47-58.
    13. Alexander Arévalo S & Víctor Giménez G & Diego Prior J, 2022. "Análisis de eficiencia en educación: una aplicación del método StoNED," Revista Desarrollo y Sociedad, Universidad de los Andes,Facultad de Economía, CEDE, vol. 92(2), pages 45-91, October.
    14. Vanesa D’Elia & Gustavo Ferro, 2019. "Empirical Efficiency Measurement in Higher Education: An Overview," CEMA Working Papers: Serie Documentos de Trabajo. 708, Universidad del CEMA.
    15. Marcel Clermont & Julia Schaefer, 2019. "Identification of Outliers in Data Envelopment Analysis," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 71(4), pages 475-496, October.
    16. Mishra, Neelesh Kumar & Chakraborty, Abhishek & Singh, Sanjeet & Ranjan, Prabhat, 2023. "Efficiency analysis of engineering colleges in India: Decomposition into parallel sub-processes systems," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    17. Gralka, Sabine, 2018. "Stochastic frontier analysis in higher education: A systematic review," CEPIE Working Papers 05/18, Technische Universität Dresden, Center of Public and International Economics (CEPIE).
    18. Ma, Zhanxin & See, Kok Fong & Yu, Ming-Miin & Zhao, Chunying, 2021. "Research efficiency analysis of China's university faculty members: A modified meta-frontier DEA approach," Socio-Economic Planning Sciences, Elsevier, vol. 76(C).
    19. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    20. Martín Rivero, Raquel, 2007. "La eficiencia productiva en el ámbito universitario: aspectos claves para su evaluación/The Productive Efficiency in the University: Key Aspects for its Evaluation," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 25, pages 793-812, Diciembre.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jbecon:v:92:y:2022:i:8:d:10.1007_s11573-022-01099-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.