IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v24y1996i1p13-28.html
   My bibliography  Save this article

The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies

Author

Listed:
  • Doyle, J. R.
  • Arthurs, A. J.
  • Green, R. H.
  • McAulay, L.
  • Pitt, M. R.
  • Bottomley, P. A.
  • Evans, W.

Abstract

This paper examines the judgments (part of a multi-million pound government-sponsored resource allocation exercise) made by a panel of experts about the research rating of UK business schools during 1988-1992. We use policy capture to determine, and critically evaluate, how business schools were judged. We suggest methods to improve the process of judgment-principally, using DEA as an idealized model whereby the judged institutions judge themselves, consistent with policy constraints.

Suggested Citation

  • Doyle, J. R. & Arthurs, A. J. & Green, R. H. & McAulay, L. & Pitt, M. R. & Bottomley, P. A. & Evans, W., 1996. "The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies," Omega, Elsevier, vol. 24(1), pages 13-28, February.
    • Handle: RePEc:eee:jomega:v:24:y:1996:i:1:p:13-28
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/0305-0483(95)00044-5
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Johnes, Geraint & Johnes, Jill, 1993. "Measuring the Research Performance of UK Economics Departments: An Application of Data Envelopment Analysis," Oxford Economic Papers, Oxford University Press, vol. 45(2), pages 332-347, April.
    2. Jim Taylor, "undated". "Measuring Research Performance in Business Managment Studies in the United Kingdom: the 1992 Research Assessment Exercise," Working Papers ec15/94, Department of Economics, University of Lancaster.
    3. Per Andersen & Niels Christian Petersen, 1993. "A Procedure for Ranking Efficient Units in Data Envelopment Analysis," Management Science, INFORMS, vol. 39(10), pages 1261-1264, October.
    4. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    5. Jim Taylor, 1995. "A Statistical Analysis of the 1992 Research Assessment Exercise," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(2), pages 241-261, March.
    6. Jill Johnes & Jim Taylor & Brian Francis, 1993. "The Research Performance of UK Universities: A Statistical Analysis of the Results of the 1989 Research Selectivity Exercise," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 156(2), pages 271-286, March.
    7. Muhittin Oral & Ossama Kettani & Pascal Lang, 1991. "A Methodology for Collective Evaluation and Selection of Industrial R&D Projects," Management Science, INFORMS, vol. 37(7), pages 871-885, July.
    8. Martin Weber & Franz Eisenführ & Detlof von Winterfeldt, 1988. "The Effects of Splitting Attributes on Weights in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 34(4), pages 431-445, April.
    9. Edwards, Ward, 1992. "Discussion: Of human skills," Organizational Behavior and Human Decision Processes, Elsevier, vol. 53(2), pages 267-277, November.
    10. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    11. Russell G. Thompson & F. D. Singleton & Robert M. Thrall & Barton A. Smith, 1986. "Comparative Site Evaluations for Locating a High-Energy Physics Lab in Texas," Interfaces, INFORMS, vol. 16(6), pages 35-49, December.
    12. Robert C. Blattberg & Stephen J. Hoch, 1990. "Database Models and Managerial Intuition: 50% Model + 50% Manager," Management Science, INFORMS, vol. 36(8), pages 887-899, August.
    13. Geraint Johnes, 1995. "Scale and technical efficiency in the production of economic research," Applied Economics Letters, Taylor & Francis Journals, vol. 2(1), pages 7-11.
    14. E. H. Bowman, 1963. "Consistency and Optimality in Managerial Decision Making," Management Science, INFORMS, vol. 9(2), pages 310-321, January.
    15. Doyle, J. R. & Arthurs, A. J., 1995. "Judging the quality of research in business schools: The UK as a case study," Omega, Elsevier, vol. 23(3), pages 257-270, June.
    16. Doyle, John R., 1995. "Multiattribute Choice for the Lazy Decision Maker: Let the Alternatives Decide!," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(1), pages 87-100, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Doyle, John R & Arthurs, Alan J, 1998. "Grade inflation in the UK's 1996 research assessment exercise?," Omega, Elsevier, vol. 26(4), pages 461-465, August.
    2. Thanassoulis, E. & Sotiros, D. & Koronakos, G. & Despotis, D., 2018. "Assessing the cost-effectiveness of university academic recruitment and promotion policies," European Journal of Operational Research, Elsevier, vol. 264(2), pages 742-755.
    3. Mitchell, George, 1996. "Judging research quality and journals: A call for debate," Omega, Elsevier, vol. 24(5), pages 613-613, October.
    4. Doyle, J. R. & Arthurs, A. J. & Mcaulay, L. & Osborne, P. G., 1996. "Citation as effortful voting: A reply to ones, Brinn and Pendlebury," Omega, Elsevier, vol. 24(5), pages 603-606, October.
    5. Colin Glass, J. & McCallion, Gillian & McKillop, Donal G. & Rasaratnam, Syamarlah & Stringer, Karl S., 2006. "Implications of variant efficiency measures for policy evaluations in UK higher education," Socio-Economic Planning Sciences, Elsevier, vol. 40(2), pages 119-142, June.
    6. Donohue, Joan M. & Fox, Jeremy B., 2000. "A multi-method evaluation of journals in the decision and management sciences by US academics," Omega, Elsevier, vol. 28(1), pages 17-36, February.
    7. Michael Cain & Stuart McLeay, 2016. "Statistical Auditing of Non-transparent Expert Assessments," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 78(2), pages 362-385, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adler, Nicole & Friedman, Lea & Sinuany-Stern, Zilla, 2002. "Review of ranking methods in the data envelopment analysis context," European Journal of Operational Research, Elsevier, vol. 140(2), pages 249-265, July.
    2. Cook, Wade D. & Green, Rodney H., 2000. "Project prioritization: a resource-constrained data envelopment analysis approach," Socio-Economic Planning Sciences, Elsevier, vol. 34(2), pages 85-99, June.
    3. Cook, Wade D. & Seiford, Larry M., 2009. "Data envelopment analysis (DEA) - Thirty years on," European Journal of Operational Research, Elsevier, vol. 192(1), pages 1-17, January.
    4. Sanjeet Singh & Prabhat Ranjan, 2018. "Efficiency analysis of non-homogeneous parallel sub-unit systems for the performance measurement of higher education," Annals of Operations Research, Springer, vol. 269(1), pages 641-666, October.
    5. Beasley, J. E., 2003. "Allocating fixed costs and resources via data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 147(1), pages 198-216, May.
    6. Helmi Hammami & Thanh Ngo & David Tripe & Dinh-Tri Vo, 2022. "Ranking with a Euclidean common set of weights in data envelopment analysis: with application to the Eurozone banking sector," Annals of Operations Research, Springer, vol. 311(2), pages 675-694, April.
    7. Liu, Hui-hui & Song, Yao-yao & Liu, Xiao-xiao & Yang, Guo-liang, 2020. "Aggregating the DEA prospect cross-efficiency with an application to state key laboratories in China," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    8. Alexandr Gedranovich & Mykhaylo Salnykov, 2012. "Productivity analysis of Belarusian higher education system," BEROC Working Paper Series 16, Belarusian Economic Research and Outreach Center (BEROC).
    9. Johnes, Jill, 1996. "Performance assessment in higher education in Britain," European Journal of Operational Research, Elsevier, vol. 89(1), pages 18-33, February.
    10. Karasakal, Esra & Aker, Pınar, 2017. "A multicriteria sorting approach based on data envelopment analysis for R&D project selection problem," Omega, Elsevier, vol. 73(C), pages 79-92.
    11. Cristian Barra & Roberto Zotti, 2016. "Measuring Efficiency in Higher Education: An Empirical Study Using a Bootstrapped Data Envelopment Analysis," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 22(1), pages 11-33, February.
    12. Liu, John S. & Lu, Wen-Min, 2010. "DEA and ranking with the network-based approach: a case of R&D performance," Omega, Elsevier, vol. 38(6), pages 453-464, December.
    13. Kong, Wei-Hsin & Fu, Tsu-Tan, 2012. "Assessing the performance of business colleges in Taiwan using data envelopment analysis and student based value-added performance indicators," Omega, Elsevier, vol. 40(5), pages 541-549.
    14. Obata, Tsuneshi & Ishii, Hiroaki, 2003. "A method for discriminating efficient candidates with ranked voting data," European Journal of Operational Research, Elsevier, vol. 151(1), pages 233-237, November.
    15. C Kao & H-T Hung, 2005. "Data envelopment analysis with common weights: the compromise solution approach," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(10), pages 1196-1203, October.
    16. Madjid Tavana & Mehdi Soltanifar & Francisco J. Santos-Arteaga, 2023. "Analytical hierarchy process: revolution and evolution," Annals of Operations Research, Springer, vol. 326(2), pages 879-907, July.
    17. Perrigot, Rozenn & Barros, Carlos Pestana, 2008. "Technical efficiency of French retailers," Journal of Retailing and Consumer Services, Elsevier, vol. 15(4), pages 296-305.
    18. Henriques, C.O. & Chavez, J.M. & Gouveia, M.C. & Marcenaro-Gutierrez, O.D., 2022. "Efficiency of secondary schools in Ecuador: A value based DEA approach," Socio-Economic Planning Sciences, Elsevier, vol. 82(PA).
    19. Hashimoto, Akihiro, 1997. "A ranked voting system using a DEA/AR exclusion model: A note," European Journal of Operational Research, Elsevier, vol. 97(3), pages 600-604, March.
    20. Foroughi, A.A. & Tamiz, M., 2005. "An effective total ranking model for a ranked voting system," Omega, Elsevier, vol. 33(6), pages 491-496, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:24:y:1996:i:1:p:13-28. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.