IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v24y1996i1p13-28.html
   My bibliography  Save this article

The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies

Author

Listed:
  • Doyle, J. R.
  • Arthurs, A. J.
  • Green, R. H.
  • McAulay, L.
  • Pitt, M. R.
  • Bottomley, P. A.
  • Evans, W.

Abstract

This paper examines the judgments (part of a multi-million pound government-sponsored resource allocation exercise) made by a panel of experts about the research rating of UK business schools during 1988-1992. We use policy capture to determine, and critically evaluate, how business schools were judged. We suggest methods to improve the process of judgment-principally, using DEA as an idealized model whereby the judged institutions judge themselves, consistent with policy constraints.

Suggested Citation

  • Doyle, J. R. & Arthurs, A. J. & Green, R. H. & McAulay, L. & Pitt, M. R. & Bottomley, P. A. & Evans, W., 1996. "The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies," Omega, Elsevier, vol. 24(1), pages 13-28, February.
    • Handle: RePEc:eee:jomega:v:24:y:1996:i:1:p:13-28
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/0305-0483(95)00044-5
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Johnes, Geraint & Johnes, Jill, 1993. "Measuring the Research Performance of UK Economics Departments: An Application of Data Envelopment Analysis," Oxford Economic Papers, Oxford University Press, vol. 45(2), pages 332-347, April.
    2. Jim Taylor, "undated". "Measuring Research Performance in Business Managment Studies in the United Kingdom: the 1992 Research Assessment Exercise," Working Papers ec15/94, Department of Economics, University of Lancaster.
    3. Per Andersen & Niels Christian Petersen, 1993. "A Procedure for Ranking Efficient Units in Data Envelopment Analysis," Management Science, INFORMS, vol. 39(10), pages 1261-1264, October.
    4. Jim Taylor, 1995. "A Statistical Analysis of the 1992 Research Assessment Exercise," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(2), pages 241-261, March.
    5. Martin Weber & Franz Eisenführ & Detlof von Winterfeldt, 1988. "The Effects of Splitting Attributes on Weights in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 34(4), pages 431-445, April.
    6. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    7. Russell G. Thompson & F. D. Singleton & Robert M. Thrall & Barton A. Smith, 1986. "Comparative Site Evaluations for Locating a High-Energy Physics Lab in Texas," Interfaces, INFORMS, vol. 16(6), pages 35-49, December.
    8. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    9. Doyle, J. R. & Arthurs, A. J., 1995. "Judging the quality of research in business schools: The UK as a case study," Omega, Elsevier, vol. 23(3), pages 257-270, June.
    10. Jill Johnes & Jim Taylor & Brian Francis, 1993. "The Research Performance of UK Universities: A Statistical Analysis of the Results of the 1989 Research Selectivity Exercise," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 156(2), pages 271-286, March.
    11. Muhittin Oral & Ossama Kettani & Pascal Lang, 1991. "A Methodology for Collective Evaluation and Selection of Industrial R&D Projects," Management Science, INFORMS, vol. 37(7), pages 871-885, July.
    12. Edwards, Ward, 1992. "Discussion: Of human skills," Organizational Behavior and Human Decision Processes, Elsevier, vol. 53(2), pages 267-277, November.
    13. Robert C. Blattberg & Stephen J. Hoch, 1990. "Database Models and Managerial Intuition: 50% Model + 50% Manager," Management Science, INFORMS, vol. 36(8), pages 887-899, August.
    14. Geraint Johnes, 1995. "Scale and technical efficiency in the production of economic research," Applied Economics Letters, Taylor & Francis Journals, vol. 2(1), pages 7-11.
    15. E. H. Bowman, 1963. "Consistency and Optimality in Managerial Decision Making," Management Science, INFORMS, vol. 9(2), pages 310-321, January.
    16. Doyle, John R., 1995. "Multiattribute Choice for the Lazy Decision Maker: Let the Alternatives Decide!," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(1), pages 87-100, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Doyle, John R & Arthurs, Alan J, 1998. "Grade inflation in the UK's 1996 research assessment exercise?," Omega, Elsevier, vol. 26(4), pages 461-465, August.
    2. Mitchell, George, 1996. "Judging research quality and journals: A call for debate," Omega, Elsevier, vol. 24(5), pages 613-613, October.
    3. Thanassoulis, E. & Sotiros, D. & Koronakos, G. & Despotis, D., 2018. "Assessing the cost-effectiveness of university academic recruitment and promotion policies," European Journal of Operational Research, Elsevier, vol. 264(2), pages 742-755.
    4. Colin Glass, J. & McCallion, Gillian & McKillop, Donal G. & Rasaratnam, Syamarlah & Stringer, Karl S., 2006. "Implications of variant efficiency measures for policy evaluations in UK higher education," Socio-Economic Planning Sciences, Elsevier, vol. 40(2), pages 119-142, June.
    5. Doyle, J. R. & Arthurs, A. J. & Mcaulay, L. & Osborne, P. G., 1996. "Citation as effortful voting: A reply to ones, Brinn and Pendlebury," Omega, Elsevier, vol. 24(5), pages 603-606, October.
    6. Donohue, Joan M. & Fox, Jeremy B., 2000. "A multi-method evaluation of journals in the decision and management sciences by US academics," Omega, Elsevier, vol. 28(1), pages 17-36, February.
    7. Michael Cain & Stuart McLeay, 2016. "Statistical Auditing of Non-transparent Expert Assessments," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 78(2), pages 362-385, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adler, Nicole & Friedman, Lea & Sinuany-Stern, Zilla, 2002. "Review of ranking methods in the data envelopment analysis context," European Journal of Operational Research, Elsevier, vol. 140(2), pages 249-265, July.
    2. Cook, Wade D. & Seiford, Larry M., 2009. "Data envelopment analysis (DEA) - Thirty years on," European Journal of Operational Research, Elsevier, vol. 192(1), pages 1-17, January.
    3. Sanjeet Singh & Prabhat Ranjan, 2018. "Efficiency analysis of non-homogeneous parallel sub-unit systems for the performance measurement of higher education," Annals of Operations Research, Springer, vol. 269(1), pages 641-666, October.
    4. Cook, Wade D. & Green, Rodney H., 2000. "Project prioritization: a resource-constrained data envelopment analysis approach," Socio-Economic Planning Sciences, Elsevier, vol. 34(2), pages 85-99, June.
    5. Beasley, J. E., 2003. "Allocating fixed costs and resources via data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 147(1), pages 198-216, May.
    6. Helmi Hammami & Thanh Ngo & David Tripe & Dinh-Tri Vo, 2022. "Ranking with a Euclidean common set of weights in data envelopment analysis: with application to the Eurozone banking sector," Annals of Operations Research, Springer, vol. 311(2), pages 675-694, April.
    7. Liu, Hui-hui & Song, Yao-yao & Liu, Xiao-xiao & Yang, Guo-liang, 2020. "Aggregating the DEA prospect cross-efficiency with an application to state key laboratories in China," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    8. Johnes, Jill, 1996. "Performance assessment in higher education in Britain," European Journal of Operational Research, Elsevier, vol. 89(1), pages 18-33, February.
    9. C Kao & H-T Hung, 2005. "Data envelopment analysis with common weights: the compromise solution approach," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(10), pages 1196-1203, October.
    10. Perrigot, Rozenn & Barros, Carlos Pestana, 2008. "Technical efficiency of French retailers," Journal of Retailing and Consumer Services, Elsevier, vol. 15(4), pages 296-305.
    11. Foroughi, A.A. & Tamiz, M., 2005. "An effective total ranking model for a ranked voting system," Omega, Elsevier, vol. 33(6), pages 491-496, December.
    12. Zhiying Zhang & Huchang Liao, 2024. "A stochastic cross-efficiency DEA approach based on the prospect theory and its application in winner determination in public procurement tenders," Annals of Operations Research, Springer, vol. 341(1), pages 509-537, October.
    13. Ghasemi, M.-R. & Ignatius, Joshua & Emrouznejad, Ali, 2014. "A bi-objective weighted model for improving the discrimination power in MCDEA," European Journal of Operational Research, Elsevier, vol. 233(3), pages 640-650.
    14. Chien-Ming Chen & Joe Zhu, 2011. "Efficient Resource Allocation via Efficiency Bootstraps: An Application to R&D Project Budgeting," Operations Research, INFORMS, vol. 59(3), pages 729-741, June.
    15. J. Glass & Donal McKillop & Gary O'Rourke, 1998. "A Cost Indirect Evaluation of Productivity Change in UK Universities," Journal of Productivity Analysis, Springer, vol. 10(2), pages 153-175, October.
    16. Ebrahimi, Bohlool & Dhamotharan, Lalitha & Ghasemi, Mohammad Reza & Charles, Vincent, 2022. "A cross-inefficiency approach based on the deviation variables framework," Omega, Elsevier, vol. 111(C).
    17. Thanassoulis, E. & Sotiros, D. & Koronakos, G. & Despotis, D., 2018. "Assessing the cost-effectiveness of university academic recruitment and promotion policies," European Journal of Operational Research, Elsevier, vol. 264(2), pages 742-755.
    18. Kanematsu, Simon Y. & Carvalho, Ney P. & Martinhon, Carlos A. & Almeida, Mariana R., 2020. "Ranking using η-efficiency and relative size measures based on DEA," Omega, Elsevier, vol. 90(C).
    19. Adler, Nicole & Yazhemsky, Ekaterina, 2010. "Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction," European Journal of Operational Research, Elsevier, vol. 202(1), pages 273-284, April.
    20. Martín Rivero, Raquel, 2007. "La eficiencia productiva en el ámbito universitario: aspectos claves para su evaluación/The Productive Efficiency in the University: Key Aspects for its Evaluation," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 25, pages 793-812, Diciembre.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:24:y:1996:i:1:p:13-28. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.