IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v131y2018icp18-30.html
   My bibliography  Save this article

Measuring organisational performance using a mix of OR methods

Author

Listed:
  • Santos, Sérgio P.
  • Belton, Valerie
  • Howick, Susan
  • Pilkington, Martin

Abstract

Performance measurement has become an increasingly important issue in recent years. In spite of the remarkable progress that has been achieved in this area of research, many performance measurement initiatives fall short of their potential in supporting decision-making. This paper argues that adopting a multi-method approach to assessing performance has the potential to result in more comprehensive and effective performance measurement systems. To support this assertion, the paper discusses the development of a performance measurement system for a Business Tax Department, which combined the use of several operational research (OR) techniques including qualitative system dynamics, data envelopment analysis and multiple criteria decision analysis. The use of these OR techniques was influential in developing and implementing the performance measurement system and has the potential to be transferred to other contexts.

Suggested Citation

  • Santos, Sérgio P. & Belton, Valerie & Howick, Susan & Pilkington, Martin, 2018. "Measuring organisational performance using a mix of OR methods," Technological Forecasting and Social Change, Elsevier, vol. 131(C), pages 18-30.
    • Handle: RePEc:eee:tefoso:v:131:y:2018:i:c:p:18-30
    DOI: 10.1016/j.techfore.2017.07.028
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162517310958
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2017.07.028?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Cooper, William W. & Ruiz, Jose L. & Sirvent, Inmaculada, 2007. "Choosing weights from alternative optimal solutions of dual multiplier models in DEA," European Journal of Operational Research, Elsevier, vol. 180(1), pages 443-458, July.
    2. Amado, Carla A.F. & Santos, Sérgio P. & Marques, Pedro M., 2012. "Integrating the Data Envelopment Analysis and the Balanced Scorecard approaches for enhanced performance assessment," Omega, Elsevier, vol. 40(3), pages 390-403.
    3. A Greasley, 2005. "Using DEA and simulation in guiding operating units to improved performance," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(6), pages 727-731, June.
    4. Howick, Susan & Eden, Colin & Ackermann, Fran & Williams, Terry, 2008. "Building confidence in models for multiple audiences: The modelling cascade," European Journal of Operational Research, Elsevier, vol. 186(3), pages 1068-1083, May.
    5. Tarja Joro & Pekka Korhonen & Jyrki Wallenius, 1998. "Structural Comparison of Data Envelopment Analysis and Multiple Objective Linear Programming," Management Science, INFORMS, vol. 44(7), pages 962-970, July.
    6. Desheng Wu, 2006. "Detecting information technology impact on firm performance using DEA and decision tree," International Journal of Information Technology and Management, Inderscience Enterprises Ltd, vol. 5(2/3), pages 162-174.
    7. Peter C. Smith & Maria Goddard, 2008. "Performance Management and Operational Research: A Marriage Made in Heaven?," Palgrave Macmillan Books, in: Richard Thorpe & Jacky Holloway (ed.), Performance Management, chapter 9, pages 120-136, Palgrave Macmillan.
    8. Fran Ackermann & Colin Eden & Terry Williams, 1997. "Modeling for Litigation: Mixing Qualitative and Quantitative Approaches," Interfaces, INFORMS, vol. 27(2), pages 48-65, April.
    9. Suwignjo, P. & Bititci, U. S & Carrie, A. S, 2000. "Quantitative models for performance measurement system," International Journal of Production Economics, Elsevier, vol. 64(1-3), pages 231-241, March.
    10. J Pollack, 2009. "Multimethodology in series and parallel: strategic planning using hard and soft OR," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(2), pages 156-167, February.
    11. F A F Ferreira & S P Santos & P M M Rodrigues, 2011. "Adding value to bank branch performance evaluation using cognitive maps and MCDA: a case study," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(7), pages 1320-1333, July.
    12. D Bouyssou, 1999. "Using DEA as a tool for MCDM: some remarks," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(9), pages 974-978, September.
    13. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    14. B Casu & D Shaw & E Thanassoulis, 2005. "Using a group support system to aid input–output identification in DEA," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(12), pages 1363-1372, December.
    15. M C Gouveia & L C Dias & C H Antunes, 2008. "Additive DEA based on MCDA with imprecise information," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(1), pages 54-63, January.
    16. Fernando A. F. Ferreira & Sérgio P. Santos & Paulo M. M. Rodrigues & Ronald W. Spahr, 2014. "How to create indices for bank branch financial performance measurement using MCDA techniques: an illustrative example," Journal of Business Economics and Management, Taylor & Francis Journals, vol. 15(4), pages 708-728, September.
    17. G Francis & J Holloway, 2002. "Beyond comparisons—the role for the operational researcher in benchmarking," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 53(3), pages 283-291, March.
    18. R. Allen & A. Athanassopoulos & R.G. Dyson & E. Thanassoulis, 1997. "Weights restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions," Annals of Operations Research, Springer, vol. 73(0), pages 13-34, October.
    19. S Howick, 2003. "Using system dynamics to analyse disruption and delay in complex projects for litigation: can the modelling purposes be met?," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(3), pages 222-229, March.
    20. Howick, Susan & Ackermann, Fran, 2011. "Mixing OR methods in practice: Past, present and future directions," European Journal of Operational Research, Elsevier, vol. 215(3), pages 503-511, December.
    21. W. Cooper & Shanling Li & L. Seiford & Kaoru Tone & R. Thrall & J. Zhu, 2001. "Sensitivity and Stability Analysis in DEA: Some Recent Developments," Journal of Productivity Analysis, Springer, vol. 15(3), pages 217-246, May.
    22. Tser‐Yieth Chen & Ling‐hua Chen, 2007. "DEA performance evaluation based on BSC indicators incorporated," International Journal of Productivity and Performance Management, Emerald Group Publishing Limited, vol. 56(4), pages 335-357, May.
    23. Fernando A. F. Ferreira & Sérgio P. Santos & Paulo M. M. Rodrigues & Ronald W. Spahr, 2014. "Evaluating retail banking service quality and convenience with MCDA techniques: a case study at the bank branch level," Journal of Business Economics and Management, Taylor & Francis Journals, vol. 15(1), pages 1-21, February.
    24. Jyoti & D.K. Banwet & S.G. Deshmukh, 2008. "Evaluating performance of national R&D organizations using integrated DEA‐AHP technique," International Journal of Productivity and Performance Management, Emerald Group Publishing Limited, vol. 57(5), pages 370-388, June.
    25. Mingers, John & Brocklesby, John, 1997. "Multimethodology: Towards a framework for mixing methodologies," Omega, Elsevier, vol. 25(5), pages 489-509, October.
    26. Henao, Felipe & Franco, L. Alberto, 2016. "Unpacking multimethodology: Impacts of a community development intervention," European Journal of Operational Research, Elsevier, vol. 253(3), pages 681-696.
    27. Pekka Korhonen & Mikko Syrjänen, 2004. "Resource Allocation Based on Efficiency Analysis," Management Science, INFORMS, vol. 50(8), pages 1134-1144, August.
    28. Jyrki Wallenius & James S. Dyer & Peter C. Fishburn & Ralph E. Steuer & Stanley Zionts & Kalyanmoy Deb, 2008. "Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead," Management Science, INFORMS, vol. 54(7), pages 1336-1349, July.
    29. J Mingers, 2003. "A classification of the philosophical assumptions of management science methods," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(6), pages 559-570, June.
    30. Yang, Jian-Bo & Wong, Brandon Y.H. & Xu, Dong-Ling & Stewart, Theodor J., 2009. "Integrating DEA-oriented performance assessment and target setting using interactive MOLP methods," European Journal of Operational Research, Elsevier, vol. 195(1), pages 205-222, May.
    31. P N de Almeida & L C Dias, 2012. "Value-based DEA models: application-driven developments," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 63(1), pages 16-27, January.
    32. S P Santos & V Belton & S Howick, 2008. "Enhanced performance measurement using OR: a case study," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(6), pages 762-775, June.
    33. Tseng, Fang-Mei & Chiu, Yu-Jing & Chen, Ja-Shen, 2009. "Measuring business performance in the high-tech manufacturing industry: A case study of Taiwan's large-sized TFT-LCD panel companies," Omega, Elsevier, vol. 37(3), pages 686-697, June.
    34. Merja Halme & Tarja Joro & Pekka Korhonen & Seppo Salo & Jyrki Wallenius, 1999. "A Value Efficiency Approach to Incorporating Preference Information in Data Envelopment Analysis," Management Science, INFORMS, vol. 45(1), pages 103-115, January.
    35. H Seol & H Lee & S Kim & Y Park, 2008. "The impact of information technology on organizational efficiency in public services: a DEA-based DT approach," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(2), pages 231-238, February.
    36. Robert G Dyson, 2000. "Strategy, performance and operational research," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 51(1), pages 5-11, January.
    37. André, Francisco J. & Herrero, Inés & Riesgo, Laura, 2010. "A modified DEA model to estimate the importance of objectives with an application to agricultural economics," Omega, Elsevier, vol. 38(5), pages 371-382, October.
    38. Mingers, John & Rosenhead, Jonathan, 2004. "Problem structuring methods in action," European Journal of Operational Research, Elsevier, vol. 152(3), pages 530-554, February.
    39. Jelle Van Camp & Johan Braet, 2016. "Taxonomizing performance measurement systems’ failures," International Journal of Productivity and Performance Management, Emerald Group Publishing Limited, vol. 65(5), pages 672-693, June.
    40. Antreas D. Athanassopoulos, 1998. "Decision Support for Target-Based Resource Allocation of Public Services in Multiunit and Multilevel Systems," Management Science, INFORMS, vol. 44(2), pages 173-187, February.
    41. Yong-bae Ji & Choonjoo Lee, 2010. "Data envelopment analysis," Stata Journal, StataCorp LP, vol. 10(2), pages 267-280, June.
    42. Morgan, Jennifer Sian & Howick, Susan & Belton, Valerie, 2017. "A toolkit of designs for mixing Discrete Event Simulation and System Dynamics," European Journal of Operational Research, Elsevier, vol. 257(3), pages 907-918.
    43. I Munro & J Mingers, 2002. "The use of multimethodology in practice—results of a survey of practitioners," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 53(4), pages 369-378, April.
    44. Sharma, Mithun J. & Yu, Song Jin, 2010. "Benchmark optimization and attribute identification for improvement of container terminals," European Journal of Operational Research, Elsevier, vol. 201(2), pages 568-580, March.
    45. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    46. Sarkis, Joseph, 2000. "A comparative analysis of DEA as a discrete alternative multiple criteria decision tool," European Journal of Operational Research, Elsevier, vol. 123(3), pages 543-557, June.
    47. M C Jackson, 1999. "Towards coherent pluralism in management science," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(1), pages 12-22, January.
    48. J Mingers & W Liu & W Meng, 2009. "Using SSM to structure the identification of inputs and outputs in DEA," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(2), pages 168-179, February.
    49. García-Valderrama, Teresa & Mulero-Mendigorri, Eva & Revuelta-Bordoy, Daniel, 2009. "Relating the perspectives of the balanced scorecard for R&D by means of DEA," European Journal of Operational Research, Elsevier, vol. 196(3), pages 1177-1189, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gomes, Luís S. & Santos, Sérgio P. & Coelho, Luís Serra & Rebelo, Efigénio L., 2023. "Using MCDA to assist an Intermunicipal community develop a resilience strategy in face of the pandemic caused by the SARS-CoV-2," Socio-Economic Planning Sciences, Elsevier, vol. 87(PB).
    2. Maruccia, Ylenia & Solazzo, Gianluca & Del Vecchio, Pasquale & Passiante, Giuseppina, 2020. "Evidence from Network Analysis application to Innovation Systems and Quintuple Helix," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    3. Ricciardi, Francesca & De Bernardi, Paola & Cantino, Valter, 2020. "System dynamics modeling as a circular process: The smart commons approach to impact management," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    4. Cao, Ting & Cook, Wade D. & Kristal, M. Murat, 2022. "Has the technological investment been worth it? Assessing the aggregate efficiency of non-homogeneous bank holding companies in the digital age," Technological Forecasting and Social Change, Elsevier, vol. 178(C).
    5. Carayannis, Elias G. & Ferreira, João J.M. & Jalali, Marjan S. & Ferreira, Fernando A.F., 2018. "MCDA in knowledge-based economies: Methodological developments and real world applications," Technological Forecasting and Social Change, Elsevier, vol. 131(C), pages 1-3.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dyckhoff, Harald & Souren, Rainer, 2022. "Integrating multiple criteria decision analysis and production theory for performance evaluation: Framework and review," European Journal of Operational Research, Elsevier, vol. 297(3), pages 795-816.
    2. Gouveia, M.C. & Dias, L.C. & Antunes, C.H. & Boucinha, J. & Inácio, C.F., 2015. "Benchmarking of maintenance and outage repair in an electricity distribution company using the value-based DEA method," Omega, Elsevier, vol. 53(C), pages 104-114.
    3. Small, Adrian & Wainwright, David, 2018. "Privacy and security of electronic patient records – Tailoring multimethodology to explore the socio-political problems associated with Role Based Access Control systems," European Journal of Operational Research, Elsevier, vol. 265(1), pages 344-360.
    4. Mingers, John & White, Leroy, 2010. "A review of the recent contribution of systems thinking to operational research and management science," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1147-1161, December.
    5. Bošković Aleksandra & Krstić Ana, 2020. "The Combined Use of Balanced Scorecard and Data Envelopment Analysis in the Banking Industry," Business Systems Research, Sciendo, vol. 11(1), pages 4-15, March.
    6. Howick, Susan & Ackermann, Fran & Walls, Lesley & Quigley, John & Houghton, Tom, 2017. "Learning from mixed OR method practice: The NINES case study," Omega, Elsevier, vol. 69(C), pages 70-81.
    7. Manuel D. N. T. Oliveira & Fernando A. F. Ferreira & Guillermo O. Pérez-Bustamante Ilander & Marjan S. Jalali, 2017. "Integrating cognitive mapping and MCDA for bankruptcy prediction in small- and medium-sized enterprises," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(9), pages 985-997, September.
    8. Henao, Felipe & Franco, L. Alberto, 2016. "Unpacking multimethodology: Impacts of a community development intervention," European Journal of Operational Research, Elsevier, vol. 253(3), pages 681-696.
    9. Ahti Salo & Antti Punkka, 2011. "Ranking Intervals and Dominance Relations for Ratio-Based Efficiency Analysis," Management Science, INFORMS, vol. 57(1), pages 200-214, January.
    10. Lobo, Maria Stella de Castro & Estellita Lins, Marcos Pereira & Rodrigues, Henrique de Castro & Soares, Gabriel Martins, 2022. "Planning feasible and efficient operational scenarios for a university hospital through multimethodology," Socio-Economic Planning Sciences, Elsevier, vol. 84(C).
    11. HOSSEINZADEH LOTFI, Farhad & HATAMI-MARBINI, Adel & AGRELL, Per & GHOLAMI, Kobra, 2013. "Centralized resource reduction and target setting under DEA control," LIDAM Discussion Papers CORE 2013005, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    12. Liesiö, Juuso & Andelmin, Juho & Salo, Ahti, 2020. "Efficient allocation of resources to a portfolio of decision making units," European Journal of Operational Research, Elsevier, vol. 286(2), pages 619-636.
    13. Yang, Jian-Bo & Wong, Brandon Y.H. & Xu, Dong-Ling & Stewart, Theodor J., 2009. "Integrating DEA-oriented performance assessment and target setting using interactive MOLP methods," European Journal of Operational Research, Elsevier, vol. 195(1), pages 205-222, May.
    14. David Lowe & Louise Martingale & Mike Yearworth, 2016. "Guiding interventions in a multi-organisational context: combining the Viable System Model and Hierarchical Process Modelling for use as a Problem Structuring Method," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 67(12), pages 1481-1495, December.
    15. Ruiz, José L. & Sirvent, Inmaculada, 2019. "Performance evaluation through DEA benchmarking adjusted to goals," Omega, Elsevier, vol. 87(C), pages 150-157.
    16. Hinojosa, M.A. & Mármol, A.M., 2011. "Axial solutions for multiple objective linear problems. An application to target setting in DEA models with preferences," Omega, Elsevier, vol. 39(2), pages 159-167, April.
    17. Halme, Merja & Korhonen, Pekka, 2000. "Restricting weights in value efficiency analysis," European Journal of Operational Research, Elsevier, vol. 126(1), pages 175-188, October.
    18. Fernando A. F. Ferreira & Marjan S. Jalali & Paulo Bento & Carla S. E. Marques & João J. M. Ferreira, 2017. "Enhancing individual entrepreneurial orientation measurement using a metacognitive decision making-based framework," International Entrepreneurship and Management Journal, Springer, vol. 13(2), pages 327-346, June.
    19. M Zohrehbandian & A Makui & A Alinezhad, 2010. "A compromise solution approach for finding common weights in DEA: an improvement to Kao and Hung's approach," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(4), pages 604-610, April.
    20. Amado, Carla A.F. & Santos, Sérgio P. & Marques, Pedro M., 2012. "Integrating the Data Envelopment Analysis and the Balanced Scorecard approaches for enhanced performance assessment," Omega, Elsevier, vol. 40(3), pages 390-403.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:131:y:2018:i:c:p:18-30. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.