IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v264y2018i2p462-471.html
   My bibliography  Save this article

Are multi-criteria decision-making tools useful? An experimental comparative study of three methods

Author

Listed:
  • Ishizaka, Alessio
  • Siraj, Sajid

Abstract

Many decision makers still question the usefulness of multi-criteria decision-making methods and prefer to rely on intuitive decisions. In this study we evaluated a number of multi-criteria decision-making tools for their usefulness using incentive-based experiments, which is a novel approach in operations research but common in psychology and experimental economics. In this experiment the participants were asked to compare five coffee shops to win a voucher for their best-rated shop. We found that, although the usefulness of different multi-criteria decision-making tools varied to some extent, all the tools were found to be useful in the sense that, when they decided to change their ranking, they followed the recommendation of the multi-criteria decision-making tool. Moreover, the level of inconsistency in the judgements provided had no significant effect on the usefulness of these tools.

Suggested Citation

  • Ishizaka, Alessio & Siraj, Sajid, 2018. "Are multi-criteria decision-making tools useful? An experimental comparative study of three methods," European Journal of Operational Research, Elsevier, vol. 264(2), pages 462-471.
    • Handle: RePEc:eee:ejores:v:264:y:2018:i:2:p:462-471
    DOI: 10.1016/j.ejor.2017.05.041
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221717304885
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2017.05.041?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jacquet-Lagreze, E. & Siskos, J., 1982. "Assessing a set of additive utility functions for multicriteria decision-making, the UTA method," European Journal of Operational Research, Elsevier, vol. 10(2), pages 151-164, June.
    2. Brice Corgnet & Joaquín Gómez-Miñambres & Roberto Hernán-González, 2015. "Goal Setting and Monetary Incentives: When Large Stakes Are Not Enough," Management Science, INFORMS, vol. 61(12), pages 2926-2944, December.
    3. JosÉ Figueira & Salvatore Greco & Matthias Ehrogott, 2005. "Multiple Criteria Decision Analysis: State of the Art Surveys," International Series in Operations Research and Management Science, Springer, number 978-0-387-23081-8, December.
    4. Greco, Salvatore & Mousseau, Vincent & Slowinski, Roman, 2008. "Ordinal regression revisited: Multiple criteria ranking using a set of additive value functions," European Journal of Operational Research, Elsevier, vol. 191(2), pages 416-436, December.
    5. A Ishizaka & D Balkenborg & T Kaplan, 2011. "Does AHP help us make a choice? An experimental evaluation," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(10), pages 1801-1812, October.
    6. Tomashevskii, I.L., 2015. "Eigenvector ranking method as a measuring tool: Formulas for errors," European Journal of Operational Research, Elsevier, vol. 240(3), pages 774-780.
    7. Pekka Korhonen & Jyrki Wallenius & Stanley Zionts, 1984. "Solving the Discrete Multiple Criteria Problem using Convex Cones," Management Science, INFORMS, vol. 30(11), pages 1336-1345, November.
    8. Yoram Wind & Thomas L. Saaty, 1980. "Marketing Applications of the Analytic Hierarchy Process," Management Science, INFORMS, vol. 26(7), pages 641-658, July.
    9. Thomas L. Saaty, 2013. "The Modern Science of Multicriteria Decision Making and Its Practical Applications: The AHP/ANP Approach," Operations Research, INFORMS, vol. 61(5), pages 1101-1118, October.
    10. Michèle Belot & Marina Schröder, 2016. "The Spillover Effects of Monitoring: A Field Experiment," Management Science, INFORMS, vol. 62(1), pages 37-45, January.
    11. Bana e Costa, Carlos A. & Vansnick, Jean-Claude, 2008. "A critical analysis of the eigenvalue method used to derive priorities in AHP," European Journal of Operational Research, Elsevier, vol. 187(3), pages 1422-1428, June.
    12. A. M. Geoffrion & J. S. Dyer & A. Feinberg, 1972. "An Interactive Approach for Multi-Criterion Optimization, with an Application to the Operation of an Academic Department," Management Science, INFORMS, vol. 19(4-Part-1), pages 357-368, December.
    13. Bernroider, Edward W.N. & Schmöllerl, Patrick, 2013. "A technological, organisational, and environmental analysis of decision making methodologies and satisfaction in the context of IT induced business transformations," European Journal of Operational Research, Elsevier, vol. 224(1), pages 141-153.
    14. Green, Paul E & Srinivasan, V, 1978. "Conjoint Analysis in Consumer Research: Issues and Outlook," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 5(2), pages 103-123, Se.
    15. Bernard Roy, 2005. "Paradigms and Challenges," International Series in Operations Research & Management Science, in: Multiple Criteria Decision Analysis: State of the Art Surveys, chapter 0, pages 3-24, Springer.
    16. Korhonen, Pekka, 1988. "A visual reference direction approach to solving discrete multiple criteria problems," European Journal of Operational Research, Elsevier, vol. 34(2), pages 152-159, March.
    17. Stanley Zionts & Jyrki Wallenius, 1976. "An Interactive Programming Method for Solving the Multiple Criteria Problem," Management Science, INFORMS, vol. 22(6), pages 652-663, February.
    18. Bertrand Mareschal & Jean Pierre Brans, 1990. "The PROMETHEE methods for MCDM: the PROMCALC, GAIA and BANK ADVISER software," ULB Institutional Repository 2013/9337, ULB -- Universite Libre de Bruxelles.
    19. Samuel D. Bond & Kurt A. Carlson & Ralph L. Keeney, 2008. "Generating Objectives: Can Decision Makers Articulate What They Want?," Management Science, INFORMS, vol. 54(1), pages 56-70, January.
    20. Jyrki Wallenius & James S. Dyer & Peter C. Fishburn & Ralph E. Steuer & Stanley Zionts & Kalyanmoy Deb, 2008. "Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead," Management Science, INFORMS, vol. 54(7), pages 1336-1349, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Corrente, S. & Figueira, J.R. & Greco, S., 2021. "Pairwise comparison tables within the deck of cards method in multiple criteria decision aiding," European Journal of Operational Research, Elsevier, vol. 291(2), pages 738-756.
    2. Behnam Malakooti, 2015. "Double Helix Value Functions, Ordinal/Cardinal Approach, Additive Utility Functions, Multiple Criteria, Decision Paradigm, Process, and Types (Z Theory I)," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 14(06), pages 1353-1400, November.
    3. Kadziński, MiŁosz & Greco, Salvatore & SŁowiński, Roman, 2012. "Extreme ranking analysis in robust ordinal regression," Omega, Elsevier, vol. 40(4), pages 488-501.
    4. Angilella, Silvia & Greco, Salvatore & Matarazzo, Benedetto, 2010. "Non-additive robust ordinal regression: A multiple criteria decision model based on the Choquet integral," European Journal of Operational Research, Elsevier, vol. 201(1), pages 277-288, February.
    5. Hurson, Christian & Siskos, Yannis, 2014. "A synergy of multicriteria techniques to assess additive value models," European Journal of Operational Research, Elsevier, vol. 238(2), pages 540-551.
    6. Corrente, Salvatore & Greco, Salvatore & Ishizaka, Alessio, 2016. "Combining analytical hierarchy process and Choquet integral within non-additive robust ordinal regression," Omega, Elsevier, vol. 61(C), pages 2-18.
    7. Ciomek, Krzysztof & Kadziński, Miłosz & Tervonen, Tommi, 2017. "Heuristics for selecting pair-wise elicitation questions in multiple criteria choice problems," European Journal of Operational Research, Elsevier, vol. 262(2), pages 693-707.
    8. Xiaoping Li & Dan Zhu, 2011. "Object technology software selection: a case study," Annals of Operations Research, Springer, vol. 185(1), pages 5-24, May.
    9. Doumpos, Michael & Zopounidis, Constantin, 2011. "Preference disaggregation and statistical learning for multicriteria decision support: A review," European Journal of Operational Research, Elsevier, vol. 209(3), pages 203-214, March.
    10. Fowler, John W. & Gel, Esma S. & Köksalan, Murat M. & Korhonen, Pekka & Marquis, Jon L. & Wallenius, Jyrki, 2010. "Interactive evolutionary multi-objective optimization for quasi-concave preference functions," European Journal of Operational Research, Elsevier, vol. 206(2), pages 417-425, October.
    11. Korhonen, Pekka J. & Wallenius, Jyrki & Genc, Tolga & Xu, Peng, 2021. "On rational behavior in multi-attribute riskless choice," European Journal of Operational Research, Elsevier, vol. 288(1), pages 331-342.
    12. Kaynar, Nur & Karsu, Özlem, 2018. "Equitable decision making approaches over allocations of multiple benefits to multiple entities," Omega, Elsevier, vol. 81(C), pages 85-98.
    13. Thomas L. Saaty, 2013. "The Modern Science of Multicriteria Decision Making and Its Practical Applications: The AHP/ANP Approach," Operations Research, INFORMS, vol. 61(5), pages 1101-1118, October.
    14. Roszkowska, Ewa & Wachowicz, Tomasz, 2015. "Application of fuzzy TOPSIS to scoring the negotiation offers in ill-structured negotiation problems," European Journal of Operational Research, Elsevier, vol. 242(3), pages 920-932.
    15. Nowak, Maciej, 2007. "Aspiration level approach in stochastic MCDM problems," European Journal of Operational Research, Elsevier, vol. 177(3), pages 1626-1640, March.
    16. Kadziński, Miłosz & Greco, Salvatore & Słowiński, Roman, 2012. "Selection of a representative value function in robust multiple criteria ranking and choice," European Journal of Operational Research, Elsevier, vol. 217(3), pages 541-553.
    17. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "Sigma-Mu efficiency analysis: A methodology for evaluating units through composite indicators," European Journal of Operational Research, Elsevier, vol. 278(3), pages 942-960.
    18. Hocine, Amine & Kouaissah, Noureddine, 2020. "XOR analytic hierarchy process and its application in the renewable energy sector," Omega, Elsevier, vol. 97(C).
    19. G Özerol & E Karasakal, 2008. "Interactive outranking approaches for multicriteria decision-making problems with imprecise information," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(9), pages 1253-1268, September.
    20. József Temesi, 2011. "Pairwise comparison matrices and the error-free property of the decision maker," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 19(2), pages 239-249, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:264:y:2018:i:2:p:462-471. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.