IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v96y2020ics0305048319307479.html
   My bibliography  Save this article

Consistency issues in the best worst method: Measurements and thresholds

Author

Listed:
  • Liang, Fuqi
  • Brunelli, Matteo
  • Rezaei, Jafar

Abstract

The Best-Worst Method (BWM) uses ratios of the relative importance of criteria in pairs based on the assessment done by decision-makers. When a decision-maker provides the pairwise comparisons in BWM, checking the acceptable inconsistency, to ensure the rationality of the assessments, is an important step. Although both the original and the extended versions of BWM have proposed several consistency measurements, there are some deficiencies, including: (i) the lack of a mechanism to provide immediate feedback to the decision-maker regarding the consistency of the pairwise comparisons being provided, (ii) the inability to consider the ordinal consistency into account, and (iii) the lack of consistency thresholds to determine the reliability of the results. To deal with these problems, this study starts by proposing a cardinal consistency measurement to provide immediate feedback, called the input-based consistency measurement, after which an ordinal consistency measurement is proposed to check the coherence of the order of the results (weights) against the order of the pairwise comparisons provided by the decision-maker. Finally, a method is proposed to balance cardinal consistency ratio under ordinal-consistent and ordinal-inconsistent conditions, to determine the thresholds for the proposed and the original consistency ratios.

Suggested Citation

  • Liang, Fuqi & Brunelli, Matteo & Rezaei, Jafar, 2020. "Consistency issues in the best worst method: Measurements and thresholds," Omega, Elsevier, vol. 96(C).
    • Handle: RePEc:eee:jomega:v:96:y:2020:i:c:s0305048319307479
    DOI: 10.1016/j.omega.2019.102175
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305048319307479
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.omega.2019.102175?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Siraj, Sajid & Mikhailov, Ludmil & Keane, John A., 2015. "Contribution of individual judgments toward inconsistency in pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 242(2), pages 557-567.
    2. Matteo Brunelli & Michele Fedrizzi, 2015. "Axiomatic properties of inconsistency indices for pairwise comparisons," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 66(1), pages 1-15, January.
    3. Fedrizzi, Michele & Giove, Silvio, 2007. "Incomplete pairwise comparison and consistency optimization," European Journal of Operational Research, Elsevier, vol. 183(1), pages 303-313, November.
    4. Monsuur, Herman, 1997. "An intrinsic consistency threshold for reciprocal matrices," European Journal of Operational Research, Elsevier, vol. 96(2), pages 387-391, January.
    5. Sándor Bozóki & János Fülöp & Attila Poesz, 2015. "On reducing inconsistency of pairwise comparison matrices below an acceptance threshold," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 23(4), pages 849-866, December.
    6. Ergu, Daji & Kou, Gang & Peng, Yi & Shi, Yong, 2011. "A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP," European Journal of Operational Research, Elsevier, vol. 213(1), pages 246-259, August.
    7. Siraj, Sajid & Mikhailov, Ludmil & Keane, John, 2012. "A heuristic method to rectify intransitive judgments in pairwise comparison matrices," European Journal of Operational Research, Elsevier, vol. 216(2), pages 420-428.
    8. Lundy, Michele & Siraj, Sajid & Greco, Salvatore, 2017. "The mathematical equivalence of the “spanning tree” and row geometric mean preference vectors and its implications for preference analysis," European Journal of Operational Research, Elsevier, vol. 257(1), pages 197-208.
    9. Gunjan Yadav & Sachin Kumar Mangla & Sunil Luthra & Suresh Jakhar, 2018. "Hybrid BWM-ELECTRE-based decision framework for effective offshore outsourcing adoption: a case study," International Journal of Production Research, Taylor & Francis Journals, vol. 56(18), pages 6259-6278, September.
    10. Valdecy Pereira & Helder Costa, 2015. "Nonlinear programming applied to the reduction of inconsistency in the AHP method," Annals of Operations Research, Springer, vol. 229(1), pages 635-655, June.
    11. Rezaei, Jafar, 2016. "Best-worst multi-criteria decision-making method: Some properties and a linear model," Omega, Elsevier, vol. 64(C), pages 126-130.
    12. Rezaei, Jafar, 2015. "Best-worst multi-criteria decision-making method," Omega, Elsevier, vol. 53(C), pages 49-57.
    13. Salo, Ahti A. & Hamalainen, Raimo P., 1995. "Preference programming through approximate ratio comparisons," European Journal of Operational Research, Elsevier, vol. 82(3), pages 458-475, May.
    14. Mi, Xiaomei & Tang, Ming & Liao, Huchang & Shen, Wenjing & Lev, Benjamin, 2019. "The state-of-the-art survey on integrations and applications of the best worst method in decision making: Why, what, what for and what's next?," Omega, Elsevier, vol. 87(C), pages 205-225.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jie Lu & Feng Li & Desheng Wu, 2024. "A Two-Stage Sustainable Supplier Selection Model Considering Disruption Risk," Sustainability, MDPI, vol. 16(9), pages 1-20, May.
    2. Junnan Wu & Xin Liu & Dianqi Pan & Yichen Zhang & Jiquan Zhang & Kai Ke, 2023. "Research on Safety Evaluation of Municipal Sewage Treatment Plant Based on Improved Best-Worst Method and Fuzzy Comprehensive Method," Sustainability, MDPI, vol. 15(11), pages 1-15, May.
    3. Feng, Jianghong & Guo, Ping & Xu, Guangyi & Xu, Gangyan & Ning, Yu, 2024. "An integrated decision framework for resilient sustainable waste electric vehicle battery recycling transfer station site selection," Applied Energy, Elsevier, vol. 373(C).
    4. Kumar, Anish & Mangla, Sachin Kumar & Kumar, Pradeep & Song, Malin, 2021. "Mitigate risks in perishable food supply chains: Learning from COVID-19," Technological Forecasting and Social Change, Elsevier, vol. 166(C).
    5. Wu, Qun & Liu, Xinwang & Zhou, Ligang & Qin, Jindong & Rezaei, Jafar, 2024. "An analytical framework for the best–worst method," Omega, Elsevier, vol. 123(C).
    6. Mohammed Mojahid Hossain Chowdhury & Ziaul Haque Munim, 2023. "Dry port location selection using a fuzzy AHP-BWM-PROMETHEE approach," Maritime Economics & Logistics, Palgrave Macmillan;International Association of Maritime Economists (IAME), vol. 25(2), pages 301-329, June.
    7. Fontana, Veronika & Ebner, Manuel & Schirpke, Uta & Ohndorf, Markus & Pritsch, Hanna & Tappeiner, Ulrike & Kurmayer, Rainer, 2023. "An integrative approach to evaluate ecosystem services of mountain lakes using multi-criteria decision analysis," Ecological Economics, Elsevier, vol. 204(PA).
    8. Siamak Kheybari & Mohammad Reza Mehrpour & Paul Bauer & Alessio Ishizaka, 2024. "How Can Risk-Averse and Risk-Taking Approaches be Considered in a Group Multi-Criteria Decision-Making Problem?," Group Decision and Negotiation, Springer, vol. 33(4), pages 883-909, August.
    9. Md. Raquibuzzaman Khan & Nazia Tabassum & Niaz Ahmed Khan & Mohammad Jahangir Alam, 2022. "Procurement challenges in public-sector agricultural development projects in Bangladesh," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-13, December.
    10. Moktadir, Md. Abdul & Ren, Jingzheng, 2024. "Towards green logistics: An innovative decision support model for zero-emission transportation modes development," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 189(C).
    11. Gao, Fei & Wang, Weixiang & Bi, Chencan & Bi, Wenhao & Zhang, An, 2023. "Prioritization of used aircraft acquisition criteria: A fuzzy best–worst method (BWM)-based approach," Journal of Air Transport Management, Elsevier, vol. 107(C).
    12. Rajak, Sonu & Mathiyazhagan, K. & Agarwal, Vernika & Sivakumar, K. & Kumar, Vikas & Appolloni, Andrea, 2022. "Issues and analysis of critical success factors for the sustainable initiatives in the supply chain during COVID- 19 pandemic outbreak in India: A case study," Research in Transportation Economics, Elsevier, vol. 93(C).
    13. Kheybari, Siamak & Javdanmehr, Mahsa & Rezaie, Fariba Mahdi & Rezaei, Jafar, 2021. "Corn cultivation location selection for bioethanol production: An application of BWM and extended PROMETHEE II," Energy, Elsevier, vol. 228(C).
    14. Sisto, Roberta & Fernández-Portillo, Luis A. & Yazdani, Morteza & Estepa-Mohedano, Lorenzo & Torkayesh, Ali Ebadi, 2022. "Strategic planning of rural areas: Integrating participatory backcasting and multiple criteria decision analysis tools," Socio-Economic Planning Sciences, Elsevier, vol. 82(PA).
    15. Madjid Tavana & Mehdi Soltanifar & Francisco J. Santos-Arteaga, 2023. "Analytical hierarchy process: revolution and evolution," Annals of Operations Research, Springer, vol. 326(2), pages 879-907, July.
    16. Ricardo, Alexandre & Figueira, José Rui & Tavares, Luís Valadares, 2024. "Integrating confidence and preservation of information in the preference elicitation process: A lexicographic order approach for inconsistent judgments," Omega, Elsevier, vol. 129(C).
    17. Corrente, Salvatore & Greco, Salvatore & Rezaei, Jafar, 2024. "Better decisions with less cognitive load: The Parsimonious BWM," Omega, Elsevier, vol. 126(C).
    18. Abdul Salam Khan, 2024. "One size does not fit all- Strategizing the vaccine supply chain in developing countries," Operations Management Research, Springer, vol. 17(3), pages 941-962, September.
    19. Vicente Javier Clemente-Suárez & Eduardo Navarro-Jiménez & Pablo Ruisoto & Athanasios A. Dalamitros & Ana Isabel Beltran-Velasco & Alberto Hormeño-Holgado & Carmen Cecilia Laborde-Cárdenas & Jose Fran, 2021. "Performance of Fuzzy Multi-Criteria Decision Analysis of Emergency System in COVID-19 Pandemic. An Extensive Narrative Review," IJERPH, MDPI, vol. 18(10), pages 1-12, May.
    20. Vladimír Bureš & Jiří Cabal & Pavel Čech & Karel Mls & Daniela Ponce, 2020. "The Influence of Criteria Selection Method on Consistency of Pairwise Comparison," Mathematics, MDPI, vol. 8(12), pages 1-13, December.
    21. Danish Farooq & Sarbast Moslem & Arshad Jamal & Farhan Muhammad Butt & Yahya Almarhabi & Rana Faisal Tufail & Meshal Almoshaogeh, 2021. "Assessment of Significant Factors Affecting Frequent Lane-Changing Related to Road Safety: An Integrated Approach of the AHP–BWM Model," IJERPH, MDPI, vol. 18(20), pages 1-17, October.
    22. Agota Giedrė Raišienė & Simonas Juozapas Raišys, 2022. "Business Customer Satisfaction with B2B Consulting Services: AHP-Based Criteria for a New Perspective," Sustainability, MDPI, vol. 14(12), pages 1-16, June.
    23. Zsombor Szádoczki & Sándor Bozóki & Patrik Juhász & Sergii V. Kadenko & Vitaliy Tsyganok, 2023. "Incomplete pairwise comparison matrices based on graphs with average degree approximately 3," Annals of Operations Research, Springer, vol. 326(2), pages 783-807, July.
    24. Mohammed Al Awadh, 2022. "Utilizing Multi-Criteria Decision Making to Evaluate the Quality of Healthcare Services," Sustainability, MDPI, vol. 14(19), pages 1-21, October.
    25. Aziz Naghizadeh Vardin & Ramin Ansari & Mohammad Khalilzadeh & Jurgita Antucheviciene & Romualdas Bausys, 2021. "An Integrated Decision Support Model Based on BWM and Fuzzy-VIKOR Techniques for Contractor Selection in Construction Projects," Sustainability, MDPI, vol. 13(12), pages 1-28, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. József Temesi, 2019. "An interactive approach to determine the elements of a pairwise comparison matrix," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 533-549, June.
    2. Siraj, Sajid & Mikhailov, Ludmil & Keane, John A., 2015. "Contribution of individual judgments toward inconsistency in pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 242(2), pages 557-567.
    3. Jean-Pierre Magnot & Jiří Mazurek & Viera Cernanova, 2021. "A gradient method for inconsistency reduction of pairwise comparisons matrices," Working Papers hal-03313878, HAL.
    4. Zsombor Szádoczki & Sándor Bozóki & Patrik Juhász & Sergii V. Kadenko & Vitaliy Tsyganok, 2023. "Incomplete pairwise comparison matrices based on graphs with average degree approximately 3," Annals of Operations Research, Springer, vol. 326(2), pages 783-807, July.
    5. Wu, Qun & Liu, Xinwang & Zhou, Ligang & Qin, Jindong & Rezaei, Jafar, 2024. "An analytical framework for the best–worst method," Omega, Elsevier, vol. 123(C).
    6. Amin Vafadarnikjoo & Madjid Tavana & Tiago Botelho & Konstantinos Chalvatzis, 2020. "A neutrosophic enhanced best–worst method for considering decision-makers’ confidence in the best and worst criteria," Annals of Operations Research, Springer, vol. 289(2), pages 391-418, June.
    7. Csató, László & Petróczy, Dóra Gréta, 2021. "On the monotonicity of the eigenvector method," European Journal of Operational Research, Elsevier, vol. 292(1), pages 230-237.
    8. Matteo Brunelli, 2017. "Studying a set of properties of inconsistency indices for pairwise comparisons," Annals of Operations Research, Springer, vol. 248(1), pages 143-161, January.
    9. Corrente, Salvatore & Greco, Salvatore & Rezaei, Jafar, 2024. "Better decisions with less cognitive load: The Parsimonious BWM," Omega, Elsevier, vol. 126(C).
    10. Lundy, Michele & Siraj, Sajid & Greco, Salvatore, 2017. "The mathematical equivalence of the “spanning tree” and row geometric mean preference vectors and its implications for preference analysis," European Journal of Operational Research, Elsevier, vol. 257(1), pages 197-208.
    11. Besharati Fard, Moein & Moradian, Parisa & Emarati, Mohammadreza & Ebadi, Mehdi & Gholamzadeh Chofreh, Abdoulmohammad & Klemeŝ, Jiří Jaromír, 2022. "Ground-mounted photovoltaic power station site selection and economic analysis based on a hybrid fuzzy best-worst method and geographic information system: A case study Guilan province," Renewable and Sustainable Energy Reviews, Elsevier, vol. 169(C).
    12. Xiao-Kang Wang & Wen-Hui Hou & Chao Song & Min-Hui Deng & Yong-Yi Li & Jian-Qiang Wang, 2021. "BW-MaxEnt: A Novel MCDM Method for Limited Knowledge," Mathematics, MDPI, vol. 9(14), pages 1-17, July.
    13. Jairo Ortega & Sarbast Moslem & János Tóth & Tamás Péter & Juan Palaguachi & Mario Paguay, 2020. "Using Best Worst Method for Sustainable Park and Ride Facility Location," Sustainability, MDPI, vol. 12(23), pages 1-18, December.
    14. Chong Li & He Huang & Ya Luo, 2022. "An Integrated Two-Dimension Linguistic Intuitionistic Fuzzy Decision-Making Approach for Unmanned Aerial Vehicle Supplier Selection," Sustainability, MDPI, vol. 14(18), pages 1-24, September.
    15. Sarbast Moslem & Muhammet Gul & Danish Farooq & Erkan Celik & Omid Ghorbanzadeh & Thomas Blaschke, 2020. "An Integrated Approach of Best-Worst Method (BWM) and Triangular Fuzzy Sets for Evaluating Driver Behavior Factors Related to Road Safety," Mathematics, MDPI, vol. 8(3), pages 1-20, March.
    16. Kusi-Sarpong, Simonov & Orji, Ifeyinwa Juliet & Gupta, Himanshu & Kunc, Martin, 2021. "Risks associated with the implementation of big data analytics in sustainable supply chains," Omega, Elsevier, vol. 105(C).
    17. van de Kaa, G. & Fens, T. & Rezaei, J. & Kaynak, D. & Hatun, Z. & Tsilimeni-Archangelidi, A., 2019. "Realizing smart meter connectivity: Analyzing the competing technologies Power line communication, mobile telephony, and radio frequency using the best worst method," Renewable and Sustainable Energy Reviews, Elsevier, vol. 103(C), pages 320-327.
    18. Gholamreza Haseli & Reza Sheikh & Jianqiang Wang & Hana Tomaskova & Erfan Babaee Tirkolaee, 2021. "A Novel Approach for Group Decision Making Based on the Best–Worst Method (G-BWM): Application to Supply Chain Management," Mathematics, MDPI, vol. 9(16), pages 1-20, August.
    19. Murad, C.A. & Bellinello, M.M. & Silva, A.J. & Netto, A. Caminada & de Souza, G.F.M. & Nabeta, S.I., 2022. "A novel methodology employed for ranking and consolidating performance indicators in holding companies with multiple power plants based on multi-criteria decision-making method," Operations Research Perspectives, Elsevier, vol. 9(C).
    20. Kheybari, Siamak & Javdanmehr, Mahsa & Rezaie, Fariba Mahdi & Rezaei, Jafar, 2021. "Corn cultivation location selection for bioethanol production: An application of BWM and extended PROMETHEE II," Energy, Elsevier, vol. 228(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:96:y:2020:i:c:s0305048319307479. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.