IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v108y2022ics0305048321001857.html
   My bibliography  Save this article

Inconsistency thresholds for incomplete pairwise comparison matrices

Author

Listed:
  • Ágoston, Kolos Csaba
  • Csató, László

Abstract

Pairwise comparison matrices are increasingly used in settings where some pairs are missing. However, there exist few inconsistency indices for similar incomplete data sets and no reasonable measure has an associated threshold. This paper generalises the famous rule of thumb for the acceptable level of inconsistency, proposed by Saaty, to incomplete pairwise comparison matrices. The extension is based on choosing the missing elements such that the maximal eigenvalue of the incomplete matrix is minimised. Consequently, the well-established values of the random index cannot be adopted: the inconsistency of random matrices is found to be the function of matrix size and the number of missing elements, with a nearly linear dependence in the case of the latter variable. Our results can be directly built into decision-making software and used by practitioners as a statistical criterion for accepting or rejecting an incomplete pairwise comparison matrix.

Suggested Citation

  • Ágoston, Kolos Csaba & Csató, László, 2022. "Inconsistency thresholds for incomplete pairwise comparison matrices," Omega, Elsevier, vol. 108(C).
    • Handle: RePEc:eee:jomega:v:108:y:2022:i:c:s0305048321001857
    DOI: 10.1016/j.omega.2021.102576
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305048321001857
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.omega.2021.102576?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Szádoczki, Zsombor & Bozóki, Sándor & Tekile, Hailemariam Abebe, 2022. "Filling in pattern designs for incomplete pairwise comparison matrices: (Quasi-)regular graphs with minimal diameter," Omega, Elsevier, vol. 107(C).
    2. Szybowski, Jacek & Kułakowski, Konrad & Prusak, Anna, 2020. "New inconsistency indicators for incomplete pairwise comparisons matrices," Mathematical Social Sciences, Elsevier, vol. 108(C), pages 138-145.
    3. Sándor Bozóki & Linda Dezső & Attila Poesz & József Temesi, 2013. "Analysis of pairwise comparison matrices: an empirical research," Annals of Operations Research, Springer, vol. 211(1), pages 511-528, December.
    4. Petróczy, Dóra Gréta, 2021. "An alternative quality of life ranking on the basis of remittances," Socio-Economic Planning Sciences, Elsevier, vol. 78(C).
    5. Chao, Xiangrui & Kou, Gang & Li, Tie & Peng, Yi, 2018. "Jie Ke versus AlphaGo: A ranking approach using decision making method for large-scale data with incomplete information," European Journal of Operational Research, Elsevier, vol. 265(1), pages 239-247.
    6. László Csató, 2013. "Ranking by pairwise comparisons for Swiss-system tournaments," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 21(4), pages 783-803, December.
    7. Forman, Ernest H., 1990. "Random indices for incomplete pairwise comparison matrices," European Journal of Operational Research, Elsevier, vol. 48(1), pages 153-155, September.
    8. Sándor Bozóki & János Fülöp & Attila Poesz, 2015. "On reducing inconsistency of pairwise comparison matrices below an acceptance threshold," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 23(4), pages 849-866, December.
    9. Ergu, Daji & Kou, Gang & Peng, Yi & Shi, Yong, 2011. "A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP," European Journal of Operational Research, Elsevier, vol. 213(1), pages 246-259, August.
    10. Changsheng Lin & Gang Kou & Daji Ergu, 2014. "A statistical approach to measure the consistency level of the pairwise comparison matrix," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 65(9), pages 1380-1386, September.
    11. Kang Xu & Jiuping Xu, 2020. "A direct consistency test and improvement method for the analytic hierarchy process," Fuzzy Optimization and Decision Making, Springer, vol. 19(3), pages 359-388, September.
    12. Changsheng Lin & Gang Kou & Daji Ergu, 2013. "An improved statistical approach for consistency test in AHP," Annals of Operations Research, Springer, vol. 211(1), pages 289-299, December.
    13. Bozóki, Sándor & Csató, László & Temesi, József, 2016. "An application of incomplete pairwise comparison matrices for ranking top tennis players," European Journal of Operational Research, Elsevier, vol. 248(1), pages 211-218.
    14. Csató, László & Tóth, Csaba, 2020. "University rankings from the revealed preferences of the applicants," European Journal of Operational Research, Elsevier, vol. 286(1), pages 309-320.
    15. Abel, Edward & Mikhailov, Ludmil & Keane, John, 2018. "Inconsistency reduction in decision making via multi-objective optimisation," European Journal of Operational Research, Elsevier, vol. 267(1), pages 212-226.
    16. Aupetit, Bernard & Genest, Christian, 1993. "On some useful properties of the Perron eigenvalue of a positive reciprocal matrix in the context of the analytic hierarchy process," European Journal of Operational Research, Elsevier, vol. 70(2), pages 263-268, October.
    17. László Csató, 2017. "On the ranking of a Swiss system chess team tournament," Annals of Operations Research, Springer, vol. 254(1), pages 17-36, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ágoston, Kolos Csaba & Csató, László, 2024. "A lexicographically optimal completion for pairwise comparison matrices with missing entries," European Journal of Operational Research, Elsevier, vol. 314(3), pages 1078-1086.
    2. Ausloos, Marcel, 2024. "Hierarchy selection: New team ranking indicators for cyclist multi-stage races," European Journal of Operational Research, Elsevier, vol. 314(2), pages 807-816.
    3. Zsombor Szádoczki & Sándor Bozóki & Patrik Juhász & Sergii V. Kadenko & Vitaliy Tsyganok, 2023. "Incomplete pairwise comparison matrices based on graphs with average degree approximately 3," Annals of Operations Research, Springer, vol. 326(2), pages 783-807, July.
    4. Tekile, Hailemariam Abebe & Brunelli, Matteo & Fedrizzi, Michele, 2023. "A numerical comparative study of completion methods for pairwise comparison matrices," Operations Research Perspectives, Elsevier, vol. 10(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Csató, László & Petróczy, Dóra Gréta, 2021. "On the monotonicity of the eigenvector method," European Journal of Operational Research, Elsevier, vol. 292(1), pages 230-237.
    2. Fernandes, Rosário & Furtado, Susana, 2022. "Efficiency of the principal eigenvector of some triple perturbed consistent matrices," European Journal of Operational Research, Elsevier, vol. 298(3), pages 1007-1015.
    3. Petróczy, Dóra Gréta, 2021. "An alternative quality of life ranking on the basis of remittances," Socio-Economic Planning Sciences, Elsevier, vol. 78(C).
    4. Zsombor Szádoczki & Sándor Bozóki & Patrik Juhász & Sergii V. Kadenko & Vitaliy Tsyganok, 2023. "Incomplete pairwise comparison matrices based on graphs with average degree approximately 3," Annals of Operations Research, Springer, vol. 326(2), pages 783-807, July.
    5. László Csató, 2019. "An impossibility theorem for paired comparisons," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 497-514, June.
    6. Szádoczki, Zsombor & Bozóki, Sándor & Tekile, Hailemariam Abebe, 2022. "Filling in pattern designs for incomplete pairwise comparison matrices: (Quasi-)regular graphs with minimal diameter," Omega, Elsevier, vol. 107(C).
    7. Jean-Pierre Magnot & Jiří Mazurek & Viera Cernanova, 2021. "A gradient method for inconsistency reduction of pairwise comparisons matrices," Working Papers hal-03313878, HAL.
    8. Vladimír Bureš & Jiří Cabal & Pavel Čech & Karel Mls & Daniela Ponce, 2020. "The Influence of Criteria Selection Method on Consistency of Pairwise Comparison," Mathematics, MDPI, vol. 8(12), pages 1-13, December.
    9. D'ora Gr'eta Petr'oczy & L'aszl'o Csat'o, 2019. "Revenue allocation in Formula One: a pairwise comparison approach," Papers 1909.12931, arXiv.org, revised Dec 2020.
    10. Ágoston, Kolos Csaba & Csató, László, 2024. "A lexicographically optimal completion for pairwise comparison matrices with missing entries," European Journal of Operational Research, Elsevier, vol. 314(3), pages 1078-1086.
    11. Csató, László, 2019. "A characterization of the Logarithmic Least Squares Method," European Journal of Operational Research, Elsevier, vol. 276(1), pages 212-216.
    12. László Csató, 2019. "Axiomatizations of inconsistency indices for triads," Annals of Operations Research, Springer, vol. 280(1), pages 99-110, September.
    13. Matteo Brunelli, 2017. "Studying a set of properties of inconsistency indices for pairwise comparisons," Annals of Operations Research, Springer, vol. 248(1), pages 143-161, January.
    14. Kun Chen & Gang Kou & J. Michael Tarn & Yan Song, 2015. "Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices," Annals of Operations Research, Springer, vol. 235(1), pages 155-175, December.
    15. Csató, László & Tóth, Csaba, 2020. "University rankings from the revealed preferences of the applicants," European Journal of Operational Research, Elsevier, vol. 286(1), pages 309-320.
    16. Sangeeta Pant & Anuj Kumar & Mangey Ram & Yury Klochkov & Hitesh Kumar Sharma, 2022. "Consistency Indices in Analytic Hierarchy Process: A Review," Mathematics, MDPI, vol. 10(8), pages 1-15, April.
    17. L'aszl'o Csat'o & Csaba T'oth, 2018. "University rankings from the revealed preferences of the applicants," Papers 1810.04087, arXiv.org, revised Feb 2020.
    18. Liang, Fuqi & Brunelli, Matteo & Rezaei, Jafar, 2020. "Consistency issues in the best worst method: Measurements and thresholds," Omega, Elsevier, vol. 96(C).
    19. Meimei Xia & Jian Chen & Juliang Zhang, 2015. "Multi-criteria decision making based on relative measures," Annals of Operations Research, Springer, vol. 229(1), pages 791-811, June.
    20. László Csató, 2024. "Club coefficients in the UEFA Champions League: Time for shift to an Elo-based formula," International Journal of Performance Analysis in Sport, Taylor & Francis Journals, vol. 24(2), pages 119-134, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:108:y:2022:i:c:s0305048321001857. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.