Theoretical estimation of the probability of weight rank reversal in pairwise comparisons

Pairwise comparison is a key component in multi-criteria decision making. The probability of rank reversal is a useful measure for evaluating the impact of uncertainty on the final outcome. In the context of this paper the type of uncertainty considered is related to the fact that experts have different opinions or that they may perform inconsistent pairwise comparisons. We provide a theoretical model for estimating the probability of the consequent rank reversal using the multivariate normal cumulative distribution function. The model is applied to two alternative weight extraction methods frequently used in the literature: the geometric mean and the eigenvalue method. We introduce a reasonable framework for incorporating uncertainty in the decision making process and calculate the mean value and cross-correlation of the average weights which are required in the application of the model. The theoretical results are compared against numerical simulations and a very good agreement is observed. We further show how our model can be extended in applications of a full multi-criteria decision making analysis, such as the analytic hierarchy process. We also discuss how the theoretical model can be used in practice where the statistical properties of the uncertainty-induced perturbations are unknown and the only information provided by the pairwise comparison matrices of a small group of experts. The methodology presented here can be used to extend the pairwise comparison framework in order to provide some information on the credibility of its outcome.