An Empirical Comparison of Rank-Based Surrogate Weights in Additive Multiattribute Decision Analysis

Many methods for creating surrogate swing weights based only on the rank order of the attributes are proposed to avoid the cost and effort of eliciting weights in multiattribute decision analysis. We explore empirically how well eight different methods perform based on a large sample of real-world elicited weights. We use the Euclidean distance from the elicited weights to judge the quality of the surrogate weights as well as three other metrics. The sum reciprocal method gives results, on average, statistically closest to the elicited weights for all metrics used. The equal ratio method using a fixed ratio of 0.716 performs just as well on three of the metrics. The rank sum method, the simplest and one of the oldest methods, performs generally next best. The rank order centroid method, which does well in simulation studies, performs relatively poorly in this evaluation using real-world data.