Evaluating performance of super-efficiency models in ranking efficient decision-making units based on Monte Carlo simulations

In response to the limitation of classical Data Envelopment Analysis (DEA) models, the super efficiency DEA models, including Andersen and Petersen (Manag Sci 39(10): 1261–1264, 1993)'s model (hereafter called AP model) and Li et al. (Eur J Oper Res 255(3): 884–892, 2016)'s cooperative-game-based model (hereafter called L–L model), have been proposed to rank efficient decision-making units (DMUs). Although both models have been widely applied in practice, there is a paucity of research examining the performance of the two models in ranking efficient DMUs. Consequently, it is unclear how close the rankings obtained by the two models are to the "true" ones. Among the very few studies, Banker et al. (Ann Oper Res 250(1): 21–35, 2017) pointed out that the ranking performance of the AP model is unsatisfactory; Li et al. (Eur J Oper Res 255(3): 884–892, 2016) and Hinojosa et al. (Exp Syst Appl 80(9): 273–283, 2017) demonstrated the L–L model's capability of ranking efficient DMUs without addressing the ranking performance. In this study, we, thus, examine the ranking performance of the two super-efficiency models. In evaluating their performance, we carry out Monte Carlo simulations based on the well-known Cobb–Douglas production function and adopt Kendall rank correlation coefficient. Unlike Banker et al. (Ann Oper Res 250(1): 21–35, 2017), we use the rankings obtained based on the two models and the "true" ones as the basis of performance evaluation in our simulations. Moreover, we consider several types of returns to scale (RS) and study the impact of changes of some parameters on the ranking performance. In view of the importance, we also carry out additional simulations to examine the influence of technical inefficiency on the two models' ranking performance. Based on the simulation results, we conclude: (1) Under different RS, the ranking performance of the two models remains the same when changing parameters, e.g., the distribution of input variables; (2) Under different RS, when technical inefficiency (in comparison with random noise) is more important, the two models have satisfactory performance by providing rankings that are close to, or the same as, the "true" ones; (3) The L–L model has better performance than the AP model and is more robust. This is especially true when technical inefficiency is less important; (4) Under different RS, when technical inefficiency is less important, both models have unsatisfactory ranking performance; and (5) The relative importance of technical inefficiency plays an prominent role in ranking efficient DMUs.