A multidimensional framework for performance evaluation of forecasting models: context-dependent DEA

The performance evaluation of competing forecasting models is generally restricted to their ranking by criterion, which generally leads to several inconsistent rankings for different criteria. The purpose of this article is to propose a multidimensional framework; namely, Data Envelopment Analysis (DEA), to overcome this problem by determining a single ranking that takes account of multiple criteria. In order to operationalize this framework, we survey the literature on forecasting criteria and measures, propose a new classification of criteria, and discuss how one might measure them. We use forecasting models of crude oil prices to illustrate the use of the proposed multidimensional performance evaluation framework. Our empirical results suggest that both the best and the worst forecasting models with respect to most performance criteria and their measures tend to maintain their unidimensional ranking positions when assessed in a multidimensional setting; however, the multidimensional ranking of some models could be substantially different from their unidimensional rankings, which highlights the importance of the proposed performance evaluation tool.