MSEBAG: a dynamic classifier ensemble generation based on ‘minimum-sufficient ensemble' and bagging

In this paper, we propose a dynamic classifier system, MSEBAG, which is characterised by searching for the ‘minimum-sufficient ensemble’ and bagging at the ensemble level. It adopts an ‘over-generation and selection’ strategy and aims to achieve a good bias–variance trade-off. In the training phase, MSEBAG first searches for the ‘minimum-sufficient ensemble’, which maximises the in-sample fitness with the minimal number of base classifiers. Then, starting from the ‘minimum-sufficient ensemble’, a backward stepwise algorithm is employed to generate a collection of ensembles. The objective is to create a collection of ensembles with a descending fitness on the data, as well as a descending complexity in the structure. MSEBAG dynamically selects the ensembles from the collection for the decision aggregation. The extended adaptive aggregation (EAA) approach, a bagging-style algorithm performed at the ensemble level, is employed for this task. EAA searches for the competent ensembles using a score function, which takes into consideration both the in-sample fitness and the confidence of the statistical inference, and averages the decisions of the selected ensembles to label the test pattern. The experimental results show that the proposed MSEBAG outperforms the benchmarks on average.