Sparse overlapped linear discriminant analysis

High-dimensional binary discriminant analysis received most of the attention in the last years compared to the multi-class discriminant analysis. The latter is often considered as a generalization to the binary case and is thus usually treated as multiple binary classification problems. However, this results in classifiers that ignore multi-class structure features. On the other hand, known methods specifically tailored for the multi-class discriminant analysis either impose strong constrains such as the hard threshold, suffer from heavy computations due to conducting calculations only sequentially, or lack theoretical guarantees regarding the consistency of estimation and prediction. In this paper, a new sparse multi-class classification method, termed Sparse Overlapped Linear Discriminant Analysis, is proposed. It takes into account the multi-class information using an adaptive penalty whose weights reflect the multi-class structure. The new procedure estimates Bayes discriminant directions in a parallel way contrary to other methods which proceed only sequentially. It enjoys consistency results in the high-dimensional case in terms of estimation and prediction errors, and performs favorably compared to other competing methods on both simulated and real datasets.