Facilitating high‐dimensional transparent classification via empirical Bayes variable selection

We present a two‐step approach to classification problems in the “large P, small N” setting, where the number of predictors may be larger than the sample size. We assume that the association between the predictors and the class variable has an approximate linear‐logistic form, but we allow the class boundaries to be nonlinear. We further assume that the number of true predictors is relatively small. In the first step, we use a binomial generalized linear model to identify which predictors are associated with each class and then restrict the data set to these predictors and run a nonlinear classifier, such as a random forest or a support vector machine. We show that, without the variable screening step, the classification performance of both the random forest and support vector machine is degraded when many among the P predictors are not related to the class.