Better subset regression

This paper studies the relationship between model fitting and screening performance to find efficient screening methods for high-dimensional linear regression models. Under a sparsity assumption we show in a general asymptotic setting that a subset that includes the true submodel always yields a smaller residual sum of squares than those that do not. To seek such a subset, we consider the optimization problem associated with best subset regression. An em algorithm, known as orthogonalizing subset screening, and its accelerated version are proposed for searching for the best subset. Although the algorithms do not always find the best subset, their monotonicity makes the subset fit the data better than initial subsets, and thus the subset can have better screening performance asymptotically. Simulation results show that our methods are very competitive.