Nonparametric Variable Selection and Its Application to Additive Models

For multivariate nonparametric regression models, existing variable selection methods with penalization require high-dimensional nonparametric approximations in objective functions. When the dimension is high, none of methods with penalization in the literature are readily available. Also, ranking and screening approaches cannot have selection consistency when iterative algorithms cannot be used due to inefficient nonparametric approximation. In this paper, a novel and easily implemented approach is proposed to make existing methods feasible for selection with no need of nonparametric approximation. Selection consistency can be achieved. As an application to additive regression models, we then suggest a two-stage procedure that separates selection and estimation steps. An adaptive estimation to the smoothness of underlying components can be constructed such that the consistency can be even at parametric rate if the underlying model is really parametric. Simulations are carried out to examine the performance of our method, and a real data example is analyzed for illustration.