Penalized likelihood estimation: Convergence under incorrect model

Penalized likelihood method is among the most effective tools for nonparametric multivariate function estimation. Recently, a generic computation-oriented asymptotic theory has been developed in the density estimation setting, and been extended to other settings such as conditional density estimation, regression, and hazard rare estimation, under the assumption that the true function resides in a reproducing kernel Hilbert space in which the estimate is sought. In this article, we illustrate that the theory may remain valid, after appropriate modifications, even when the true function resides outside of the function space under consideration. Through a certain moment identity, it is shown that the Kullback-Leibler projection of the true function in the function space under consideration, if it exists, acts as the proxy of the true function as the destination of asymptotic convergence.