Efficient two-dimensional smoothing with P-spline ANOVA mixed models and nested bases

Low-rank smoothing techniques have gained much popularity in non-standard regression modeling. In particular, penalized splines and tensor product smooths are used as flexible tools to study non-parametric relationships among several covariates. The use of standard statistical software facilitates their use for several types of problems and applications. However, when interaction terms are considered in the modeling, and multiple smoothing parameters need to be estimated standard software does not work well when datasets are large or higher-order interactions are included or need to be tested. In this paper, a general approach for constructing and estimating bivariate smooth models for additive and interaction terms using penalized splines is proposed. The formulation is based on the mixed model representation of the smooth-ANOVA model by Lee and Durbán (in press), and several nested models in terms of random effects components are proposed. Each component has a clear interpretation in terms of function shape and model identifiability constraints. The term PS-ANOVA is coined for this type of models. The estimation method is relatively straightforward based on the algorithm by Schall (1991) for generalized linear mixed models. Further, a simplification of the smooth interaction term is used by constructing lower-rank basis (nested basis). Finally, some simulation studies and real data examples are presented to evaluate the new model and the estimation method.