Communication-efficient model averaging prediction for massive data with asymptotic optimality

This paper focuses on model averaging prediction for massive dataset. Specifically, in the framework of Mallows model averaging, we propose two distributed approaches to estimate the parameters of each submodel and weights in the final weighted estimator, respectively. The first approach is an one-shot procedure that aggregates the estimated parameters and weights from each local machine via simple average. The second approach is an iterative procedure that approximates the global loss by a surrogate loss in parameter estimation. The two proposed distributed estimators are communication-efficient, where the former requires only one round of communication and the latter requires two rounds of communications between central and local machines for parameter estimation to achieve the globally statistical efficiency. To estimate weight vector, two distributed algorithms are presented. Furthermore, we theoretically justify the two approaches by proving convergence rates and asymptotic normalities. More importantly, we establish the asymptotic optimality of distributed estimator of weight vector in terms of the out-of-sample prediction error criterion. Finally, simulations and a real data analysis are carried out to illustrate the proposed methods.