Asymptotic covariance estimation by Gaussian random perturbation

In most cases, the asymptotic covariance matrix of an M-estimator is in a sandwich form. This sandwich form involves calculations of the first and second order derivatives of the loss function, which is intractable if the loss function is complex. To alleviate this problem, we propose in this article a novel method called Gaussian random perturbation. This method can be used to estimate the asymptotic covariance matrix of a general M-estimator without derivative calculations. The idea can be summarized as follows. We first generate a small random perturbation around the M-estimator. Then, we re-evaluate the loss function at the randomly perturbed M-estimator and obtain the estimators of the first and second order derivatives of the loss function via Taylor series expansion. This leads to a novel estimator for the asymptotic covariance matrix. We then rigorously show that the resulting covariance estimator is statistically consistent with two elegant characteristics. First, it involves no computation of derivatives. This makes it easier to estimate the covariance matrix of an M-estimator with a complex loss function. Second, it is convenient for parallel computing and thus attractive for massive data analysis. The consistency of the proposed asymptotic covariance estimator is demonstrated under appropriate regularity conditions. The practical usefulness of the method is further demonstrated with both simulation studies and real data analysis.