The Simplified Expression of Machine Learning and Multivariate Statistical Analysis Based on the Centering Matrix

In machine learning (ML) algorithms and multivariate statistical analysis (MVA) problems, it is usually necessary to center (zero centered or mean subtraction) the original data. The centering matrix plays an important role in this process. The full consideration and use of its properties may contribute to the speed or stability improvement of some related ML algorithms. Therefore, in this paper, we discussed in detail the properties of the centering matrix, proved some previously known properties, and deduced some new properties. The involved properties mainly consisted of the centering, quadratic form, spectral decomposition, null space, projection, exchangeability, Kronecker square, and so on. Based on this, we explored the simplified expression role of the centering matrix in the principal component analysis (PCA) and regression analysis theory. The results show that the sum of deviation squares which is widely used in regression theory and variance analysis can be expressed by the quadratic form with the centering matrix as the kernel matrix. The ML algorithms introducing the centering matrix can greatly simplify the learning process and improve predictive ability.