A Universal Expectation Bound on Empirical Projections of Deformed Random Matrices

Let $$C$$ C be a real-valued $$M\times M$$ M × M matrix with singular values $$\lambda _1\ge \cdots \ge \lambda _M$$ λ 1 ≥ ⋯ ≥ λ M , and $$E$$ E a random matrix of centered i.i.d. entries with finite fourth moment. In this paper, we give a universal upper bound on the expectation of $$||\hat{\pi }_rX||_{S_2}^2-||\pi _rX||^2_{S_2}$$ | | π ^ r X | | S 2 2 - | | π r X | | S 2 2 , where $$X:=C+E$$ X : = C + E and $$\hat{\pi }_r$$ π ^ r (resp. $$\pi _r$$ π r ) is a rank- $$r$$ r projection maximizing the Hilbert–Schmidt norm $$||{\tilde{\pi }}_rX||_{S_2}$$ | | π ~ r X | | S 2 (resp. $$||{\tilde{\pi }}_rC||_{S_2}$$ | | π ~ r C | | S 2 ) over the set $$\mathcal{S }_{M,r}$$ S M , r of all orthogonal rank- $$r$$ r projections. This result is a generalization of a theorem for Gaussian matrices due to [7]. Our approach differs substantially from the techniques of the mentioned article. We analyze $$||\hat{\pi }_rX||_{S_2}^2-||\pi _rX||^2_{S_2}$$ | | π ^ r X | | S 2 2 - | | π r X | | S 2 2 from a rather deterministic point of view by an upper bound on $$||\hat{\pi }_rX||_{S_2}^2-||\pi _rX||^2_{S_2}$$ | | π ^ r X | | S 2 2 - | | π r X | | S 2 2 , whose randomness is totally determined by the largest singular value of $$E$$ E .