Convergence and evaluation-complexity analysis of a regularized tensor-Newton method for solving nonlinear least-squares problems

Given a twice-continuously differentiable vector-valued function r(x), a local minimizer of $$\Vert r(x)\Vert _2$$ ‖ r ( x ) ‖ 2 is sought. We propose and analyse tensor-Newton methods, in which r(x) is replaced locally by its second-order Taylor approximation. Convergence is controlled by regularization of various orders. We establish global convergence to a first-order critical point of $$\Vert r(x)\Vert _2$$ ‖ r ( x ) ‖ 2 , and provide function evaluation bounds that agree with the best-known bounds for methods using second derivatives. Numerical experiments comparing tensor-Newton methods with regularized Gauss–Newton and Newton methods demonstrate the practical performance of the newly proposed method.