Newton Methods to Solve a System of Nonlinear Algebraic Equations

Fundamental insight into the solution of systems of nonlinear equations was provided by Powell. It was found that Newton iterations, with exact line searches, did not converge to a stationary point of the natural merit function, i.e., the Euclidean norm of the residuals. Extensive numerical simulation of Powell’s equations produced the unexpected result that Newton iterations converged to the solution from all initial points, where the function is defined, or from those points where the Jacobian is nonsingular, if no line search is used. The significance of Powell’s example is that an important requirement exists when utilizing Newton’s method to solve such a system of nonlinear equations. Specifically, a merit function, which is used in a line search, must have properties consistent with those of a Lyapunov function to provide sufficient conditions for convergence. This implies that level sets of the merit function are properly nested, either globally, or in some finite local region. Therefore, they are topologically equivalent to concentric spherical surfaces, either globally or in a finite local region. Furthermore, an exact line search at a point, far from the solution, may be counterproductive. This observation, and a primary aim of the present analysis, is to demonstrate that it is desirable to construct new Newton iterations, which do not require a merit function with associated line searches.