Primal Methods: The Generalized Reduced Gradient with Sequential Linearization

By primal method, we understand a search method that works directly on the original problem by searching the optimal solution taking a path through the feasible region. Every iteration in these methods is feasible and along the iterations, the values of the minimizing function constantly decrease. The most important primal methods are the feasible direction method of Zoutendijk (1960), the gradient projection method of Rosen (1960, 1961), the reduced gradient method of Wolfe (1967), the convex simplex method of Zangwill (1967), and the generalized reduced gradient method of Abadie and Carpentier (1969). The last four methods can be embedded into the class of the active set methods. The idea of the active set methods is to partition the inequality constraints into two groups: those that can be treated as active (satisfied with equality in the current point) and those that have to be treated as inactive. The constraints treated as inactive are essentially ignored. Of course, the fundamental component of an active set method is the algorithm for solving an optimization problem with equality constraints only.