IDEAS home Printed from https://ideas.repec.org/a/spr/eurjco/v5y2017i4d10.1007_s13675-017-0080-8.html
   My bibliography  Save this article

Nonsmooth spectral gradient methods for unconstrained optimization

Author

Listed:
  • Milagros Loreto

    (University of Washington Bothell)

  • Hugo Aponte

    (Microsoft)

  • Debora Cores

    (Universidad Simón Bolívar)

  • Marcos Raydan

    (Universidad Simón Bolívar)

Abstract

To solve nonsmooth unconstrained minimization problems, we combine the spectral choice of step length with two well-established subdifferential-type schemes: the gradient sampling method and the simplex gradient method. We focus on the interesting case in which the objective function is continuously differentiable almost everywhere, and it is often not differentiable at minimizers. In the case of the gradient sampling method, we also present a simple differentiability test that allows us to use the exact gradient direction as frequently as possible, and to build a stochastic subdifferential direction only if the test fails. The proposed spectral gradient sampling method is combined with a monotone line search globalization strategy. On the other hand, the simplex gradient method is a direct search method that only requires function evaluations to build an approximation to the gradient direction. In this case, the proposed spectral simplex gradient method is combined with a suitable nonmonotone line search strategy. For both scenarios, we discuss convergence properties and present numerical results on a set of nonsmooth test functions. These numerical results indicate that using a spectral step length can improve the practical performance of both methods.

Suggested Citation

  • Milagros Loreto & Hugo Aponte & Debora Cores & Marcos Raydan, 2017. "Nonsmooth spectral gradient methods for unconstrained optimization," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 5(4), pages 529-553, December.
  • Handle: RePEc:spr:eurjco:v:5:y:2017:i:4:d:10.1007_s13675-017-0080-8
    DOI: 10.1007/s13675-017-0080-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s13675-017-0080-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s13675-017-0080-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Birgin, Ernesto G. & Martínez, Jose Mario & Raydan, Marcos, 2014. "Spectral Projected Gradient Methods: Review and Perspectives," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 60(i03).
    2. L. Grippo & F. Rinaldi, 2015. "A class of derivative-free nonmonotone optimization algorithms employing coordinate rotations and gradient approximations," Computational Optimization and Applications, Springer, vol. 60(1), pages 1-33, January.
    3. J. V. Burke & A. S. Lewis & M. L. Overton, 2002. "Approximating Subdifferentials by Random Sampling of Gradients," Mathematics of Operations Research, INFORMS, vol. 27(3), pages 567-584, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elias S. Helou & Sandra A. Santos & Lucas E. A. Simões, 2018. "A fast gradient and function sampling method for finite-max functions," Computational Optimization and Applications, Springer, vol. 71(3), pages 673-717, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roberto Andreani & Marcos Raydan, 2021. "Properties of the delayed weighted gradient method," Computational Optimization and Applications, Springer, vol. 78(1), pages 167-180, January.
    2. Filippozzi, Rafaela & Gonçalves, Douglas S. & Santos, Luiz-Rafael, 2023. "First-order methods for the convex hull membership problem," European Journal of Operational Research, Elsevier, vol. 306(1), pages 17-33.
    3. W. Hare & J. Nutini, 2013. "A derivative-free approximate gradient sampling algorithm for finite minimax problems," Computational Optimization and Applications, Springer, vol. 56(1), pages 1-38, September.
    4. Andrej Čopar & Blaž Zupan & Marinka Zitnik, 2019. "Fast optimization of non-negative matrix tri-factorization," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-15, June.
    5. Wei Bian & Xiaojun Chen, 2017. "Optimality and Complexity for Constrained Optimization Problems with Nonconvex Regularization," Mathematics of Operations Research, INFORMS, vol. 42(4), pages 1063-1084, November.
    6. N. Krejić & E. H. M. Krulikovski & M. Raydan, 2023. "A Low-Cost Alternating Projection Approach for a Continuous Formulation of Convex and Cardinality Constrained Optimization," SN Operations Research Forum, Springer, vol. 4(4), pages 1-24, December.
    7. Pospíšil, Lukáš & Dostál, Zdeněk, 2018. "The projected Barzilai–Borwein method with fall-back for strictly convex QCQP problems with separable constraints," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 145(C), pages 79-89.
    8. Dmitriy Drusvyatskiy & Alexander D. Ioffe & Adrian S. Lewis, 2015. "Clarke Subgradients for Directionally Lipschitzian Stratifiable Functions," Mathematics of Operations Research, INFORMS, vol. 40(2), pages 328-349, February.
    9. W. L. Hare & A. S. Lewis, 2005. "Estimating Tangent and Normal Cones Without Calculus," Mathematics of Operations Research, INFORMS, vol. 30(4), pages 785-799, November.
    10. Nataša Krejić & Nataša Krklec Jerinkić, 2019. "Spectral projected gradient method for stochastic optimization," Journal of Global Optimization, Springer, vol. 73(1), pages 59-81, January.
    11. Na Zhao & Qingzhi Yang & Yajun Liu, 2017. "Computing the generalized eigenvalues of weakly symmetric tensors," Computational Optimization and Applications, Springer, vol. 66(2), pages 285-307, March.
    12. Giovanni Colombo & Antonio Marigonda & Peter R. Wolenski, 2013. "The Clarke Generalized Gradient for Functions Whose Epigraph Has Positive Reach," Mathematics of Operations Research, INFORMS, vol. 38(3), pages 451-468, August.
    13. J. M. Martínez & M. Raydan, 2017. "Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization," Journal of Global Optimization, Springer, vol. 68(2), pages 367-385, June.
    14. J. Martínez & M. Raydan, 2015. "Separable cubic modeling and a trust-region strategy for unconstrained minimization with impact in global optimization," Journal of Global Optimization, Springer, vol. 63(2), pages 319-342, October.
    15. V. S. Amaral & R. Andreani & E. G. Birgin & D. S. Marcondes & J. M. Martínez, 2022. "On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization," Journal of Global Optimization, Springer, vol. 84(3), pages 527-561, November.
    16. Sajjad Kazemi & Nader Kanzi, 2018. "Constraint Qualifications and Stationary Conditions for Mathematical Programming with Non-differentiable Vanishing Constraints," Journal of Optimization Theory and Applications, Springer, vol. 179(3), pages 800-819, December.
    17. Michael R. Metel & Akiko Takeda, 2022. "Perturbed Iterate SGD for Lipschitz Continuous Loss Functions," Journal of Optimization Theory and Applications, Springer, vol. 195(2), pages 504-547, November.
    18. di Serafino, Daniela & Ruggiero, Valeria & Toraldo, Gerardo & Zanni, Luca, 2018. "On the steplength selection in gradient methods for unconstrained optimization," Applied Mathematics and Computation, Elsevier, vol. 318(C), pages 176-195.
    19. Wolfgang Schadner, 2021. "Feasible Implied Correlation Matrices from Factor Structures," Papers 2107.00427, arXiv.org.
    20. Marco Viola & Mara Sangiovanni & Gerardo Toraldo & Mario R. Guarracino, 2019. "Semi-supervised generalized eigenvalues classification," Annals of Operations Research, Springer, vol. 276(1), pages 249-266, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:eurjco:v:5:y:2017:i:4:d:10.1007_s13675-017-0080-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.