IDEAS home Printed from https://ideas.repec.org/a/spr/coopap/v81y2022i1d10.1007_s10589-021-00322-2.html
   My bibliography  Save this article

On large-scale unconstrained optimization and arbitrary regularization

Author

Listed:
  • J. M. Martínez

    (IMECC–UNICAMP, University of Campinas)

  • L. T. Santos

    (IMECC–UNICAMP, University of Campinas)

Abstract

We present a new algorithm for large-scale unconstrained minimization that, at each iteration, minimizes, approximately, a quadratic model of the objective function plus a regularization term, not necessarily based on a norm. We prove convergence assuming only gradient continuity and complexity results assuming Lipschitz conditions. For solving the subproblems in the case of regularizations based on the 3-norm, we introduce a new method that quickly obtains the approximate solutions required by the theory. We present numerical experiments.

Suggested Citation

  • J. M. Martínez & L. T. Santos, 2022. "On large-scale unconstrained optimization and arbitrary regularization," Computational Optimization and Applications, Springer, vol. 81(1), pages 1-30, January.
    • Handle: RePEc:spr:coopap:v:81:y:2022:i:1:d:10.1007_s10589-021-00322-2
    DOI: 10.1007/s10589-021-00322-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10589-021-00322-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10589-021-00322-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. El Houcine Bergou & Youssef Diouane & Serge Gratton, 2018. "A Line-Search Algorithm Inspired by the Adaptive Cubic Regularization Framework and Complexity Analysis," Journal of Optimization Theory and Applications, Springer, vol. 178(3), pages 885-913, September.
    2. J. M. Martínez & M. Raydan, 2017. "Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization," Journal of Global Optimization, Springer, vol. 68(2), pages 367-385, June.
    3. E. Bergou & Y. Diouane & S. Gratton, 2017. "On the use of the energy norm in trust-region and adaptive cubic regularization subproblems," Computational Optimization and Applications, Springer, vol. 68(3), pages 533-554, December.
    4. E. G. Birgin & J. M. Martínez, 2019. "A Newton-like method with mixed factorizations and cubic regularization for unconstrained minimization," Computational Optimization and Applications, Springer, vol. 73(3), pages 707-753, July.
    5. NESTEROV, Yurii & POLYAK, B.T., 2006. "Cubic regularization of Newton method and its global performance," LIDAM Reprints CORE 1927, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    6. Geovani N. GRAPIGLIA & Yurii NESTEROV, 2017. "Regularized Newton methods for minimizing functions with Hölder continuous Hessians," LIDAM Reprints CORE 2846, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. V. S. Amaral & R. Andreani & E. G. Birgin & D. S. Marcondes & J. M. Martínez, 2022. "On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization," Journal of Global Optimization, Springer, vol. 84(3), pages 527-561, November.
    2. E. G. Birgin & J. M. Martínez, 2019. "A Newton-like method with mixed factorizations and cubic regularization for unconstrained minimization," Computational Optimization and Applications, Springer, vol. 73(3), pages 707-753, July.
    3. C. P. Brás & J. M. Martínez & M. Raydan, 2020. "Large-scale unconstrained optimization using separable cubic modeling and matrix-free subspace minimization," Computational Optimization and Applications, Springer, vol. 75(1), pages 169-205, January.
    4. Seonho Park & Seung Hyun Jung & Panos M. Pardalos, 2020. "Combining Stochastic Adaptive Cubic Regularization with Negative Curvature for Nonconvex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 184(3), pages 953-971, March.
    5. Chuan He & Heng Huang & Zhaosong Lu, 2024. "A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization," Computational Optimization and Applications, Springer, vol. 89(3), pages 843-894, December.
    6. Yonggang Pei & Shaofang Song & Detong Zhu, 2023. "A sequential adaptive regularisation using cubics algorithm for solving nonlinear equality constrained optimization," Computational Optimization and Applications, Springer, vol. 84(3), pages 1005-1033, April.
    7. Nicholas I. M. Gould & Tyrone Rees & Jennifer A. Scott, 2019. "Convergence and evaluation-complexity analysis of a regularized tensor-Newton method for solving nonlinear least-squares problems," Computational Optimization and Applications, Springer, vol. 73(1), pages 1-35, May.
    8. Rujun Jiang & Man-Chung Yue & Zhishuo Zhou, 2021. "An accelerated first-order method with complexity analysis for solving cubic regularization subproblems," Computational Optimization and Applications, Springer, vol. 79(2), pages 471-506, June.
    9. Nikita Doikov & Yurii Nesterov, 2021. "Minimizing Uniformly Convex Functions by Cubic Regularization of Newton Method," Journal of Optimization Theory and Applications, Springer, vol. 189(1), pages 317-339, April.
    10. Anton Rodomanov & Yurii Nesterov, 2020. "Smoothness Parameter of Power of Euclidean Norm," Journal of Optimization Theory and Applications, Springer, vol. 185(2), pages 303-326, May.
    11. Geovani N. Grapiglia & Ekkehard W. Sachs, 2017. "On the worst-case evaluation complexity of non-monotone line search algorithms," Computational Optimization and Applications, Springer, vol. 68(3), pages 555-577, December.
    12. El Houcine Bergou & Youssef Diouane & Serge Gratton, 2018. "A Line-Search Algorithm Inspired by the Adaptive Cubic Regularization Framework and Complexity Analysis," Journal of Optimization Theory and Applications, Springer, vol. 178(3), pages 885-913, September.
    13. Silvia Berra & Alessandro Torraca & Federico Benvenuto & Sara Sommariva, 2024. "Combined Newton-Gradient Method for Constrained Root-Finding in Chemical Reaction Networks," Journal of Optimization Theory and Applications, Springer, vol. 200(1), pages 404-427, January.
    14. Ariizumi, Shumpei & Yamakawa, Yuya & Yamashita, Nobuo, 2024. "Convergence properties of Levenberg–Marquardt methods with generalized regularization terms," Applied Mathematics and Computation, Elsevier, vol. 463(C).
    15. Bo Jiang & Tianyi Lin & Shiqian Ma & Shuzhong Zhang, 2019. "Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis," Computational Optimization and Applications, Springer, vol. 72(1), pages 115-157, January.
    16. Kenji Ueda & Nobuo Yamashita, 2012. "Global Complexity Bound Analysis of the Levenberg–Marquardt Method for Nonsmooth Equations and Its Application to the Nonlinear Complementarity Problem," Journal of Optimization Theory and Applications, Springer, vol. 152(2), pages 450-467, February.
    17. Liaoyuan Zeng & Ting Kei Pong, 2022. "$$\rho$$ ρ -regularization subproblems: strong duality and an eigensolver-based algorithm," Computational Optimization and Applications, Springer, vol. 81(2), pages 337-368, March.
    18. Fedor Stonyakin & Ilya Kuruzov & Boris Polyak, 2023. "Stopping Rules for Gradient Methods for Non-convex Problems with Additive Noise in Gradient," Journal of Optimization Theory and Applications, Springer, vol. 198(2), pages 531-551, August.
    19. Yuquan Chen & Yunkang Sun & Bing Wang, 2023. "Improving the Performance of Optimization Algorithms Using the Adaptive Fixed-Time Scheme and Reset Scheme," Mathematics, MDPI, vol. 11(22), pages 1-16, November.
    20. Yurii Nesterov, 2021. "Superfast Second-Order Methods for Unconstrained Convex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 191(1), pages 1-30, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:coopap:v:81:y:2022:i:1:d:10.1007_s10589-021-00322-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.