IDEAS home Printed from https://ideas.repec.org/a/spr/coopap/v89y2024i1d10.1007_s10589-024-00589-1.html
   My bibliography  Save this article

The Levenberg–Marquardt method: an overview of modern convergence theories and more

Author

Listed:
  • Andreas Fischer

    (Technische Universität Dresden)

  • Alexey F. Izmailov

    (Lomonosov Moscow State University (MSU)
    Derzhavin Tambov State University (TSU))

  • Mikhail V. Solodov

    (IMPA – Instituto de Matemática Pura e Aplicada)

Abstract

The Levenberg–Marquardt method is a fundamental regularization technique for the Newton method applied to nonlinear equations, possibly constrained, and possibly with singular or even nonisolated solutions. We review the literature on the subject, in particular relating to each other various convergence frameworks and results. In this process, the analysis is performed from a unified perspective, and some new results are obtained as well. We discuss smooth and piecewise smooth equations, inexact solution of subproblems, and globalization techniques. Attention is also paid to the LP-Newton method, because of its relations to the Levenberg–Marquardt method.

Suggested Citation

  • Andreas Fischer & Alexey F. Izmailov & Mikhail V. Solodov, 2024. "The Levenberg–Marquardt method: an overview of modern convergence theories and more," Computational Optimization and Applications, Springer, vol. 89(1), pages 33-67, September.
    • Handle: RePEc:spr:coopap:v:89:y:2024:i:1:d:10.1007_s10589-024-00589-1
    DOI: 10.1007/s10589-024-00589-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10589-024-00589-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10589-024-00589-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. El Houcine Bergou & Youssef Diouane & Vyacheslav Kungurtsev, 2020. "Convergence and Complexity Analysis of a Levenberg–Marquardt Algorithm for Inverse Problems," Journal of Optimization Theory and Applications, Springer, vol. 185(3), pages 927-944, June.
    2. Francisco Facchinei & Andreas Fischer & Markus Herrich, 2013. "A family of Newton methods for nonsmooth constrained systems with nonisolated solutions," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 77(3), pages 433-443, June.
    3. R. Behling & A. Fischer & M. Herrich & A. Iusem & Y. Ye, 2014. "A Levenberg-Marquardt method with approximate projections," Computational Optimization and Applications, Springer, vol. 59(1), pages 5-26, October.
    4. Naoki Marumo & Takayuki Okuno & Akiko Takeda, 2023. "Majorization-minimization-based Levenberg–Marquardt method for constrained nonlinear least squares," Computational Optimization and Applications, Springer, vol. 84(3), pages 833-874, April.
    5. Ariizumi, Shumpei & Yamakawa, Yuya & Yamashita, Nobuo, 2024. "Convergence properties of Levenberg–Marquardt methods with generalized regularization terms," Applied Mathematics and Computation, Elsevier, vol. 463(C).
    6. Letícia Becher & Damián Fernández & Alberto Ramos, 2023. "A trust-region LP-Newton method for constrained nonsmooth equations under Hölder metric subregularity," Computational Optimization and Applications, Springer, vol. 86(2), pages 711-743, November.
    7. Jifeng Bao & Carisa Kwok Wai Yu & Jinhua Wang & Yaohua Hu & Jen-Chih Yao, 2019. "Modified inexact Levenberg–Marquardt methods for solving nonlinear least squares problems," Computational Optimization and Applications, Springer, vol. 74(2), pages 547-582, November.
    8. Marguerite Frank & Philip Wolfe, 1956. "An algorithm for quadratic programming," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 3(1‐2), pages 95-110, March.
    9. Roger Behling & Douglas S. Gonçalves & Sandra A. Santos, 2019. "Local Convergence Analysis of the Levenberg–Marquardt Framework for Nonzero-Residue Nonlinear Least-Squares Problems Under an Error Bound Condition," Journal of Optimization Theory and Applications, Springer, vol. 183(3), pages 1099-1122, December.
    10. Axel Dreves & Francisco Facchinei & Andreas Fischer & Markus Herrich, 2014. "A new error bound result for Generalized Nash Equilibrium Problems and its algorithmic application," Computational Optimization and Applications, Springer, vol. 59(1), pages 63-84, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Naoki Marumo & Takayuki Okuno & Akiko Takeda, 2023. "Majorization-minimization-based Levenberg–Marquardt method for constrained nonlinear least squares," Computational Optimization and Applications, Springer, vol. 84(3), pages 833-874, April.
    2. Fabiana R. Oliveira & Orizon P. Ferreira & Gilson N. Silva, 2019. "Newton’s method with feasible inexact projections for solving constrained generalized equations," Computational Optimization and Applications, Springer, vol. 72(1), pages 159-177, January.
    3. A. Fischer & M. Herrich & A. F. Izmailov & W. Scheck & M. V. Solodov, 2018. "A globally convergent LP-Newton method for piecewise smooth constrained equations: escaping nonstationary accumulation points," Computational Optimization and Applications, Springer, vol. 69(2), pages 325-349, March.
    4. Yaohua Hu & Jiawen Li & Carisa Kwok Wai Yu, 2020. "Convergence rates of subgradient methods for quasi-convex optimization problems," Computational Optimization and Applications, Springer, vol. 77(1), pages 183-212, September.
    5. Andreas Fischer & Markus Herrich & Alexey Izmailov & Mikhail Solodov, 2016. "Convergence conditions for Newton-type methods applied to complementarity systems with nonisolated solutions," Computational Optimization and Applications, Springer, vol. 63(2), pages 425-459, March.
    6. Boos, Everton & Gonçalves, Douglas S. & Bazán, Fermín S.V., 2024. "Levenberg-Marquardt method with singular scaling and applications," Applied Mathematics and Computation, Elsevier, vol. 474(C).
    7. Guillaume Sagnol & Edouard Pauwels, 2019. "An unexpected connection between Bayes A-optimal designs and the group lasso," Statistical Papers, Springer, vol. 60(2), pages 565-584, April.
    8. Abdelfettah Laouzai & Rachid Ouafi, 2022. "A prediction model for atmospheric pollution reduction from urban traffic," Environment and Planning B, , vol. 49(2), pages 566-584, February.
    9. Chou, Chang-Chi & Chiang, Wen-Chu & Chen, Albert Y., 2022. "Emergency medical response in mass casualty incidents considering the traffic congestions in proximity on-site and hospital delays," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 158(C).
    10. Francesco Rinaldi & Damiano Zeffiro, 2023. "Avoiding bad steps in Frank-Wolfe variants," Computational Optimization and Applications, Springer, vol. 84(1), pages 225-264, January.
    11. Beck, Yasmine & Ljubić, Ivana & Schmidt, Martin, 2023. "A survey on bilevel optimization under uncertainty," European Journal of Operational Research, Elsevier, vol. 311(2), pages 401-426.
    12. Tiến-Sơn Phạm, 2019. "Optimality Conditions for Minimizers at Infinity in Polynomial Programming," Management Science, INFORMS, vol. 44(4), pages 1381-1395, November.
    13. Filippozzi, Rafaela & Gonçalves, Douglas S. & Santos, Luiz-Rafael, 2023. "First-order methods for the convex hull membership problem," European Journal of Operational Research, Elsevier, vol. 306(1), pages 17-33.
    14. Ke, Ginger Y. & Zhang, Huiwen & Bookbinder, James H., 2020. "A dual toll policy for maintaining risk equity in hazardous materials transportation with fuzzy incident rate," International Journal of Production Economics, Elsevier, vol. 227(C).
    15. Friesz, Terry L. & Tourreilles, Francisco A. & Han, Anthony Fu-Wha, 1979. "Multi-Criteria Optimization Methods in Transport Project Evaluation: The Case of Rural Roads in Developing Countries," Transportation Research Forum Proceedings 1970s 318817, Transportation Research Forum.
    16. Damian Clarke & Daniel Paila~nir & Susan Athey & Guido Imbens, 2023. "Synthetic Difference In Differences Estimation," Papers 2301.11859, arXiv.org, revised Feb 2023.
    17. Ali Fattahi & Sriram Dasu & Reza Ahmadi, 2019. "Mass Customization and “Forecasting Options’ Penetration Rates Problem”," Operations Research, INFORMS, vol. 67(4), pages 1120-1134, July.
    18. Pokojovy, Michael & Jobe, J. Marcus, 2022. "A robust deterministic affine-equivariant algorithm for multivariate location and scatter," Computational Statistics & Data Analysis, Elsevier, vol. 172(C).
    19. Wei-jie Cong & Le Wang & Hui Sun, 2020. "Rank-two update algorithm versus Frank–Wolfe algorithm with away steps for the weighted Euclidean one-center problem," Computational Optimization and Applications, Springer, vol. 75(1), pages 237-262, January.
    20. Bo Jiang & Tianyi Lin & Shiqian Ma & Shuzhong Zhang, 2019. "Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis," Computational Optimization and Applications, Springer, vol. 72(1), pages 115-157, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:coopap:v:89:y:2024:i:1:d:10.1007_s10589-024-00589-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.