IDEAS home Printed from https://ideas.repec.org/a/spr/jglopt/v89y2024i3d10.1007_s10898-023-01361-1.html
   My bibliography  Save this article

The appeals of quadratic majorization–minimization

Author

Listed:
  • Marc C. Robini

    (INSA Lyon)

  • Lihui Wang

    (Guizhou University)

  • Yuemin Zhu

    (INSA Lyon)

Abstract

Majorization–minimization (MM) is a versatile optimization technique that operates on surrogate functions satisfying tangency and domination conditions. Our focus is on differentiable optimization using inexact MM with quadratic surrogates, which amounts to approximately solving a sequence of symmetric positive definite systems. We begin by investigating the convergence properties of this process, from subconvergence to R-linear convergence, with emphasis on tame objectives. Then we provide a numerically stable implementation based on truncated conjugate gradient. Applications to multidimensional scaling and regularized inversion are discussed and illustrated through numerical experiments on graph layout and X-ray tomography. In the end, quadratic MM not only offers solid guarantees of convergence and stability, but is robust to the choice of its control parameters.

Suggested Citation

  • Marc C. Robini & Lihui Wang & Yuemin Zhu, 2024. "The appeals of quadratic majorization–minimization," Journal of Global Optimization, Springer, vol. 89(3), pages 509-558, July.
    • Handle: RePEc:spr:jglopt:v:89:y:2024:i:3:d:10.1007_s10898-023-01361-1
    DOI: 10.1007/s10898-023-01361-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10898-023-01361-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10898-023-01361-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Emilie Chouzenoux & Jean-Christophe Pesquet & Audrey Repetti, 2014. "Variable Metric Forward–Backward Algorithm for Minimizing the Sum of a Differentiable Function and a Convex Function," Journal of Optimization Theory and Applications, Springer, vol. 162(1), pages 107-132, July.
    2. Groenen, Patrick J. F. & van de Velden, Michel, 2016. "Multidimensional Scaling by Majorization: A Review," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 73(i08).
    3. Jan Leeuw, 1984. "Differentiability of Kruskal's stress at a local minimum," Psychometrika, Springer;The Psychometric Society, vol. 49(1), pages 111-113, March.
    4. Jérôme Bolte & Edouard Pauwels, 2016. "Majorization-Minimization Procedures and Convergence of SQP Methods for Semi-Algebraic and Tame Programs," Mathematics of Operations Research, INFORMS, vol. 41(2), pages 442-465, May.
    5. Jan Leeuw, 1988. "Convergence of the majorization method for multidimensional scaling," Journal of Classification, Springer;The Classification Society, vol. 5(2), pages 163-180, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Ochs, 2018. "Local Convergence of the Heavy-Ball Method and iPiano for Non-convex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 177(1), pages 153-180, April.
    2. Silvia Bonettini & Peter Ochs & Marco Prato & Simone Rebegoldi, 2023. "An abstract convergence framework with application to inertial inexact forward–backward methods," Computational Optimization and Applications, Springer, vol. 84(2), pages 319-362, March.
    3. Patrick Groenen & Rudolf Mathar & Willem Heiser, 1995. "The majorization approach to multidimensional scaling for Minkowski distances," Journal of Classification, Springer;The Classification Society, vol. 12(1), pages 3-19, March.
    4. Hanafi, Mohamed & Kiers, Henk A.L., 2006. "Analysis of K sets of data, with differential emphasis on agreement between and within sets," Computational Statistics & Data Analysis, Elsevier, vol. 51(3), pages 1491-1508, December.
    5. Emilie Chouzenoux & Jean-Christophe Pesquet & Audrey Repetti, 2016. "A block coordinate variable metric forward–backward algorithm," Journal of Global Optimization, Springer, vol. 66(3), pages 457-485, November.
    6. Michael J. Greenacre & Patrick J. F. Groenen, 2016. "Weighted Euclidean Biplots," Journal of Classification, Springer;The Classification Society, vol. 33(3), pages 442-459, October.
    7. Simon J. L. Billinge & Phillip M. Duxbury & Douglas S. Gonçalves & Carlile Lavor & Antonio Mucherino, 2018. "Recent results on assigned and unassigned distance geometry with applications to protein molecules and nanostructures," Annals of Operations Research, Springer, vol. 271(1), pages 161-203, December.
    8. Antanas Žilinskas & Julius Žilinskas, 2008. "A hybrid method for multidimensional scaling using city-block distances," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 68(3), pages 429-443, December.
    9. Peter Verboon & Willem Heiser, 1992. "Resistant orthogonal procrustes analysis," Journal of Classification, Springer;The Classification Society, vol. 9(2), pages 237-256, December.
    10. Julius Žilinskas, 2012. "Parallel branch and bound for multidimensional scaling with city-block distances," Journal of Global Optimization, Springer, vol. 54(2), pages 261-274, October.
    11. S. Bonettini & M. Prato & S. Rebegoldi, 2018. "A block coordinate variable metric linesearch based proximal gradient method," Computational Optimization and Applications, Springer, vol. 71(1), pages 5-52, September.
    12. Ching-pei Lee & Stephen J. Wright, 2019. "Inexact Successive quadratic approximation for regularized optimization," Computational Optimization and Applications, Springer, vol. 72(3), pages 641-674, April.
    13. de Leeuw, Jan & Mair, Patrick, 2009. "Multidimensional Scaling Using Majorization: SMACOF in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 31(i03).
    14. Wei Liu & Li Yang & Bo Yu, 2020. "A Lifting-Penalty Method for Quadratic Programming with a Quadratic Matrix Inequality Constraint," Mathematics, MDPI, vol. 8(2), pages 1-11, January.
    15. van den Burg, G.J.J. & Groenen, P.J.F., 2014. "GenSVM: A Generalized Multiclass Support Vector Machine," Econometric Institute Research Papers EI 2014-33, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    16. Sixuan Bai & Minghua Li & Chengwu Lu & Daoli Zhu & Sien Deng, 2022. "The Equivalence of Three Types of Error Bounds for Weakly and Approximately Convex Functions," Journal of Optimization Theory and Applications, Springer, vol. 194(1), pages 220-245, July.
    17. Kiers, Henk A. L., 2002. "Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems," Computational Statistics & Data Analysis, Elsevier, vol. 41(1), pages 157-170, November.
    18. Kagie, M. & van Wezel, M.C. & Groenen, P.J.F., 2007. "A graphical shopping interface bases on product attributes," Econometric Institute Research Papers EI 2007-02, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    19. Shuqin Sun & Ting Kei Pong, 2023. "Doubly iteratively reweighted algorithm for constrained compressed sensing models," Computational Optimization and Applications, Springer, vol. 85(2), pages 583-619, June.
    20. Emilie Chouzenoux & Jean-Baptiste Fest, 2022. "SABRINA: A Stochastic Subspace Majorization-Minimization Algorithm," Journal of Optimization Theory and Applications, Springer, vol. 195(3), pages 919-952, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jglopt:v:89:y:2024:i:3:d:10.1007_s10898-023-01361-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.