IDEAS home Printed from https://ideas.repec.org/a/spr/jglopt/v86y2023i4d10.1007_s10898-023-01299-4.html
   My bibliography  Save this article

An extrapolated proximal iteratively reweighted method for nonconvex composite optimization problems

Author

Listed:
  • Zhili Ge

    (Nanjing Normal University of Special Education)

  • Zhongming Wu

    (Nanjing University of Information Science and Technology)

  • Xin Zhang

    (Suqian University)

  • Qin Ni

    (Nanjing University of Aeronautics and Astronautics)

Abstract

We consider a class of problems where the objective function is the sum of a smooth function and a composition of nonconvex and nonsmooth functions. Such optimization problems arise frequently in machine learning and data processing. The proximal iteratively reweighted method has been widely used and popularized in solving these problems. In this paper, we develop an extrapolated proximal iteratively reweighted method that incorporates two different flexible inertial steps at each iteration. We first prove the subsequential convergence of the proposed method under parameter constraints. Moreover, if the objective function satisfies the Kurdyka-Łojasiewicz property, the global convergence of the new method is established. In addition, we analyze the local convergence rate by making assumptions on the Kurdyka-Łojasiewicz exponent of the objective function. Finally, numerical results on $$l_p$$ l p minimization and feature selection problems are reported to show the effectiveness and superiority of the proposed algorithm.

Suggested Citation

  • Zhili Ge & Zhongming Wu & Xin Zhang & Qin Ni, 2023. "An extrapolated proximal iteratively reweighted method for nonconvex composite optimization problems," Journal of Global Optimization, Springer, vol. 86(4), pages 821-844, August.
    • Handle: RePEc:spr:jglopt:v:86:y:2023:i:4:d:10.1007_s10898-023-01299-4
    DOI: 10.1007/s10898-023-01299-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10898-023-01299-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10898-023-01299-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hédy Attouch & Jérôme Bolte & Patrick Redont & Antoine Soubeyran, 2010. "Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality," Mathematics of Operations Research, INFORMS, vol. 35(2), pages 438-457, May.
    2. Xiaojun Chen & Weijun Zhou, 2014. "Convergence of the reweighted ℓ 1 minimization algorithm for ℓ 2 –ℓ p minimization," Computational Optimization and Applications, Springer, vol. 59(1), pages 47-61, October.
    3. Radu Ioan Boţ & Ernö Robert Csetnek & Szilárd Csaba László, 2016. "An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 4(1), pages 3-25, February.
    4. Ke Guo & Deren Han, 2018. "A note on the Douglas–Rachford splitting method for optimization problems involving hypoconvex functions," Journal of Global Optimization, Springer, vol. 72(3), pages 431-441, November.
    5. Zhongming Wu & Min Li, 2019. "General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems," Computational Optimization and Applications, Springer, vol. 73(1), pages 129-158, May.
    6. Tao Sun & Hao Jiang & Lizhi Cheng, 2017. "Global convergence of proximal iteratively reweighted algorithm," Journal of Global Optimization, Springer, vol. 68(4), pages 815-826, August.
    7. Bo Wen & Xiaoping Xue, 2019. "On the convergence of the iterates of proximal gradient algorithm with extrapolation for convex nonsmooth minimization problems," Journal of Global Optimization, Springer, vol. 75(3), pages 767-787, November.
    8. Zhongming Wu & Chongshou Li & Min Li & Andrew Lim, 2021. "Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems," Journal of Global Optimization, Springer, vol. 79(3), pages 617-644, March.
    9. R. Horst & N. V. Thoai, 1999. "DC Programming: Overview," Journal of Optimization Theory and Applications, Springer, vol. 103(1), pages 1-43, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhongming Wu & Min Li, 2019. "General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems," Computational Optimization and Applications, Springer, vol. 73(1), pages 129-158, May.
    2. Szilárd Csaba László, 2023. "A Forward–Backward Algorithm With Different Inertial Terms for Structured Non-Convex Minimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 198(1), pages 387-427, July.
    3. Tao Sun & Hao Jiang & Lizhi Cheng, 2017. "Global convergence of proximal iteratively reweighted algorithm," Journal of Global Optimization, Springer, vol. 68(4), pages 815-826, August.
    4. Zhongming Wu & Chongshou Li & Min Li & Andrew Lim, 2021. "Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems," Journal of Global Optimization, Springer, vol. 79(3), pages 617-644, March.
    5. Min Li & Zhongming Wu, 2019. "Convergence Analysis of the Generalized Splitting Methods for a Class of Nonconvex Optimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 183(2), pages 535-565, November.
    6. Xianfu Wang & Ziyuan Wang, 2022. "Malitsky-Tam forward-reflected-backward splitting method for nonconvex minimization problems," Computational Optimization and Applications, Springer, vol. 82(2), pages 441-463, June.
    7. Abbaszadehpeivasti, Hadi, 2024. "Performance analysis of optimization methods for machine learning," Other publications TiSEM 3050a62d-1a1f-494e-99ef-7, Tilburg University, School of Economics and Management.
    8. Yitian Qian & Shaohua Pan & Yulan Liu, 2023. "Calmness of partial perturbation to composite rank constraint systems and its applications," Journal of Global Optimization, Springer, vol. 85(4), pages 867-889, April.
    9. Zhongming Wu & Min Li & David Z. W. Wang & Deren Han, 2017. "A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 34(06), pages 1-27, December.
    10. Xue Gao & Xingju Cai & Xiangfeng Wang & Deren Han, 2023. "An alternating structure-adapted Bregman proximal gradient descent algorithm for constrained nonconvex nonsmooth optimization problems and its inertial variant," Journal of Global Optimization, Springer, vol. 87(1), pages 277-300, September.
    11. Peiran Yu & Ting Kei Pong, 2019. "Iteratively reweighted $$\ell _1$$ ℓ 1 algorithms with extrapolation," Computational Optimization and Applications, Springer, vol. 73(2), pages 353-386, June.
    12. Chih-Sheng Chuang & Hongjin He & Zhiyuan Zhang, 2022. "A unified Douglas–Rachford algorithm for generalized DC programming," Journal of Global Optimization, Springer, vol. 82(2), pages 331-349, February.
    13. Shota Takahashi & Mituhiro Fukuda & Mirai Tanaka, 2022. "New Bregman proximal type algorithms for solving DC optimization problems," Computational Optimization and Applications, Springer, vol. 83(3), pages 893-931, December.
    14. Xiangyu Cui & Xun Li & Duan Li & Yun Shi, 2014. "Time Consistent Behavior Portfolio Policy for Dynamic Mean-Variance Formulation," Papers 1408.6070, arXiv.org, revised Aug 2015.
    15. Nadja Harms & Tim Hoheisel & Christian Kanzow, 2015. "On a Smooth Dual Gap Function for a Class of Player Convex Generalized Nash Equilibrium Problems," Journal of Optimization Theory and Applications, Springer, vol. 166(2), pages 659-685, August.
    16. Le Thi Khanh Hien & Duy Nhat Phan & Nicolas Gillis, 2022. "Inertial alternating direction method of multipliers for non-convex non-smooth optimization," Computational Optimization and Applications, Springer, vol. 83(1), pages 247-285, September.
    17. Boglárka G.-Tóth & Kristóf Kovács, 2016. "Solving a Huff-like Stackelberg location problem on networks," Journal of Global Optimization, Springer, vol. 64(2), pages 233-247, February.
    18. Francesco Rinaldi & Damiano Zeffiro, 2023. "Avoiding bad steps in Frank-Wolfe variants," Computational Optimization and Applications, Springer, vol. 84(1), pages 225-264, January.
    19. Rafael Blanquero & Emilio Carrizosa & Amaya Nogales-Gómez & Frank Plastria, 2014. "Single-facility huff location problems on networks," Annals of Operations Research, Springer, vol. 222(1), pages 175-195, November.
    20. J. X. Cruz Neto & P. R. Oliveira & A. Soubeyran & J. C. O. Souza, 2020. "A generalized proximal linearized algorithm for DC functions with application to the optimal size of the firm problem," Annals of Operations Research, Springer, vol. 289(2), pages 313-339, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jglopt:v:86:y:2023:i:4:d:10.1007_s10898-023-01299-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.