IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v202y2024i1d10.1007_s10957-023-02199-z.html
   My bibliography  Save this article

On the Rate of Convergence of the Difference-of-Convex Algorithm (DCA)

Author

Listed:
  • Hadi Abbaszadehpeivasti

    (Tilburg University)

  • Etienne Klerk

    (Tilburg University)

  • Moslem Zamani

    (Tilburg University)

Abstract

In this paper, we study the non-asymptotic convergence rate of the DCA (difference-of-convex algorithm), also known as the convex–concave procedure, with two different termination criteria that are suitable for smooth and non-smooth decompositions, respectively. The DCA is a popular algorithm for difference-of-convex (DC) problems and known to converge to a stationary point of the objective under some assumptions. We derive a worst-case convergence rate of $$O(1/\sqrt{N})$$ O ( 1 / N ) after N iterations of the objective gradient norm for certain classes of DC problems, without assuming strong convexity in the DC decomposition and give an example which shows the convergence rate is exact. We also provide a new convergence rate of O(1/N) for the DCA with the second termination criterion. Moreover, we derive a new linear convergence rate result for the DCA under the assumption of the Polyak–Łojasiewicz inequality. The novel aspect of our analysis is that it employs semidefinite programming performance estimation.

Suggested Citation

  • Hadi Abbaszadehpeivasti & Etienne Klerk & Moslem Zamani, 2024. "On the Rate of Convergence of the Difference-of-Convex Algorithm (DCA)," Journal of Optimization Theory and Applications, Springer, vol. 202(1), pages 475-496, July.
    • Handle: RePEc:spr:joptap:v:202:y:2024:i:1:d:10.1007_s10957-023-02199-z
    DOI: 10.1007/s10957-023-02199-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-023-02199-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10957-023-02199-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pey-Chun Chen & Pierre Hansen & Brigitte Jaumard & Hoang Tuy, 1998. "Solution of the Multisource Weber and Conditional Weber Problems by D.-C. Programming," Operations Research, INFORMS, vol. 46(4), pages 548-562, August.
    2. de Klerk, Etienne & Glineur, Francois & Taylor, Adrien, 2020. "Worst-case convergence analysis of inexact gradient and Newton methods through semidefinite programming performance estimation," Other publications TiSEM 03a65c83-e88f-4723-a50e-2, Tilburg University, School of Economics and Management.
    3. Hoai An Le Thi & Manh Cuong Nguyen, 2017. "DCA based algorithms for feature selection in multi-class support vector machine," Annals of Operations Research, Springer, vol. 249(1), pages 273-300, February.
    4. TAYLOR, Adrien B. & HENDRICKX, Julien M. & François GLINEUR, 2016. "Exact worst-case performance of first-order methods for composite convex optimization," LIDAM Discussion Papers CORE 2016052, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    5. Yurii Nesterov, 2018. "Lectures on Convex Optimization," Springer Optimization and Its Applications, Springer, edition 2, number 978-3-319-91578-4, December.
    6. de Klerk, Etienne & Glineur, Francois & Taylor, Adrien, 2020. "Worst-case convergence analysis of inexact gradient and Newton methods through semidefinite programming performance estimation," Other publications TiSEM 03a65c83-e88f-4723-a50e-2, Tilburg University, School of Economics and Management.
    7. de Klerk, Etienne & Glineur, Francois & Taylor, Adrien, 2017. "On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions," Other publications TiSEM 8cc0e8dd-b6cd-4f4f-9dcb-9, Tilburg University, School of Economics and Management.
    8. DE KLERK, Etienne & GLINEUR, François & TAYLOR, Adrien B., 2016. "On the Worst-case Complexity of the Gradient Method with Exact Line Search for Smooth Strongly Convex Functions," LIDAM Discussion Papers CORE 2016027, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    9. Annabella Astorino & Antonio Fuduli & Manlio Gaudioso, 2012. "Margin maximization in spherical separation," Computational Optimization and Applications, Springer, vol. 53(2), pages 301-322, October.
    10. Taylor, A. & Hendrickx, J. & Glineur, F., 2015. "Smooth Strongly Convex Interpolation and Exact Worst-case Performance of First-order Methods," LIDAM Discussion Papers CORE 2015013, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    11. R. Horst & N. V. Thoai, 1999. "DC Programming: Overview," Journal of Optimization Theory and Applications, Springer, vol. 103(1), pages 1-43, October.
    12. Abbaszadehpeivasti, Hadi & de Klerk, Etienne & Zamani, Moslem, 2022. "The exact worst-case convergence rate of the gradient method with fixed step lengths for L-smooth functions," Other publications TiSEM 061688c6-f97c-4024-bb5b-1, Tilburg University, School of Economics and Management.
    13. Jong-Shi Pang & Meisam Razaviyayn & Alberth Alvarado, 2017. "Computing B-Stationary Points of Nonsmooth DC Programs," Mathematics of Operations Research, INFORMS, vol. 42(1), pages 95-118, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abbaszadehpeivasti, Hadi, 2024. "Performance analysis of optimization methods for machine learning," Other publications TiSEM 3050a62d-1a1f-494e-99ef-7, Tilburg University, School of Economics and Management.
    2. Roland Hildebrand, 2021. "Optimal step length for the Newton method: case of self-concordant functions," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 94(2), pages 253-279, October.
    3. Abbaszadehpeivasti, Hadi & de Klerk, Etienne & Zamani, Moslem, 2022. "The exact worst-case convergence rate of the gradient method with fixed step lengths for L-smooth functions," Other publications TiSEM 061688c6-f97c-4024-bb5b-1, Tilburg University, School of Economics and Management.
    4. Sandra S. Y. Tan & Antonios Varvitsiotis & Vincent Y. F. Tan, 2021. "Analysis of Optimization Algorithms via Sum-of-Squares," Journal of Optimization Theory and Applications, Springer, vol. 190(1), pages 56-81, July.
    5. André Uschmajew & Bart Vandereycken, 2022. "A Note on the Optimal Convergence Rate of Descent Methods with Fixed Step Sizes for Smooth Strongly Convex Functions," Journal of Optimization Theory and Applications, Springer, vol. 194(1), pages 364-373, July.
    6. Adrien B. Taylor & Julien M. Hendrickx & François Glineur, 2018. "Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization," Journal of Optimization Theory and Applications, Springer, vol. 178(2), pages 455-476, August.
    7. Guoyong Gu & Junfeng Yang, 2024. "Tight Ergodic Sublinear Convergence Rate of the Relaxed Proximal Point Algorithm for Monotone Variational Inequalities," Journal of Optimization Theory and Applications, Springer, vol. 202(1), pages 373-387, July.
    8. Kaisa Joki & Adil M. Bagirov & Napsu Karmitsa & Marko M. Mäkelä, 2017. "A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes," Journal of Global Optimization, Springer, vol. 68(3), pages 501-535, July.
    9. F. Mashkoorzadeh & N. Movahedian & S. Nobakhtian, 2022. "The DTC (difference of tangentially convex functions) programming: optimality conditions," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(2), pages 270-295, July.
    10. Shota Takahashi & Mituhiro Fukuda & Mirai Tanaka, 2022. "New Bregman proximal type algorithms for solving DC optimization problems," Computational Optimization and Applications, Springer, vol. 83(3), pages 893-931, December.
    11. Manlio Gaudioso & Giovanni Giallombardo & Giovanna Miglionico & Adil M. Bagirov, 2018. "Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations," Journal of Global Optimization, Springer, vol. 71(1), pages 37-55, May.
    12. A. M. Bagirov & N. Hoseini Monjezi & S. Taheri, 2021. "An augmented subgradient method for minimizing nonsmooth DC functions," Computational Optimization and Applications, Springer, vol. 80(2), pages 411-438, November.
    13. João Carlos O. Souza & Paulo Roberto Oliveira & Antoine Soubeyran, 2016. "Global convergence of a proximal linearized algorithm for difference of convex functions," Post-Print hal-01440298, HAL.
    14. Abbaszadehpeivasti, Hadi & de Klerk, Etienne & Zamani, Moslem, 2023. "Convergence rate analysis of randomized and cyclic coordinate descent for convex optimization through semidefinite programming," Other publications TiSEM 88512ac0-c26a-4a99-b840-3, Tilburg University, School of Economics and Management.
    15. Rafael Blanquero & Emilio Carrizosa & Pierre Hansen, 2009. "Locating Objects in the Plane Using Global Optimization Techniques," Mathematics of Operations Research, INFORMS, vol. 34(4), pages 837-858, November.
    16. Xiangyu Cui & Xun Li & Duan Li & Yun Shi, 2014. "Time Consistent Behavior Portfolio Policy for Dynamic Mean-Variance Formulation," Papers 1408.6070, arXiv.org, revised Aug 2015.
    17. Nadja Harms & Tim Hoheisel & Christian Kanzow, 2015. "On a Smooth Dual Gap Function for a Class of Player Convex Generalized Nash Equilibrium Problems," Journal of Optimization Theory and Applications, Springer, vol. 166(2), pages 659-685, August.
    18. Pawel Kalczynski & Jack Brimberg & Zvi Drezner, 2022. "Less is more: discrete starting solutions in the planar p-median problem," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(1), pages 34-59, April.
    19. Boglárka G.-Tóth & Kristóf Kovács, 2016. "Solving a Huff-like Stackelberg location problem on networks," Journal of Global Optimization, Springer, vol. 64(2), pages 233-247, February.
    20. Xin Jiang & Lieven Vandenberghe, 2022. "Bregman primal–dual first-order method and application to sparse semidefinite programming," Computational Optimization and Applications, Springer, vol. 81(1), pages 127-159, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:202:y:2024:i:1:d:10.1007_s10957-023-02199-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.