IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v182y2019i2d10.1007_s10957-019-01524-9.html
   My bibliography  Save this article

Convergence Rates of Forward–Douglas–Rachford Splitting Method

Author

Listed:
  • Cesare Molinari

    (Normandie Université)

  • Jingwei Liang

    (University of Cambridge)

  • Jalal Fadili

    (Normandie Université)

Abstract

Over the past decades, operator splitting methods have become ubiquitous for non-smooth optimization owing to their simplicity and efficiency. In this paper, we consider the Forward–Douglas–Rachford splitting method and study both global and local convergence rates of this method. For the global rate, we establish a sublinear convergence rate in terms of a Bregman divergence suitably designed for the objective function. Moreover, when specializing to the Forward–Backward splitting, we prove a stronger convergence rate result for the objective function value. Then locally, based on the assumption that the non-smooth part of the optimization problem is partly smooth, we establish local linear convergence of the method. More precisely, we show that the sequence generated by Forward–Douglas–Rachford first (i) identifies a smooth manifold in a finite number of iteration and then (ii) enters a local linear convergence regime, which is for instance characterized in terms of the structure of the underlying active smooth manifold. To exemplify the usefulness of the obtained result, we consider several concrete numerical experiments arising from applicative fields including, for instance, signal/image processing, inverse problems and machine learning.

Suggested Citation

  • Cesare Molinari & Jingwei Liang & Jalal Fadili, 2019. "Convergence Rates of Forward–Douglas–Rachford Splitting Method," Journal of Optimization Theory and Applications, Springer, vol. 182(2), pages 606-639, August.
  • Handle: RePEc:spr:joptap:v:182:y:2019:i:2:d:10.1007_s10957-019-01524-9
    DOI: 10.1007/s10957-019-01524-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-019-01524-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10957-019-01524-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    2. Unknown, 2005. "Forward," 2005 Conference: Slovenia in the EU - Challenges for Agriculture, Food Science and Rural Affairs, November 10-11, 2005, Moravske Toplice, Slovenia 183804, Slovenian Association of Agricultural Economists (DAES).
    3. Jingwei Liang & Jalal Fadili & Gabriel Peyré, 2017. "Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers," Journal of Optimization Theory and Applications, Springer, vol. 172(3), pages 874-913, March.
    4. Samuel Vaiter & Charles Deledalle & Jalal Fadili & Gabriel Peyré & Charles Dossal, 2017. "The degrees of freedom of partly smooth regularizers," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 69(4), pages 791-832, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hongwei Liu & Ting Wang & Zexian Liu, 2022. "Some modified fast iterative shrinkage thresholding algorithms with a new adaptive non-monotone stepsize strategy for nonsmooth and convex minimization problems," Computational Optimization and Applications, Springer, vol. 83(2), pages 651-691, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kenneth Lange & Eric C. Chi & Hua Zhou, 2014. "A Brief Survey of Modern Optimization for Statisticians," International Statistical Review, International Statistical Institute, vol. 82(1), pages 46-70, April.
    2. Jie Chen & Joe Suzuki, 2021. "An Efficient Algorithm for Convex Biclustering," Mathematics, MDPI, vol. 9(23), pages 1-18, November.
    3. Tung Duy Luu & Jalal Fadili & Christophe Chesneau, 2020. "Sharp oracle inequalities for low-complexity priors," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(2), pages 353-397, April.
    4. Tutz, Gerhard & Pößnecker, Wolfgang & Uhlmann, Lorenz, 2015. "Variable selection in general multinomial logit models," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 207-222.
    5. Pilar Lopez-Llompart & G. Mathias Kondolf, 2016. "Encroachments in floodways of the Mississippi River and Tributaries Project," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 81(1), pages 513-542, March.
    6. Cheng, Jianquan & Bertolini, Luca, 2013. "Measuring urban job accessibility with distance decay, competition and diversity," Journal of Transport Geography, Elsevier, vol. 30(C), pages 100-109.
    7. M. De Donno & M. Pratelli, 2006. "A theory of stochastic integration for bond markets," Papers math/0602532, arXiv.org.
    8. Prilly Oktoviany & Robert Knobloch & Ralf Korn, 2021. "A machine learning-based price state prediction model for agricultural commodities using external factors," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 44(2), pages 1063-1085, December.
    9. Michelle Sheran Sylvester, 2007. "The Career and Family Choices of Women: A Dynamic Analysis of Labor Force Participation, Schooling, Marriage and Fertility Decisions," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 10(3), pages 367-399, July.
    10. Henrekson, Magnus & Johansson, Dan, 2010. "Firm Growth, Institutions and Structural Transformation," Ratio Working Papers 150, The Ratio Institute.
    11. Karen K. Lewis, 2011. "Global Asset Pricing," Annual Review of Financial Economics, Annual Reviews, vol. 3(1), pages 435-466, December.
    12. DAVID M. BLAU & WILBERT van der KLAAUW, 2013. "What Determines Family Structure?," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 579-604, January.
    13. Panagiota DIONYSOPOULOU & Georgios SVARNIAS & Theodore PAPAILIAS, 2021. "Total Quality Management In Public Sector, Case Study: Customs Service," Regional Science Inquiry, Hellenic Association of Regional Scientists, vol. 0(1), pages 153-168, June.
    14. Afanasyev, Dmitriy O. & Fedorova, Elena A. & Popov, Viktor U., 2015. "Fine structure of the price–demand relationship in the electricity market: Multi-scale correlation analysis," Energy Economics, Elsevier, vol. 51(C), pages 215-226.
    15. Peter Viggo Jakobsen, 2009. "Small States, Big Influence: The Overlooked Nordic Influence on the Civilian ESDP," Journal of Common Market Studies, Wiley Blackwell, vol. 47(1), pages 81-102, January.
    16. Julie Holland Mortimer, 2007. "Price Discrimination, Copyright Law, and Technological Innovation: Evidence from the Introduction of DVDs," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1307-1350.
    17. Suwan Shen & Xi Feng & Zhong Ren Peng, 2016. "A framework to analyze vulnerability of critical infrastructure to climate change: the case of a coastal community in Florida," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 84(1), pages 589-609, October.
    18. Jean-Bernard Chatelain & Kirsten Ralf, 2017. "Can We Identify the Fed's Preferences?," Working Papers halshs-01549908, HAL.
    19. Billio, Monica & Casarin, Roberto & Osuntuyi, Anthony, 2016. "Efficient Gibbs sampling for Markov switching GARCH models," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 37-57.
    20. Jan Babecký & Fabrizio Coricelli & Roman Horváth, 2009. "Assessing Inflation Persistence: Micro Evidence on an Inflation Targeting Economy," Czech Journal of Economics and Finance (Finance a uver), Charles University Prague, Faculty of Social Sciences, vol. 59(2), pages 102-127, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:182:y:2019:i:2:d:10.1007_s10957-019-01524-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.