IDEAS home Printed from https://ideas.repec.org/p/cor/louvco/2012002.html
   My bibliography  Save this paper

Subgradient methods for huge-scale optimization problems

Author

Listed:
  • NESTEROV, Yu.

    (Université catholique de Louvain, CORE & INMA, B-1348 Louvain-la-Neuve, Belgium)

Abstract

We consider a new class of huge-scale problems, the problems with sparse subgradients. The most important functions of this type are piece-wise linear. For optimization problems with uniform sparsity of corresponding linear operators, we suggest a very efficient implementation of subgradient iterations, which total cost depends logarithmically in the dimension. This technique is based on a recursive update of the results of matrix/vector products and the values of symmetric functions. It works well, for example, for matrices with few nonzero diagonals and for max-type functions. We show that the updating technique can be efficiently coupled with the simplest subgradient methods, the unconstrained minimization method by B. Polyak, and the constrained minimization scheme by N. Shor. Similar results can be obtained for a new non- smooth random variant of a coordinate descent scheme. We present also the promising results of preliminary computational experiments.

Suggested Citation

  • NESTEROV, Yu., 2012. "Subgradient methods for huge-scale optimization problems," LIDAM Discussion Papers CORE 2012002, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
  • Handle: RePEc:cor:louvco:2012002
    as

    Download full text from publisher

    File URL: https://sites.uclouvain.be/core/publications/coredp/coredp2012.html
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fleurbaey,Marc & Maniquet,François, 2011. "A Theory of Fairness and Social Welfare," Cambridge Books, Cambridge University Press, number 9780521715348, October.
    2. Belleflamme,Paul & Peitz,Martin, 2015. "Industrial Organization," Cambridge Books, Cambridge University Press, number 9781107687899, September.
    3. Zhi-Quan Luo & Paul Tseng, 1993. "On the Convergence Rate of Dual Ascent Methods for Linearly Constrained Convex Minimization," Mathematics of Operations Research, INFORMS, vol. 18(4), pages 846-867, November.
    4. Duranton, Gilles & Martin, Philippe & Mayer, Thierry & Mayneris, Florian, 2010. "The Economics of Clusters: Lessons from the French Experience," OUP Catalogue, Oxford University Press, number 9780199592203.
    5. Belleflamme,Paul & Peitz,Martin, 2010. "Industrial Organization," Cambridge Books, Cambridge University Press, number 9780521681599, November.
    6. Huriot,Jean-Marie & Thisse,Jacques-François (ed.), 2009. "Economics of Cities," Cambridge Books, Cambridge University Press, number 9780521118279, October.
    7. NESTEROV, Yu., 2005. "Smooth minimization of non-smooth functions," LIDAM Reprints CORE 1819, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pavel Dvurechensky & Alexander Gasnikov, 2016. "Stochastic Intermediate Gradient Method for Convex Problems with Stochastic Inexact Oracle," Journal of Optimization Theory and Applications, Springer, vol. 171(1), pages 121-145, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. DEVOLDER, Olivier, 2011. "Stochastic first order methods in smooth convex optimization," LIDAM Discussion Papers CORE 2011070, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. Dimitri Paolini & Pasquale Pistone & Giuseppe Pulina & Martin Zagler, 2016. "Tax treaties with developing countries and the allocation of taxing rights," European Journal of Law and Economics, Springer, vol. 42(3), pages 383-404, December.
    3. Pierre Pestieau & Gregory Ponthiere, 2012. "The Public Economics of Increasing Longevity," Hacienda Pública Española / Review of Public Economics, IEF, vol. 200(1), pages 41-74, March.
    4. Luc Bauwens & Dimitris Korobilis, 2013. "Bayesian methods," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 16, pages 363-380, Edward Elgar Publishing.
    5. Hougaard, Jens Leth & Moreno-Ternero, Juan D. & Østerdal, Lars Peter, 2012. "A unifying framework for the problem of adjudicating conflicting claims," Journal of Mathematical Economics, Elsevier, vol. 48(2), pages 107-114.
    6. Pierre M. Picard & Tim Worrall, 2010. "Sustainable Migration Policies," DEM Discussion Paper Series 10-12, Department of Economics at the University of Luxembourg.
    7. Bocart, Fabian Y.R.P. & Hafner, Christian M., 2012. "Econometric analysis of volatile art markets," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3091-3104.
    8. Julio Dávila, 2011. "Optimal population and education," Documents de travail du Centre d'Economie de la Sorbonne 11069, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    9. VAN VYVE, Mathieu, 2011. "Linear prices for non-convex electricity markets: models and algorithms," LIDAM Discussion Papers CORE 2011050, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    10. CHANDER, Parkash & TULKENS, Henry, 2011. "The kyoto Protocol, the Copenhagen Accord, the Cancun Agreements, and beyond: an economic and game theoretical exploration and interpretation," LIDAM Discussion Papers CORE 2011051, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    11. Rombouts, Jeroen & Stentoft, Lars & Violante, Franceso, 2014. "The value of multivariate model sophistication: An application to pricing Dow Jones Industrial Average options," International Journal of Forecasting, Elsevier, vol. 30(1), pages 78-98.
    12. GAHUNGU, Joachim & SMEERS, Yves, 2011. "A real options model for electricity capacity expansion," LIDAM Discussion Papers CORE 2011044, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    13. BAUWENS, Luc & DUFAYS, Arnaud & DE BACKER, Bruno, 2011. "Estimating and forecasting structural breaks in financial time series," LIDAM Discussion Papers CORE 2011055, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    14. BAUWENS, Luc & HAFNER, Christian & LAURENT, Sébastien, 2011. "Volatility models," LIDAM Discussion Papers CORE 2011058, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
      • Bauwens, L. & Hafner, C. & Laurent, S., 2012. "Volatility Models," LIDAM Reprints ISBA 2012028, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
      • Bauwens, L. & Hafner C. & Laurent, S., 2011. "Volatility Models," LIDAM Discussion Papers ISBA 2011044, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    15. GABSZEWICZ, Jean & TAROLA, Ornella, 2011. "Migration, wage differentials and fiscal competition," LIDAM Discussion Papers CORE 2011065, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    16. CALCIANO, Filippo L., 2011. "Oligopolistic competition with general complementarities," LIDAM Discussion Papers CORE 2011054, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    17. Per J. Agrell & Axel Gautier, 2012. "Rethinking Regulatory Capture," Chapters, in: Joseph E. Harrington Jr & Yannis Katsoulacos (ed.), Recent Advances in the Analysis of Competition Policy and Regulation, chapter 14, Edward Elgar Publishing.
    18. GABSZEWICZ, Jean J. & VAN YPERSELE, Tanguy & ZANAJ, Skerdilajda, 2011. "Does the seller of a house facing a large number of buyers always decrease its price when its first offer is rejected?," LIDAM Discussion Papers CORE 2011049, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    19. Florian MAYNERIS & Sandra PONCET, 2011. "Entry on difficult export markets by Chinese domestic firms: the role of foreign export spillovers," LIDAM Discussion Papers IRES 2011041, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    20. FLEURBAEY, Marc & SCHOKKAERT, Erik, 2011. "Equity in health and health care," LIDAM Discussion Papers CORE 2011026, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cor:louvco:2012002. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Alain GILLIS (email available below). General contact details of provider: https://edirc.repec.org/data/coreebe.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.