IDEAS home Printed from https://ideas.repec.org/a/spr/comgts/v11y2014i4p319-340.html
   My bibliography  Save this article

Numerical study of learning algorithms on Stiefel manifold

Author

Listed:
  • Takafumi Kanamori
  • Akiko Takeda

Abstract

Convex optimization methods are used for many machine learning models such as support vector machine. However, the requirement of a convex formulation can place limitations on machine learning models. In recent years, a number of machine learning methods not requiring convexity have emerged. In this paper, we study non-convex optimization problems on the Stiefel manifold in which the feasible set consists of a set of rectangular matrices with orthonormal column vectors. We present examples of non-convex optimization problems in machine learning and apply three nonlinear optimization methods for finding a local optimal solution; geometric gradient descent method, augmented Lagrangian method of multipliers, and alternating direction method of multipliers. Although the geometric gradient method is often used to solve non-convex optimization problems on the Stiefel manifold, we show that the alternating direction method of multipliers generally produces higher quality numerical solutions within a reasonable computation time. Copyright Springer-Verlag Berlin Heidelberg 2014

Suggested Citation

  • Takafumi Kanamori & Akiko Takeda, 2014. "Numerical study of learning algorithms on Stiefel manifold," Computational Management Science, Springer, vol. 11(4), pages 319-340, October.
  • Handle: RePEc:spr:comgts:v:11:y:2014:i:4:p:319-340
    DOI: 10.1007/s10287-013-0181-7
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10287-013-0181-7
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s10287-013-0181-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. David G. Luenberger & Yinyu Ye, 2008. "Linear and Nonlinear Programming," International Series in Operations Research and Management Science, Springer, edition 0, number 978-0-387-74503-9, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shahmohammadi, Ali & Sioshansi, Ramteen & Conejo, Antonio J. & Afsharnia, Saeed, 2018. "Market equilibria and interactions between strategic generation, wind, and storage," Applied Energy, Elsevier, vol. 220(C), pages 876-892.
    2. Alp Atakan & Mehmet Ekmekci & Ludovic Renou, 2021. "Cross-verification and Persuasive Cheap Talk," Papers 2102.13562, arXiv.org, revised Apr 2021.
    3. Arthur Medeiros & Thales Ramos & José Tavares de Oliveira & Manoel F. Medeiros Júnior, 2020. "Direct Voltage Control of a Doubly Fed Induction Generator by Means of Optimal Strategy," Energies, MDPI, vol. 13(3), pages 1-28, February.
    4. Ivorra, Benjamin & Mohammadi, Bijan & Manuel Ramos, Angel, 2015. "A multi-layer line search method to improve the initialization of optimization algorithms," European Journal of Operational Research, Elsevier, vol. 247(3), pages 711-720.
    5. Tanaka, Ken'ichiro & Toda, Alexis Akira, 2015. "Discretizing Distributions with Exact Moments: Error Estimate and Convergence Analysis," University of California at San Diego, Economics Working Paper Series qt7g23r5kh, Department of Economics, UC San Diego.
    6. Ashrafi, M. & Khanjani, M.J. & Fadaei-Kermani, E. & Barani, G.A., 2015. "Farm drainage channel network optimization by improved modified minimal spanning tree," Agricultural Water Management, Elsevier, vol. 161(C), pages 1-8.
    7. Sergey Badikov & Antoine Jacquier & Daphne Qing Liu & Patrick Roome, 2016. "No-arbitrage bounds for the forward smile given marginals," Papers 1603.06389, arXiv.org, revised Oct 2016.
    8. Szidarovszky, Ferenc & Luo, Yi, 2014. "Incorporating risk seeking attitude into defense strategy," Reliability Engineering and System Safety, Elsevier, vol. 123(C), pages 104-109.
    9. Giorgio, 2019. "On Second-Order Optimality Conditions in Smooth Nonlinear Programming Problems," DEM Working Papers Series 171, University of Pavia, Department of Economics and Management.
    10. Csaba I. Fábián, 2021. "Gaining traction: on the convergence of an inner approximation scheme for probability maximization," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 29(2), pages 491-519, June.
    11. Bouslah, B. & Gharbi, A. & Pellerin, R., 2016. "Integrated production, sampling quality control and maintenance of deteriorating production systems with AOQL constraint," Omega, Elsevier, vol. 61(C), pages 110-126.
    12. Rafał Wiśniowski & Krzysztof Skrzypaszek & Tomasz Małachowski, 2020. "Selection of a Suitable Rheological Model for Drilling Fluid Using Applied Numerical Methods," Energies, MDPI, vol. 13(12), pages 1-17, June.
    13. Martins Barros, Rafael & Guimarães Lage, Guilherme & de Andrade Lira Rabêlo, Ricardo, 2022. "Sequencing paths of optimal control adjustments determined by the optimal reactive dispatch via Lagrange multiplier sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 301(1), pages 373-385.
    14. Yuichi Takano & Renata Sotirov, 2012. "A polynomial optimization approach to constant rebalanced portfolio selection," Computational Optimization and Applications, Springer, vol. 52(3), pages 645-666, July.
    15. Enrique I. Acuña & Ian S. Lowndes, 2014. "A Review of Primary Mine Ventilation System Optimization," Interfaces, INFORMS, vol. 44(2), pages 163-175, April.
    16. Nadia Demarteau & Thomas Breuer & Baudouin Standaert, 2012. "Selecting a Mix of Prevention Strategies against Cervical Cancer for Maximum Efficiency with an Optimization Program," PharmacoEconomics, Springer, vol. 30(4), pages 337-353, April.
    17. repec:cte:wsrepe:38369 is not listed on IDEAS
    18. Charles R. Weisbin & Joseph Mrozinski & William Lincoln & Alberto Elfes & Kacie Shelton & Hook Hua & Jeffrey H. Smith & Virgil Adumitroaie & Robert Silberg, 2010. "Lunar architecture and technology analysis driven by lunar science scenarios," Systems Engineering, John Wiley & Sons, vol. 13(3), pages 217-231, September.
    19. Andres Quiros-Granados & JAvier Trejos-Zelaya, 2019. "Estimation of the yield curve for Costa Rica using combinatorial optimization metaheuristics applied to nonlinear regression," Papers 2001.00920, arXiv.org.
    20. Vittorio Nicolardi, 2013. "Simultaneously Balancing Supply--Use Tables At Current And Constant Prices: A New Procedure," Economic Systems Research, Taylor & Francis Journals, vol. 25(4), pages 409-434, December.
    21. Alina R. Battalova* & Nadezhda A. Opokina, 2018. "Economic-Mathematical Model of the Structure of Nutrition of Population in the Region," The Journal of Social Sciences Research, Academic Research Publishing Group, pages 275-280:1.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:comgts:v:11:y:2014:i:4:p:319-340. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.