IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i11p2431-d1154891.html
   My bibliography  Save this article

A Modified Gradient Method for Distributionally Robust Logistic Regression over the Wasserstein Ball

Author

Listed:
  • Luyun Wang

    (College of Economics and Management, Southwest University, Chongqing 400715, China
    School of Economics, Chongqing Financial and Economic College, Chongqing 401320, China)

  • Bo Zhou

    (College of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing 400074, China)

Abstract

In this paper, a modified conjugate gradient method under the forward-backward splitting framework is proposed to further improve the numerical efficiency for solving the distributionally robust Logistic regression model over the Wasserstein ball, which comprises two phases: in the first phase, a conjugate gradient descent step is performed, and in the second phase, an instantaneous optimization problem is formulated and solved with a trade-off minimization of the regularization term, while simultaneously staying in close proximity to the interim point obtained in the first phase. The modified conjugate gradient method is proven to attain the optimal solution of the Wasserstein distributionally robust Logistic regression model with nonsummable steplength at a convergence rate of 1 / T . Finally, several numerical experiments to validate the effectiveness of theoretical analysis are conducted, which demonstrate that this method outperforms the off-the-shelf solver and the existing first-order algorithmic frameworks.

Suggested Citation

  • Luyun Wang & Bo Zhou, 2023. "A Modified Gradient Method for Distributionally Robust Logistic Regression over the Wasserstein Ball," Mathematics, MDPI, vol. 11(11), pages 1-15, May.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:11:p:2431-:d:1154891
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/11/2431/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/11/2431/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gonglin Yuan & Zehong Meng & Yong Li, 2016. "A Modified Hestenes and Stiefel Conjugate Gradient Algorithm for Large-Scale Nonsmooth Minimizations and Nonlinear Equations," Journal of Optimization Theory and Applications, Springer, vol. 168(1), pages 129-152, January.
    2. M. L. N. Gonçalves & L. F. Prudente, 2020. "On the extension of the Hager–Zhang conjugate gradient method for vector optimization," Computational Optimization and Applications, Springer, vol. 76(3), pages 889-916, July.
    3. Lorenzo Stella & Andreas Themelis & Panagiotis Patrinos, 2017. "Forward–backward quasi-Newton methods for nonsmooth optimization problems," Computational Optimization and Applications, Springer, vol. 67(3), pages 443-487, July.
    4. A. Chambolle & Ch. Dossal, 2015. "On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”," Journal of Optimization Theory and Applications, Springer, vol. 166(3), pages 968-982, September.
    5. Jose Blanchet & Karthyek Murthy, 2019. "Quantifying Distributional Model Risk via Optimal Transport," Mathematics of Operations Research, INFORMS, vol. 44(2), pages 565-600, May.
    6. Aharon Ben-Tal & Dick den Hertog & Anja De Waegenaere & Bertrand Melenberg & Gijs Rennen, 2013. "Robust Solutions of Optimization Problems Affected by Uncertain Probabilities," Management Science, INFORMS, vol. 59(2), pages 341-357, April.
    7. Tsegay Giday Woldu & Haibin Zhang & Xin Zhang & Yemane Hailu Fissuh, 2020. "A Modified Nonlinear Conjugate Gradient Algorithm for Large-Scale Nonsmooth Convex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 185(1), pages 223-238, April.
    8. Wolfram Wiesemann & Daniel Kuhn & Melvyn Sim, 2014. "Distributionally Robust Convex Optimization," Operations Research, INFORMS, vol. 62(6), pages 1358-1376, December.
    9. Luo, Fengqiao & Mehrotra, Sanjay, 2019. "Decomposition algorithm for distributionally robust optimization using Wasserstein metric with an application to a class of regression models," European Journal of Operational Research, Elsevier, vol. 278(1), pages 20-35.
    10. Dariush Khezrimotlagh & Yao Chen, 2018. "The Optimization Approach," International Series in Operations Research & Management Science, in: Decision Making and Performance Evaluation Using Data Envelopment Analysis, chapter 0, pages 107-134, Springer.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aliyu Muhammed Awwal & Mahmoud Muhammad Yahaya & Nuttapol Pakkaranang & Nattawut Pholasa, 2024. "A New Variant of the Conjugate Descent Method for Solving Unconstrained Optimization Problems and Applications," Mathematics, MDPI, vol. 12(15), pages 1-13, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Haolin Ruan & Zhi Chen & Chin Pang Ho, 2023. "Adjustable Distributionally Robust Optimization with Infinitely Constrained Ambiguity Sets," INFORMS Journal on Computing, INFORMS, vol. 35(5), pages 1002-1023, September.
    2. Aleksandrina Goeva & Henry Lam & Huajie Qian & Bo Zhang, 2019. "Optimization-Based Calibration of Simulation Input Models," Operations Research, INFORMS, vol. 67(5), pages 1362-1382, September.
    3. Shanshan Wang & Erick Delage, 2024. "A Column Generation Scheme for Distributionally Robust Multi-Item Newsvendor Problems," INFORMS Journal on Computing, INFORMS, vol. 36(3), pages 849-867, May.
    4. Feng Liu & Zhi Chen & Shuming Wang, 2023. "Globalized Distributionally Robust Counterpart," INFORMS Journal on Computing, INFORMS, vol. 35(5), pages 1120-1142, September.
    5. Zhi Chen & Weijun Xie, 2021. "Regret in the Newsvendor Model with Demand and Yield Randomness," Production and Operations Management, Production and Operations Management Society, vol. 30(11), pages 4176-4197, November.
    6. Henry Lam, 2019. "Recovering Best Statistical Guarantees via the Empirical Divergence-Based Distributionally Robust Optimization," Operations Research, INFORMS, vol. 67(4), pages 1090-1105, July.
    7. Shunichi Ohmori, 2021. "A Predictive Prescription Using Minimum Volume k -Nearest Neighbor Enclosing Ellipsoid and Robust Optimization," Mathematics, MDPI, vol. 9(2), pages 1-16, January.
    8. L. Jeff Hong & Zhiyuan Huang & Henry Lam, 2021. "Learning-Based Robust Optimization: Procedures and Statistical Guarantees," Management Science, INFORMS, vol. 67(6), pages 3447-3467, June.
    9. Dey, Shibshankar & Kim, Cheolmin & Mehrotra, Sanjay, 2024. "An algorithm for stochastic convex-concave fractional programs with applications to production efficiency and equitable resource allocation," European Journal of Operational Research, Elsevier, vol. 315(3), pages 980-990.
    10. Antonio J. Conejo & Nicholas G. Hall & Daniel Zhuoyu Long & Runhao Zhang, 2021. "Robust Capacity Planning for Project Management," INFORMS Journal on Computing, INFORMS, vol. 33(4), pages 1533-1550, October.
    11. Ran Ji & Miguel A. Lejeune, 2021. "Data-Driven Optimization of Reward-Risk Ratio Measures," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 1120-1137, July.
    12. Wang, Fan & Zhang, Chao & Zhang, Hui & Xu, Liang, 2021. "Short-term physician rescheduling model with feature-driven demand for mental disorders outpatients," Omega, Elsevier, vol. 105(C).
    13. Zhi Chen & Melvyn Sim & Peng Xiong, 2020. "Robust Stochastic Optimization Made Easy with RSOME," Management Science, INFORMS, vol. 66(8), pages 3329-3339, August.
    14. Jose Blanchet & Karthyek Murthy & Nian Si, 2022. "Confidence regions in Wasserstein distributionally robust estimation [Distributionally robust groupwise regularization estimator]," Biometrika, Biometrika Trust, vol. 109(2), pages 295-315.
    15. Tommi Ekholm & Erin Baker, 2022. "Multiple Beliefs, Dominance and Dynamic Consistency," Management Science, INFORMS, vol. 68(1), pages 529-540, January.
    16. Yu Wang & Yu Zhang & Minglong Zhou & Jiafu Tang, 2023. "Feature‐driven robust surgery scheduling," Production and Operations Management, Production and Operations Management Society, vol. 32(6), pages 1921-1938, June.
    17. Postek, Krzysztof & Ben-Tal, A. & den Hertog, Dick & Melenberg, Bertrand, 2015. "Exact Robust Counterparts of Ambiguous Stochastic Constraints Under Mean and Dispersion Information," Other publications TiSEM d718e419-a375-4707-b206-e, Tilburg University, School of Economics and Management.
    18. Ketkov, Sergey S., 2024. "A study of distributionally robust mixed-integer programming with Wasserstein metric: on the value of incomplete data," European Journal of Operational Research, Elsevier, vol. 313(2), pages 602-615.
    19. Arlen Dean & Amirhossein Meisami & Henry Lam & Mark P. Van Oyen & Christopher Stromblad & Nick Kastango, 2022. "Quantile regression forests for individualized surgery scheduling," Health Care Management Science, Springer, vol. 25(4), pages 682-709, December.
    20. Xiangyi Fan & Grani A. Hanasusanto, 2024. "A Decision Rule Approach for Two-Stage Data-Driven Distributionally Robust Optimization Problems with Random Recourse," INFORMS Journal on Computing, INFORMS, vol. 36(2), pages 526-542, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:11:p:2431-:d:1154891. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.