IDEAS home Printed from https://ideas.repec.org/a/spr/coopap/v86y2023i2d10.1007_s10589-023-00496-x.html
   My bibliography  Save this article

Zero-norm regularized problems: equivalent surrogates, proximal MM method and statistical error bound

Author

Listed:
  • Dongdong Zhang

    (South China University of Technology)

  • Shaohua Pan

    (South China University of Technology)

  • Shujun Bi

    (South China University of Technology)

  • Defeng Sun

    (The Hong Kong Polytechnic University)

Abstract

For the zero-norm regularized problem, we verify that the penalty problem of its equivalent MPEC reformulation is a global exact penalty, which implies a family of equivalent surrogates. For a subfamily of these surrogates, the critical point set is demonstrated to coincide with the d-directional stationary point set and when a critical point has no too small nonzero component, it is a strongly local optimal solution of the surrogate problem and the zero-norm regularized problem. We also develop a proximal majorization-minimization (MM) method for solving the DC (difference of convex functions) surrogates, and provide its global and linear convergence analysis. For the limit of the generated sequence, the statistical error bound is established under a mild condition, which implies its good quality from a statistical respective. Numerical comparisons with ADMM for solving the DC surrogate and APG for solving its partially smoothed form indicate that our proximal MM method armed with an inexact dual PPA plus the semismooth Newton method (PMMSN for short) is remarkably superior to ADMM and APG in terms of the quality of solutions and the CPU time.

Suggested Citation

  • Dongdong Zhang & Shaohua Pan & Shujun Bi & Defeng Sun, 2023. "Zero-norm regularized problems: equivalent surrogates, proximal MM method and statistical error bound," Computational Optimization and Applications, Springer, vol. 86(2), pages 627-667, November.
    • Handle: RePEc:spr:coopap:v:86:y:2023:i:2:d:10.1007_s10589-023-00496-x
    DOI: 10.1007/s10589-023-00496-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10589-023-00496-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10589-023-00496-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hédy Attouch & Jérôme Bolte & Patrick Redont & Antoine Soubeyran, 2010. "Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality," Mathematics of Operations Research, INFORMS, vol. 35(2), pages 438-457, May.
    2. R. T. Rockafellar, 1976. "Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming," Mathematics of Operations Research, INFORMS, vol. 1(2), pages 97-116, May.
    3. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    4. F. Rinaldi & F. Schoen & M. Sciandrone, 2010. "Concave programming for minimizing the zero-norm over polyhedral sets," Computational Optimization and Applications, Springer, vol. 46(3), pages 467-486, July.
    5. Fan Wu & Wei Bian, 2020. "Accelerated iterative hard thresholding algorithm for $$l_0$$l0 regularized regression problem," Journal of Global Optimization, Springer, vol. 76(4), pages 819-840, April.
    6. Yulan Liu & Shujun Bi & Shaohua Pan, 2018. "Equivalent Lipschitz surrogates for zero-norm and rank optimization problems," Journal of Global Optimization, Springer, vol. 72(4), pages 679-704, December.
    7. Jong-Shi Pang & Meisam Razaviyayn & Alberth Alvarado, 2017. "Computing B-Stationary Points of Nonsmooth DC Programs," Mathematics of Operations Research, INFORMS, vol. 42(1), pages 95-118, January.
    8. Lan Wang & Yichao Wu & Runze Li, 2012. "Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(497), pages 214-222, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yitian Qian & Shaohua Pan & Yulan Liu, 2023. "Calmness of partial perturbation to composite rank constraint systems and its applications," Journal of Global Optimization, Springer, vol. 85(4), pages 867-889, April.
    2. Guo, Xu & Li, Runze & Liu, Jingyuan & Zeng, Mudong, 2023. "Statistical inference for linear mediation models with high-dimensional mediators and application to studying stock reaction to COVID-19 pandemic," Journal of Econometrics, Elsevier, vol. 235(1), pages 166-179.
    3. Bo Jiang & Tianyi Lin & Shiqian Ma & Shuzhong Zhang, 2019. "Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis," Computational Optimization and Applications, Springer, vol. 72(1), pages 115-157, January.
    4. Sokbae Lee & Myung Hwan Seo & Youngki Shin, 2016. "The lasso for high dimensional regression with a possible change point," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 193-210, January.
    5. Zhao, Jun & Chen, Yingyu & Zhang, Yi, 2018. "Expectile regression for analyzing heteroscedasticity in high dimension," Statistics & Probability Letters, Elsevier, vol. 137(C), pages 304-311.
    6. Wenxing Zhu & Huating Huang & Lanfan Jiang & Jianli Chen, 0. "Weighted thresholding homotopy method for sparsity constrained optimization," Journal of Combinatorial Optimization, Springer, vol. 0, pages 1-29.
    7. W. Ackooij & S. Demassey & P. Javal & H. Morais & W. Oliveira & B. Swaminathan, 2021. "A bundle method for nonsmooth DC programming with application to chance-constrained problems," Computational Optimization and Applications, Springer, vol. 78(2), pages 451-490, March.
    8. Zhang, Shucong & Zhou, Yong, 2018. "Variable screening for ultrahigh dimensional heterogeneous data via conditional quantile correlations," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 1-13.
    9. Manlio Gaudioso & Giovanni Giallombardo & Giovanna Miglionico, 2023. "Sparse optimization via vector k-norm and DC programming with an application to feature selection for support vector machines," Computational Optimization and Applications, Springer, vol. 86(2), pages 745-766, November.
    10. Xiang Zhang & Yichao Wu & Lan Wang & Runze Li, 2016. "Variable selection for support vector machines in moderately high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 53-76, January.
    11. Zhaoping Hong & Yuao Hu & Heng Lian, 2013. "Variable selection for high-dimensional varying coefficient partially linear models via nonconcave penalty," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 76(7), pages 887-908, October.
    12. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    13. Liu, Jingjing & Ma, Ruijie & Zeng, Xiaoyang & Liu, Wanquan & Wang, Mingyu & Chen, Hui, 2021. "An efficient non-convex total variation approach for image deblurring and denoising," Applied Mathematics and Computation, Elsevier, vol. 397(C).
    14. Yao, Fang & Sue-Chee, Shivon & Wang, Fan, 2017. "Regularized partially functional quantile regression," Journal of Multivariate Analysis, Elsevier, vol. 156(C), pages 39-56.
    15. Min Li & Zhongming Wu, 2019. "Convergence Analysis of the Generalized Splitting Methods for a Class of Nonconvex Optimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 183(2), pages 535-565, November.
    16. Bo Wen & Xiaojun Chen & Ting Kei Pong, 2018. "A proximal difference-of-convex algorithm with extrapolation," Computational Optimization and Applications, Springer, vol. 69(2), pages 297-324, March.
    17. Kean Ming Tan & Lan Wang & Wen‐Xin Zhou, 2022. "High‐dimensional quantile regression: Convolution smoothing and concave regularization," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(1), pages 205-233, February.
    18. Wang, Shangshan & Xiang, Liming, 2017. "Two-layer EM algorithm for ALD mixture regression models: A new solution to composite quantile regression," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 136-154.
    19. Zhongming Wu & Min Li, 2019. "General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems," Computational Optimization and Applications, Springer, vol. 73(1), pages 129-158, May.
    20. Mohamed Ouhourane & Yi Yang & Andréa L. Benedet & Karim Oualkacha, 2022. "Group penalized quantile regression," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 31(3), pages 495-529, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:coopap:v:86:y:2023:i:2:d:10.1007_s10589-023-00496-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.