IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v124y2018icp235-251.html
   My bibliography  Save this article

A Forward and Backward Stagewise algorithm for nonconvex loss functions with adaptive Lasso

Author

Listed:
  • Shi, Xingjie
  • Huang, Yuan
  • Huang, Jian
  • Ma, Shuangge

Abstract

Penalization is a popular tool for multi- and high-dimensional data. Most of the existing computational algorithms have been developed for convex loss functions. Nonconvex loss functions can sometimes generate more robust results and have important applications. Motivated by the BLasso algorithm, this study develops the Forward and Backward Stagewise (Fabs) algorithm for nonconvex loss functions with the adaptive Lasso (aLasso) penalty. It is shown that each point along the Fabs paths is a δ-approximate solution to the aLasso problem and the Fabs paths converge to the stationary points of the aLasso problem when δ goes to zero, given that the loss function has second-order derivatives bounded from above. This study exemplifies the Fabs with an application to the penalized smooth partial rank (SPR) estimation, for which there is still a lack of effective algorithm. Extensive numerical studies are conducted to demonstrate the benefit of penalized SPR estimation using Fabs, especially under high-dimensional settings. Application to the smoothed 0-1 loss in binary classification is introduced to demonstrate its capability to work with other differentiable nonconvex loss function.

Suggested Citation

  • Shi, Xingjie & Huang, Yuan & Huang, Jian & Ma, Shuangge, 2018. "A Forward and Backward Stagewise algorithm for nonconvex loss functions with adaptive Lasso," Computational Statistics & Data Analysis, Elsevier, vol. 124(C), pages 235-251.
  • Handle: RePEc:eee:csdana:v:124:y:2018:i:c:p:235-251
    DOI: 10.1016/j.csda.2018.03.006
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947318300549
    Download Restriction: Full text for ScienceDirect subscribers only.

    File URL: https://libkey.io/10.1016/j.csda.2018.03.006?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lin, Huazhen & Peng, Heng, 2013. "Smoothed rank correlation of the linear transformation regression model," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 615-630.
    2. Xiao Song & Shuangge Ma, 2010. "Penalised variable selection with U-estimates," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 22(4), pages 499-515.
    3. Han, Aaron K., 1987. "Non-parametric analysis of a generalized regression model : The maximum rank correlation estimator," Journal of Econometrics, Elsevier, vol. 35(2-3), pages 303-316, July.
    4. Scrucca, Luca, 2013. "GA: A Package for Genetic Algorithms in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 53(i04).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liang, Weijuan & Ma, Shuangge & Lin, Cunjie, 2021. "Marginal false discovery rate for a penalized transformation survival model," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    2. Jacquemain, Alexandre & Heuchenne, Cédric & Pircalabelu, Eugen, 2024. "A penalised bootstrap estimation procedure for the explained Gini coefficient," LIDAM Discussion Papers ISBA 2024005, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yiwei Fan & Jiaqi Gu & Guosheng Yin, 2023. "Sparse concordance‐based ordinal classification," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(3), pages 934-961, September.
    2. Tan, Xin Lu, 2019. "Optimal estimation of slope vector in high-dimensional linear transformation models," Journal of Multivariate Analysis, Elsevier, vol. 169(C), pages 179-204.
    3. Lin, Xiefang & Fang, Fang, 2024. "Variable selection of Kolmogorov-Smirnov maximization with a penalized surrogate loss," Computational Statistics & Data Analysis, Elsevier, vol. 195(C).
    4. Liang, Weijuan & Ma, Shuangge & Lin, Cunjie, 2021. "Marginal false discovery rate for a penalized transformation survival model," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    5. Jacquemain, Alexandre & Heuchenne, Cédric & Pircalabelu, Eugen, 2024. "A penalised bootstrap estimation procedure for the explained Gini coefficient," LIDAM Discussion Papers ISBA 2024005, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    6. Yu, Tao & Li, Pengfei & Chen, Baojiang & Yuan, Ao & Qin, Jing, 2023. "Maximum pairwise-rank-likelihood-based inference for the semiparametric transformation model," Journal of Econometrics, Elsevier, vol. 235(2), pages 454-469.
    7. Patrick Bajari & Jeremy Fox & Stephen Ryan, 2008. "Evaluating wireless carrier consolidation using semiparametric demand estimation," Quantitative Marketing and Economics (QME), Springer, vol. 6(4), pages 299-338, December.
    8. Bergeaud, Antonin & Raimbault, Juste, 2020. "An empirical analysis of the spatial variability of fuel prices in the United States," Transportation Research Part A: Policy and Practice, Elsevier, vol. 132(C), pages 131-143.
    9. Khan, Shakeeb & Tamer, Elie, 2007. "Partial rank estimation of duration models with general forms of censoring," Journal of Econometrics, Elsevier, vol. 136(1), pages 251-280, January.
    10. Jochmans, Koen, 2015. "Multiplicative-error models with sample selection," Journal of Econometrics, Elsevier, vol. 184(2), pages 315-327.
    11. Nandan, Rohit & Bandaru, Varaprasad & Meduri, Pridhvi & Jones, Curtis & Lollato, Romulo, 2024. "Evaluating the utility of weather generators in crop simulation models for in-season yield forecasting," Agricultural Systems, Elsevier, vol. 220(C).
    12. Delgado, Miguel A. & Rodriguez-Poo, Juan M. & Wolf, Michael, 2001. "Subsampling inference in cube root asymptotics with an application to Manski's maximum score estimator," Economics Letters, Elsevier, vol. 73(2), pages 241-250, November.
    13. Finn Olesen, 1999. "Monetær integration i EU," Working Papers 2/99, University of Southern Denmark, Department of Sociology, Environmental and Business Economics.
    14. Ming-Yueh Huang & Chin-Tsang Chiang, 2017. "Estimation and Inference Procedures for Semiparametric Distribution Models with Varying Linear-Index," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 44(2), pages 396-424, June.
    15. repec:hal:wpspec:info:hdl:2441/3vl5fe4i569nbr005tctlc8ll5 is not listed on IDEAS
    16. Ronning Gerd & Rosemann Martin & Strotmann Harald, 2005. "Post-Randomization Under Test: Estimation of the Probit Model," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 225(5), pages 544-566, October.
    17. Shakeeb Khan & Arnaud Maurel & Yichong Zhang, 2023. "Informational Content of Factor Structures in Simultaneous Binary Response Models," Advances in Econometrics, in: Essays in Honor of Joon Y. Park: Econometric Methodology in Empirical Applications, volume 45, pages 385-410, Emerald Group Publishing Limited.
    18. Alfio Marazzi, 2021. "Improving the Efficiency of Robust Estimators for the Generalized Linear Model," Stats, MDPI, vol. 4(1), pages 1-20, February.
    19. Castellares, Fredy & Patrício, Silvio C. & Lemonte, Artur J., 2020. "On gamma-Gompertz life expectancy," Statistics & Probability Letters, Elsevier, vol. 165(C).
    20. Mittelhammer, Ron C. & Judge, George, 2011. "A family of empirical likelihood functions and estimators for the binary response model," Journal of Econometrics, Elsevier, vol. 164(2), pages 207-217, October.
    21. repec:spo:wpmain:info:hdl:2441/dambferfb7dfprc9m01h6f4h2 is not listed on IDEAS
    22. Koen Jochmans, 2011. "Identification in Bivariate binary-choice Models with elliptical innovations," Working Papers hal-01069483, HAL.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:124:y:2018:i:c:p:235-251. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.