IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v52y2008i7p3331-3341.html
   My bibliography  Save this article

Robustified L2 boosting

Author

Listed:
  • Lutz, Roman Werner
  • Kalisch, Markus
  • Buhlmann, Peter

Abstract

Five robustifications of L2 boosting for linear regression with various robustness properties are considered. The first two use the Huber loss as implementing loss function for boosting and the second two use robust simple linear regression for the fitting in L2 boosting (i.e. robust base learners). Both concepts can be applied with or without down-weighting of leverage points. Our last method uses robust correlation estimates and appears to be most robust. Crucial advantages of all methods are that they do not compute covariance matrices of all covariates and that they do not have to identify multivariate leverage points. When there are no outliers, the robust methods are only slightly worse than L2 boosting. In the contaminated case though, the robust methods outperform L2 boosting by a large margin. Some of the robustifications are also computationally highly efficient and therefore well suited for truly high-dimensional problems.

Suggested Citation

  • Lutz, Roman Werner & Kalisch, Markus & Buhlmann, Peter, 2008. "Robustified L2 boosting," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3331-3341, March.
  • Handle: RePEc:eee:csdana:v:52:y:2008:i:7:p:3331-3341
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(07)00441-0
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. McCann, Lauren & Welsch, Roy E., 2007. "Robust variable selection using least angle regression and elemental set sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 249-257, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ju, Xiaomeng & Salibián-Barrera, Matías, 2021. "Robust boosting for regression problems," Computational Statistics & Data Analysis, Elsevier, vol. 153(C).
    2. Riani, Marco & Atkinson, Anthony C., 2010. "Robust model selection with flexible trimming," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3300-3312, December.
    3. Martinez, Waldyn & Gray, J. Brian, 2016. "Noise peeling methods to improve boosting algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 483-497.
    4. Zhang, Chun-Xia & Zhang, Jiang-She & Zhang, Gai-Ying, 2009. "Using Boosting to prune Double-Bagging ensembles," Computational Statistics & Data Analysis, Elsevier, vol. 53(4), pages 1218-1231, February.
    5. Khan, Jafar A. & Van Aelst, Stefan & Zamar, Ruben H., 2010. "Fast robust estimation of prediction error based on resampling," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3121-3130, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Menjoge, Rajiv S. & Welsch, Roy E., 2010. "A diagnostic method for simultaneous feature selection and outlier identification in linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3181-3193, December.
    2. Zhang, Yuexia & Qin, Guoyou & Zhu, Zhongyi & Xu, Wanghong, 2019. "A novel robust approach for analysis of longitudinal data," Computational Statistics & Data Analysis, Elsevier, vol. 138(C), pages 83-95.
    3. Rand R. Wilcox, 2018. "Robust regression: an inferential method for determining which independent variables are most important," Journal of Applied Statistics, Taylor & Francis Journals, vol. 45(1), pages 100-111, January.
    4. Khan, Jafar A. & Van Aelst, Stefan & Zamar, Ruben H., 2010. "Fast robust estimation of prediction error based on resampling," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3121-3130, December.
    5. Riani, Marco & Atkinson, Anthony C., 2010. "Robust model selection with flexible trimming," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3300-3312, December.
    6. Elvezio Ronchetti, 2021. "The main contributions of robust statistics to statistical science and a new challenge," METRON, Springer;Sapienza Università di Roma, vol. 79(2), pages 127-135, August.
    7. Alfons, Andreas & Croux, Christophe & Gelper, Sarah, 2016. "Robust groupwise least angle regression," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 421-435.
    8. Thompson, Ryan, 2022. "Robust subset selection," Computational Statistics & Data Analysis, Elsevier, vol. 169(C).
    9. Andreas Alfons & Wolfgang Baaske & Peter Filzmoser & Wolfgang Mader & Roland Wieser, 2011. "Robust variable selection with application to quality of life research," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 20(1), pages 65-82, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:52:y:2008:i:7:p:3331-3341. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.