IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v270y2018i3p931-942.html
   My bibliography  Save this article

Characterization of the equivalence of robustification and regularization in linear and matrix regression

Author

Listed:
  • Bertsimas, Dimitris
  • Copenhaver, Martin S.

Abstract

The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods which appear as a loss function plus a penalty. In this paper we deepen and extend the understanding of the connection between robustification and regularization (as achieved by penalization) in regression problems. Specifically, (a)In the context of linear regression, we characterize precisely under which conditions on the model of uncertainty used and on the loss function penalties robustification and regularization are equivalent.(b)We extend the characterization of robustification and regularization to matrix regression problems (matrix completion and Principal Component Analysis).

Suggested Citation

  • Bertsimas, Dimitris & Copenhaver, Martin S., 2018. "Characterization of the equivalence of robustification and regularization in linear and matrix regression," European Journal of Operational Research, Elsevier, vol. 270(3), pages 931-942.
    • Handle: RePEc:eee:ejores:v:270:y:2018:i:3:p:931-942
    DOI: 10.1016/j.ejor.2017.03.051
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221717302734
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2017.03.051?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bottmer, Lea & Croux, Christophe & Wilms, Ines, 2022. "Sparse regression for large data sets with outliers," European Journal of Operational Research, Elsevier, vol. 297(2), pages 782-794.
    2. Jun-ya Gotoh & Michael Jong Kim & Andrew E. B. Lim, 2020. "Worst-case sensitivity," Papers 2010.10794, arXiv.org.
    3. Mengshi Lu & Zuo‐Jun Max Shen, 2021. "A Review of Robust Operations Management under Model Uncertainty," Production and Operations Management, Production and Operations Management Society, vol. 30(6), pages 1927-1943, June.
    4. Jan Pablo Burgard & Joscha Krause & Ralf Münnich, 2019. "Penalized Small Area Models for the Combination of Unit- and Area-level Data," Research Papers in Economics 2019-05, University of Trier, Department of Economics.
    5. Jan Pablo Burgard & Joscha Krause & Dennis Kreber & Domingo Morales, 2021. "The generalized equivalence of regularization and min–max robustification in linear mixed models," Statistical Papers, Springer, vol. 62(6), pages 2857-2883, December.
    6. Carina Moreira Costa & Dennis Kreber & Martin Schmidt, 2022. "An Alternating Method for Cardinality-Constrained Optimization: A Computational Study for the Best Subset Selection and Sparse Portfolio Problems," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 2968-2988, November.
    7. Jack Dunn & Ying Daisy Zhuo, 2022. "Detecting Racial Bias in Jury Selection," SN Operations Research Forum, Springer, vol. 3(3), pages 1-17, September.
    8. Bertsimas, Dimitris & Koukouvinos, Thodoris, 2024. "Robust linear algebra," European Journal of Operational Research, Elsevier, vol. 314(3), pages 1174-1184.
    9. Joscha Krause & Jan Pablo Burgard & Domingo Morales, 2022. "Robust prediction of domain compositions from uncertain data using isometric logratio transformations in a penalized multivariate Fay–Herriot model," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 76(1), pages 65-96, February.
    10. Jan Pablo Burgard & Joscha Krause & Dennis Kreber, 2019. "Regularized Area-level Modelling for Robust Small Area Estimation in the Presence of Unknown Covariate Measurement Errors," Research Papers in Economics 2019-04, University of Trier, Department of Economics.
    11. Gambella, Claudio & Ghaddar, Bissan & Naoum-Sawaya, Joe, 2021. "Optimization problems for machine learning: A survey," European Journal of Operational Research, Elsevier, vol. 290(3), pages 807-828.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:270:y:2018:i:3:p:931-942. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.