IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v319y2024i2p494-504.html
   My bibliography  Save this article

Large-scale robust regression with truncated loss via majorization-minimization algorithm

Author

Listed:
  • Huang, Ling-Wei
  • Shao, Yuan-Hai
  • Lv, Xiao-Jing
  • Li, Chun-Na

Abstract

The utilization of regression methods employing truncated loss functions is widely praised for its robustness in handling outliers and representing the solution in the sparse form of the samples. However, due to the non-convexity of the truncated loss, the commonly used algorithms such as difference of convex algorithm (DCA) fail to maintain sparsity when dealing with non-convex loss functions, and adapting DCA for efficient optimization also incurs additional development costs. To address these challenges, we propose a novel approach called truncated loss regression via majorization-minimization algorithm (TLRM). TLRM employs a surrogate function to approximate the original truncated loss regression and offers several desirable properties: (i) Eliminating outliers before the training process and encapsulating general convex loss regression within its structure as iterative subproblems, (ii) Solving the convex loss problem iteratively thereby facilitating the use of a well-established toolbox for convex optimization. (iii) Converging to a truncated loss regression and providing a solution with sample sparsity. Extensive experiments demonstrate that TLRM achieves superior sparsity without sacrificing robustness, and it can be several tens of thousands of times faster than traditional DCA on large-scale problems. Moreover, TLRM is also applicable to datasets with millions of samples, making it a practical choice for real-world scenarios. The codebase for methods with truncated loss functions is accessible at https://i-do-lab.github.io/optimal-group.org/Resources/Code/TLRM.html.

Suggested Citation

  • Huang, Ling-Wei & Shao, Yuan-Hai & Lv, Xiao-Jing & Li, Chun-Na, 2024. "Large-scale robust regression with truncated loss via majorization-minimization algorithm," European Journal of Operational Research, Elsevier, vol. 319(2), pages 494-504.
  • Handle: RePEc:eee:ejores:v:319:y:2024:i:2:p:494-504
    DOI: 10.1016/j.ejor.2024.04.028
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221724003278
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2024.04.028?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Liao, Zhiqiang & Dai, Sheng & Kuosmanen, Timo, 2024. "Convex support vector regression," European Journal of Operational Research, Elsevier, vol. 313(3), pages 858-870.
    2. Mike G. Tsionas, 2023. "Linex and double-linex regression for parameter estimation and forecasting," Annals of Operations Research, Springer, vol. 323(1), pages 229-245, April.
    3. Bottmer, Lea & Croux, Christophe & Wilms, Ines, 2022. "Sparse regression for large data sets with outliers," European Journal of Operational Research, Elsevier, vol. 297(2), pages 782-794.
    4. Liang, Xijun & Zhang, Zhipeng & Song, Yunquan & Jian, Ling, 2022. "Kernel-based online regression with canal loss," European Journal of Operational Research, Elsevier, vol. 297(1), pages 268-279.
    5. Roger Koenker & Kevin F. Hallock, 2001. "Quantile Regression," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 143-156, Fall.
    6. Fu, Saiji & Tian, Yingjie & Tang, Long, 2023. "Robust regression under the general framework of bounded loss functions," European Journal of Operational Research, Elsevier, vol. 310(3), pages 1325-1339.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fu, Saiji & Tian, Yingjie & Tang, Long, 2023. "Robust regression under the general framework of bounded loss functions," European Journal of Operational Research, Elsevier, vol. 310(3), pages 1325-1339.
    2. Akosah, Nana Kwame & Alagidede, Imhotep Paul & Schaling, Eric, 2020. "Testing for asymmetry in monetary policy rule for small-open developing economies: Multiscale Bayesian quantile evidence from Ghana," The Journal of Economic Asymmetries, Elsevier, vol. 22(C).
    3. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    4. Héctor Manuel Zárate S., 2005. "Cambios en la estructura salarial: una historia desde la regresión cuanfílica," Monetaria, CEMLA, vol. 0(4), pages 339-364, octubre-d.
    5. Efobi, Uchenna & Asongu, Simplice & Okafor, Chinelo & Tchamyou, Vanessa & Tanankem, Belmondo, 2016. "Diaspora Remittance Inflow, Financial Development and the Industrialisation of Africa," MPRA Paper 76121, University Library of Munich, Germany.
    6. Leon Zolotoy & Don O’Sullivan & Keke Song, 2021. "The Role of Ethical Standards in the Relationship Between Religious Social Norms and M&A Announcement Returns," Journal of Business Ethics, Springer, vol. 170(4), pages 721-742, May.
    7. Trojanek, Radoslaw & Huderek-Glapska, Sonia, 2018. "Measuring the noise cost of aviation – The association between the Limited Use Area around Warsaw Chopin Airport and property values," Journal of Air Transport Management, Elsevier, vol. 67(C), pages 103-114.
    8. Paulo M.M. Rodrigues & Rita Fradique Lourenço, 2015. "House prices: bubbles, exuberance or something else? Evidence from euro area countries," Working Papers w201517, Banco de Portugal, Economics and Research Department.
    9. repec:rre:publsh:v:39:y:2009:i:2:p:149-69 is not listed on IDEAS
    10. Muller, Christophe, 2018. "Heterogeneity and nonconstant effect in two-stage quantile regression," Econometrics and Statistics, Elsevier, vol. 8(C), pages 3-12.
    11. Juan Mora & Antonia Febrer, 2005. "Wage Distribution In Spain, 1994-1999: An Application Of A Flexible Estimator Of Conditional Distributions," Working Papers. Serie EC 2005-04, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    12. Xiaoying Liu & Jere R. Behrman & Emily Hannum & Fan Wang & Qingguo Zhao, 2022. "Same environment, stratified impacts? Air pollution, extreme temperatures, and birth weight in south China," Papers 2204.00219, arXiv.org.
    13. Asongu, Simplice A., 2017. "Assessing marginal, threshold, and net effects of financial globalisation on financial development in Africa," Journal of Multinational Financial Management, Elsevier, vol. 40(C), pages 103-114.
    14. Rajeev K. Goel, 2023. "Seek foreign funds or technology? Relative impacts of different spillover modes on innovation," The Journal of Technology Transfer, Springer, vol. 48(4), pages 1466-1488, August.
    15. Asongu Simplice, 2014. "Globalization and health worker crisis: what do wealth-effects tell us?," International Journal of Social Economics, Emerald Group Publishing Limited, vol. 41(12), pages 1243-1264, November.
    16. Simplice A. Asongu & Nicholas M. Odhiambo, 2023. "Female unemployment, mobile money innovations and doing business by females," Journal of Innovation and Entrepreneurship, Springer, vol. 12(1), pages 1-26, December.
    17. Yayan Hernuryadin & Koji Kotani & Tatsuyoshi Saijo, 2020. "Time Preferences of Food Producers: Does “Cultivate and Grow” Matter?," Land Economics, University of Wisconsin Press, vol. 96(1), pages 132-148.
    18. Klomp, Jeroen, 2013. "Government interventions and default risk: Does one size fit all?," Journal of Financial Stability, Elsevier, vol. 9(4), pages 641-653.
    19. Simplice A. Asongu & Valentine B. Soumtang & Ofeh M. Edoh, 2021. "Financial determinants of informal financial development in Sub-Saharan Africa," Research Africa Network Working Papers 21/077, Research Africa Network (RAN).
    20. Bampinas, Georgios & Panagiotidis, Theodore, 2016. "Hedging inflation with individual US stocks: A long-run portfolio analysis," The North American Journal of Economics and Finance, Elsevier, vol. 37(C), pages 374-392.
    21. Richard Kwabena Nkrumah & Samuel Kobina Annim & Benedict Afful, 2021. "Household Social Expenditure in Ghana: Examining the Ex-Post Effects and Vulnerability to Poverty," Social Sciences, MDPI, vol. 10(2), pages 1-15, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:319:y:2024:i:2:p:494-504. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.