IDEAS home Printed from https://ideas.repec.org/p/zbw/sfb475/200603.html
   My bibliography  Save this paper

Robust Learning from Bites for Data Mining

Author

Listed:
  • Christmann, Andreas
  • Steinwart, Ingo
  • Hubert, Mia

Abstract

Some methods from statistical machine learning and from robust statistics have two drawbacks. Firstly, they are computer-intensive such that they can hardly be used for massive data sets, say with millions of data points. Secondly, robust and non-parametric confidence intervals for the predictions according to the fitted models are often unknown. Here, we propose a simple but general method to overcome these problems in the context of huge data sets. The method is scalable to the memory of the computer, can be distributed on several processors if available, and can help to reduce the computation time substantially. Our main focus is on robust general support vector machines (SVM) based on minimizing regularized risks. The method offers distribution-free confidence intervals for the median of the predictions. The approach can also be helpful to fit robust estimators in parametric models for huge data sets.

Suggested Citation

  • Christmann, Andreas & Steinwart, Ingo & Hubert, Mia, 2006. "Robust Learning from Bites for Data Mining," Technical Reports 2006,03, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    • Handle: RePEc:zbw:sfb475:200603
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/22651/1/tr03-06.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christmann, Andreas & Steinwart, Ingo, 2005. "Consistency and robustness of kernel based regression," Technical Reports 2005,01, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    2. Christophe Croux & Stefan Aelst & Catherine Dehon, 2003. "Bounded influence regression using high breakdown scatter matrices," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 55(2), pages 265-285, June.
    3. Struyf, Anja J. & Rousseeuw, Peter J., 1999. "Halfspace Depth and Regression Depth Characterize the Empirical Distribution," Journal of Multivariate Analysis, Elsevier, vol. 69(1), pages 135-153, April.
    4. David J. Hand & Heikki Mannila & Padhraic Smyth, 2001. "Principles of Data Mining," MIT Press Books, The MIT Press, edition 1, volume 1, number 026208290x, December.
    5. Bartlett, Peter L. & Jordan, Michael I. & McAuliffe, Jon D., 2006. "Convexity, Classification, and Risk Bounds," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 138-156, March.
    6. Van Aelst, Stefan & Rousseeuw, Peter J. & Hubert, Mia & Struyf, Anja, 2002. "The Deepest Regression Method," Journal of Multivariate Analysis, Elsevier, vol. 81(1), pages 138-166, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christmann, Andreas & Steinwart, Ingo & Hubert, Mia, 2007. "Robust learning from bites for data mining," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 347-361, September.
    2. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2015. "Multivariate functional outlier detection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 177-202, July.
    3. Steffen Borgwardt & Rafael M. Frongillo, 2019. "Power Diagram Detection with Applications to Information Elicitation," Journal of Optimization Theory and Applications, Springer, vol. 181(1), pages 184-196, April.
    4. Stephan Morgenthaler, 2007. "A survey of robust statistics," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 15(3), pages 271-293, February.
    5. Stanislav Nagy, 2021. "Halfspace depth does not characterize probability distributions," Statistical Papers, Springer, vol. 62(3), pages 1135-1139, June.
    6. Weiyang Ding & Michael K. Ng & Wenxing Zhang, 2024. "A generalized alternating direction implicit method for consensus optimization: application to distributed sparse logistic regression," Journal of Global Optimization, Springer, vol. 90(3), pages 727-753, November.
    7. Fernandez del Pozo, J. A. & Bielza, C. & Gomez, M., 2005. "A list-based compact representation for large decision tables management," European Journal of Operational Research, Elsevier, vol. 160(3), pages 638-662, February.
    8. Le, Hong Hanh & Viviani, Jean-Laurent, 2018. "Predicting bank failure: An improvement by implementing a machine-learning approach to classical financial ratios," Research in International Business and Finance, Elsevier, vol. 44(C), pages 16-25.
    9. Li, Hui & Sun, Jie, 2009. "Hybridizing principles of the Electre method with case-based reasoning for data mining: Electre-CBR-I and Electre-CBR-II," European Journal of Operational Research, Elsevier, vol. 197(1), pages 214-224, August.
    10. Gather, Ursula & Fried, Roland & Lanius, Vivian, 2005. "Robust detail-preserving signal extraction," Technical Reports 2005,54, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    11. Xue, Puning & Zhou, Zhigang & Fang, Xiumu & Chen, Xin & Liu, Lin & Liu, Yaowen & Liu, Jing, 2017. "Fault detection and operation optimization in district heating substations based on data mining techniques," Applied Energy, Elsevier, vol. 205(C), pages 926-940.
    12. Min-feng Lee & Guey-shya Chen & Shao-pin Lin & Wei-jie Wang, 2022. "A Data Mining Study on House Price in Central Regions of Taiwan Using Education Categorical Data, Environmental Indicators, and House Features Data," Sustainability, MDPI, vol. 14(11), pages 1-15, May.
    13. Caruso, Germán & Scartascini, Carlos & Tommasi, Mariano, 2015. "Are we all playing the same game? The economic effects of constitutions depend on the degree of institutionalization," European Journal of Political Economy, Elsevier, vol. 38(C), pages 212-228.
    14. Zuo, Yijun, 2024. "Non-asymptotic robustness analysis of regression depth median," Journal of Multivariate Analysis, Elsevier, vol. 199(C).
    15. Debruyne, M. & Hubert, M. & Portnoy, S. & Vanden Branden, K., 2008. "Censored depth quantiles," Computational Statistics & Data Analysis, Elsevier, vol. 52(3), pages 1604-1614, January.
    16. MARIA Dan Stefan, 2009. "Improving The Quality Of The Decision Making By Using Business Intelligence Solutions," Annals of Faculty of Economics, University of Oradea, Faculty of Economics, vol. 4(1), pages 996-1000, May.
    17. M. Almiñana & L. Escudero & A. Pérez-Martín & A. Rabasa & L. Santamaría, 2014. "A classification rule reduction algorithm based on significance domains," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 22(1), pages 397-418, April.
    18. Wellmann, Robin & Harmand, Peter & Müller, Christine H., 2009. "Distribution-free tests for polynomial regression based on simplicial depth," Journal of Multivariate Analysis, Elsevier, vol. 100(4), pages 622-635, April.
    19. Silvia Figini & Ron Kenett & SILVIA SALINI, 2010. "Integrating Operational and Financial Risk Assessments," UNIMI - Research Papers in Economics, Business, and Statistics unimi-1099, Universitá degli Studi di Milano.
    20. Ursula Gather & Karen Schettlinger & Roland Fried, 2006. "Online signal extraction by robust linear regression," Computational Statistics, Springer, vol. 21(1), pages 33-51, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:sfb475:200603. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/isdorde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.