IDEAS home Printed from https://ideas.repec.org/a/spr/advdac/v12y2018i4d10.1007_s11634-016-0265-7.html
   My bibliography  Save this article

A comparative study on large scale kernelized support vector machines

Author

Listed:
  • Daniel Horn

    (Technische Universität Dortmund)

  • Aydın Demircioğlu

    (Ruhr-Universität Bochum)

  • Bernd Bischl

    (LMU München)

  • Tobias Glasmachers

    (Ruhr-Universität Bochum)

  • Claus Weihs

    (Technische Universität Dortmund)

Abstract

Kernelized support vector machines (SVMs) belong to the most widely used classification methods. However, in contrast to linear SVMs, the computation time required to train such a machine becomes a bottleneck when facing large data sets. In order to mitigate this shortcoming of kernel SVMs, many approximate training algorithms were developed. While most of these methods claim to be much faster than the state-of-the-art solver LIBSVM, a thorough comparative study is missing. We aim to fill this gap. We choose several well-known approximate SVM solvers and compare their performance on a number of large benchmark data sets. Our focus is to analyze the trade-off between prediction error and runtime for different learning and accuracy parameter settings. This includes simple subsampling of the data, the poor-man’s approach to handling large scale problems. We employ model-based multi-objective optimization, which allows us to tune the parameters of learning machine and solver over the full range of accuracy/runtime trade-offs. We analyze (differences between) solvers by studying and comparing the Pareto fronts formed by the two objectives classification error and training time. Unsurprisingly, given more runtime most solvers are able to find more accurate solutions, i.e., achieve a higher prediction accuracy. It turns out that LIBSVM with subsampling of the data is a strong baseline. Some solvers systematically outperform others, which allows us to give concrete recommendations of when to use which solver.

Suggested Citation

  • Daniel Horn & Aydın Demircioğlu & Bernd Bischl & Tobias Glasmachers & Claus Weihs, 2018. "A comparative study on large scale kernelized support vector machines," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(4), pages 867-883, December.
  • Handle: RePEc:spr:advdac:v:12:y:2018:i:4:d:10.1007_s11634-016-0265-7
    DOI: 10.1007/s11634-016-0265-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11634-016-0265-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11634-016-0265-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bischl, Bernd & Lang, Michel & Mersmann, Olaf & Rahnenführer, Jörg & Weihs, Claus, 2015. "BatchJobs and BatchExperiments: Abstraction Mechanisms for Using R in Batch Environments," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 64(i11).
    2. Joachims, Thorsten, 1998. "Making large-scale SVM learning practical," Technical Reports 1998,28, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Behzad Pirouz & Behrouz Pirouz, 2023. "Multi-Objective Models for Sparse Optimization in Linear Support Vector Machine Classification," Mathematics, MDPI, vol. 11(17), pages 1-18, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luca Zanni, 2006. "An Improved Gradient Projection-based Decomposition Technique for Support Vector Machines," Computational Management Science, Springer, vol. 3(2), pages 131-145, April.
    2. Peng Han & Xinyue Yang & Yifei Zhao & Xiangmin Guan & Shengjie Wang, 2022. "Quantitative Ground Risk Assessment for Urban Logistical Unmanned Aerial Vehicle (UAV) Based on Bayesian Network," Sustainability, MDPI, vol. 14(9), pages 1-13, May.
    3. Andrej Čopar & Blaž Zupan & Marinka Zitnik, 2019. "Fast optimization of non-negative matrix tri-factorization," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-15, June.
    4. Hoi-Ming Chi & Okan K. Ersoy & Herbert Moskowitz & Kemal Altinkemer, 2007. "Toward Automated Intelligent Manufacturing Systems (AIMS)," INFORMS Journal on Computing, INFORMS, vol. 19(2), pages 302-312, May.
    5. Andrea Manno & Laura Palagi & Simone Sagratella, 2018. "Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training," Computational Optimization and Applications, Springer, vol. 71(1), pages 115-145, September.
    6. Tianrui Yin & Wei Chen & Bo Liu & Changzhen Li & Luyao Du, 2023. "Light “You Only Look Once”: An Improved Lightweight Vehicle-Detection Model for Intelligent Vehicles under Dark Conditions," Mathematics, MDPI, vol. 12(1), pages 1-19, December.
    7. Härdle, Wolfgang Karl & Prastyo, Dedy Dwi & Hafner, Christian, 2012. "Support vector machines with evolutionary feature selection for default prediction," SFB 649 Discussion Papers 2012-030, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    8. Prabowo, Rudy & Thelwall, Mike, 2009. "Sentiment analysis: A combined approach," Journal of Informetrics, Elsevier, vol. 3(2), pages 143-157.
    9. Luminita STATE & Catalina COCIANU & Cristian USCATU & Marinela MIRCEA, 2013. "Extensions of the SVM Method to the Non-Linearly Separable Data," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 17(2), pages 173-182.
    10. C. J. Lin & S. Lucidi & L. Palagi & A. Risi & M. Sciandrone, 2009. "Decomposition Algorithm Model for Singly Linearly-Constrained Problems Subject to Lower and Upper Bounds," Journal of Optimization Theory and Applications, Springer, vol. 141(1), pages 107-126, April.
    11. Andrea Manno & Laura Palagi & Simone Sagratella, 2014. "A Class of Convergent Parallel Algorithms for SVMs Training," DIAG Technical Reports 2014-17, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    12. Giampaolo Liuzzi & Laura Palagi & Mauro Piacentini, 2010. "On the convergence of a Jacobi-type algorithm for Singly Linearly-Constrained Problems Subject to simple Bounds," DIS Technical Reports 2010-01, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    13. Yu Bian & Hao Chen & Zujian Liu & Ling Chen & Ya Guo & Yongpeng Yang, 2024. "Geological Disaster Susceptibility Evaluation Using Machine Learning: A Case Study of the Atal Tunnel in Tibetan Plateau," Sustainability, MDPI, vol. 16(11), pages 1-23, May.
    14. Farah Mohammad & Saad Al Ahmadi, 2023. "Alzheimer’s Disease Prediction Using Deep Feature Extraction and Optimization," Mathematics, MDPI, vol. 11(17), pages 1-17, August.
    15. Wang, Yongqiang & Huang, Donghua & Sun, Kexin & Shen, Hongzheng & Xing, Xuguang & Liu, Xiao & Ma, Xiaoyi, 2023. "Multiobjective optimization of regional irrigation and nitrogen schedules by using the CERES-Maize model with crop parameters determined from the remotely sensed leaf area index," Agricultural Water Management, Elsevier, vol. 286(C).
    16. Luminita STATE & Catalina COCIANU & Doina FUSARU, 2010. "A Survey on Potential of the Support Vector Machines in Solving Classification and Regression Problems," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 14(3), pages 128-139.
    17. repec:hum:wpaper:sfb649dp2012-030 is not listed on IDEAS
    18. Weiwei Ding & Yuhong Zhang & Liya Huang, 2022. "Using a Novel Functional Brain Network Approach to Locate Important Nodes for Working Memory Tasks," IJERPH, MDPI, vol. 19(6), pages 1-14, March.
    19. Heguang Sun & Lin Zhou & Meiyan Shu & Jie Zhang & Ziheng Feng & Haikuan Feng & Xiaoyu Song & Jibo Yue & Wei Guo, 2024. "Estimation of Peanut Southern Blight Severity in Hyperspectral Data Using the Synthetic Minority Oversampling Technique and Fractional-Order Differentiation," Agriculture, MDPI, vol. 14(3), pages 1-18, March.
    20. Sachin Kumar & Aditya Sharma & B Kartheek Reddy & Shreyas Sachan & Vaibhav Jain & Jagvinder Singh, 2022. "An intelligent model based on integrated inverse document frequency and multinomial Naive Bayes for current affairs news categorisation," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 13(3), pages 1341-1355, June.
    21. Wang, Yongqiang & Sun, Kexin & Gao, Yunhe & Liu, Ruizhe & Shen, Hongzheng & Xing, Xuguang & Ma, Xiaoyi, 2024. "Improving crop model accuracy in the development of regional irrigation and nitrogen schedules by using data assimilation and spatial clustering algorithms," Agricultural Water Management, Elsevier, vol. 291(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:advdac:v:12:y:2018:i:4:d:10.1007_s11634-016-0265-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.