IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v9y2021i6p690-d522653.html
   My bibliography  Save this article

Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research

Author

Listed:
  • Ranjit Panigrahi

    (Department of Computer Applications, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, India
    These authors contributed equally to this work and are first co-authors.)

  • Samarjeet Borah

    (Department of Computer Applications, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, India)

  • Akash Kumar Bhoi

    (Department of Electrical and Electronics Engineering, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, India)

  • Muhammad Fazal Ijaz

    (Department of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, Korea
    These authors contributed equally to this work and are first co-authors.)

  • Moumita Pramanik

    (Department of Computer Applications, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, India)

  • Rutvij H. Jhaveri

    (Department of Computer Science and Engineering, School of Technology, Pandit Deendayal Energy University, Gandhinagar 382007, India)

  • Chiranji Lal Chowdhary

    (School of Information Technology & Engineering, Vellore Institute of Technology, Vellore 632014, India)

Abstract

Supervised learning and pattern recognition is a crucial area of research in information retrieval, knowledge engineering, image processing, medical imaging, and intrusion detection. Numerous algorithms have been designed to address such complex application domains. Despite an enormous array of supervised classifiers, researchers are yet to recognize a robust classification mechanism that accurately and quickly classifies the target dataset, especially in the field of intrusion detection systems (IDSs). Most of the existing literature considers the accuracy and false-positive rate for assessing the performance of classification algorithms. The absence of other performance measures, such as model build time, misclassification rate, and precision, should be considered the main limitation for classifier performance evaluation. This paper’s main contribution is to analyze the current literature status in the field of network intrusion detection, highlighting the number of classifiers used, dataset size, performance outputs, inferences, and research gaps. Therefore, fifty-four state-of-the-art classifiers of various different groups, i.e., Bayes, functions, lazy, rule-based, and decision tree, have been analyzed and explored in detail, considering the sixteen most popular performance measures. This research work aims to recognize a robust classifier, which is suitable for consideration as the base learner, while designing a host-based or network-based intrusion detection system. The NSLKDD, ISCXIDS2012, and CICIDS2017 datasets have been used for training and testing purposes. Furthermore, a widespread decision-making algorithm, referred to as Techniques for Order Preference by Similarity to the Ideal Solution (TOPSIS), allocated ranks to the classifiers based on observed performance reading on the concern datasets. The J48Consolidated provided the highest accuracy of 99.868%, a misclassification rate of 0.1319%, and a Kappa value of 0.998. Therefore, this classifier has been proposed as the ideal classifier for designing IDSs.

Suggested Citation

  • Ranjit Panigrahi & Samarjeet Borah & Akash Kumar Bhoi & Muhammad Fazal Ijaz & Moumita Pramanik & Rutvij H. Jhaveri & Chiranji Lal Chowdhary, 2021. "Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research," Mathematics, MDPI, vol. 9(6), pages 1-32, March.
    • Handle: RePEc:gam:jmathe:v:9:y:2021:i:6:p:690-:d:522653
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/9/6/690/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/9/6/690/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Diego Raphael Amancio & Cesar Henrique Comin & Dalcimar Casanova & Gonzalo Travieso & Odemir Martinez Bruno & Francisco Aparecido Rodrigues & Luciano da Fontoura Costa, 2014. "A Systematic Comparison of Supervised Classifiers," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-14, April.
    2. Maindonald, John, 2007. "Pattern Recognition and Machine Learning," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(b05).
    3. Ahmed Ahmim & Mohamed Amine Ferrag & Leandros Maglaras & Makhlouf Derdour & Helge Janicke, 2020. "A Detailed Analysis of Using Supervised Machine Learning for Intrusion Detection," Springer Proceedings in Business and Economics, in: Androniki Kavoura & Efstathios Kefallonitis & Prokopios Theodoridis (ed.), Strategic Innovative Marketing and Tourism, pages 629-639, Springer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Priscila T M Saito & Rodrigo Y M Nakamura & Willian P Amorim & João P Papa & Pedro J de Rezende & Alexandre X Falcão, 2015. "Choosing the Most Effective Pattern Classification Model under Learning-Time Constraint," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-23, June.
    2. Diego R Amancio, 2015. "Probing the Topological Properties of Complex Networks Modeling Short Written Texts," PLOS ONE, Public Library of Science, vol. 10(2), pages 1-17, February.
    3. Mariane Barros Neiva & Patrick Guidotti & Odemir Martinez Bruno, 2018. "Enhancing LBP by preprocessing via anisotropic diffusion," International Journal of Modern Physics C (IJMPC), World Scientific Publishing Co. Pte. Ltd., vol. 29(08), pages 1-29, August.
    4. Adilson Vital & Diego R. Amancio, 2022. "A comparative analysis of local similarity metrics and machine learning approaches: application to link prediction in author citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 6011-6028, October.
    5. Jorge A. V. Tohalino & Laura V. C. Quispe & Diego R. Amancio, 2021. "Analyzing the relationship between text features and grants productivity," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4255-4275, May.
    6. Ferraz de Arruda, Henrique & Reia, Sandro Martinelli & Silva, Filipi Nascimento & Amancio, Diego Raphael & da Fontoura Costa, Luciano, 2022. "Finding contrasting patterns in rhythmic properties between prose and poetry," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 598(C).
    7. Diego R. Amancio & Osvaldo N. Oliveira jr & Luciano F. Costa, 2015. "Topological-collaborative approach for disambiguating authors’ names in collaborative networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 465-485, January.
    8. Nguyen Minh Tien & Cyril Labbé, 2018. "Detecting automatically generated sentences with grammatical structure similarity," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 1247-1271, August.
    9. Yu-Tso Chen & Chi-Hua Chen & Szu Wu & Chi-Chun Lo, 2018. "A Two-Step Approach for Classifying Music Genre on the Strength of AHP Weighted Musical Features," Mathematics, MDPI, vol. 7(1), pages 1-17, December.
    10. Tohalino, Jorge A.V. & Amancio, Diego R., 2022. "On predicting research grants productivity via machine learning," Journal of Informetrics, Elsevier, vol. 16(2).
    11. Diego Raphael Amancio, 2015. "Comparing the topological properties of real and artificially generated scientific manuscripts," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1763-1779, December.
    12. Mayra Z Rodriguez & Cesar H Comin & Dalcimar Casanova & Odemir M Bruno & Diego R Amancio & Luciano da F Costa & Francisco A Rodrigues, 2019. "Clustering algorithms: A comparative approach," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-34, January.
    13. Guerreiro, Lucas & Silva, Filipi N. & Amancio, Diego R., 2024. "Recovering network topology and dynamics from sequences: A machine learning approach," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 638(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:9:y:2021:i:6:p:690-:d:522653. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.