IDEAS home Printed from https://ideas.repec.org/a/sae/intdis/v14y2018i11p1550147718814471.html
   My bibliography  Save this article

A parallel algorithm for network traffic anomaly detection based on Isolation Forest

Author

Listed:
  • Xiaoling Tao
  • Yang Peng
  • Feng Zhao
  • Peichao Zhao
  • Yong Wang

Abstract

With the rapid development of large-scale complex networks and proliferation of various social network applications, the amount of network traffic data generated is increasing tremendously, and efficient anomaly detection on those massive network traffic data is crucial to many network applications, such as malware detection, load balancing, network intrusion detection. Although there are many methods around for network traffic anomaly detection, they are all designed for single machine, failing to deal with the case that the network traffic data are so large that it is prohibitive for a single computer to store and process the data. To solve these problems, we propose a parallel algorithm based on Isolation Forest and Spark for network traffic anomaly detection. We combine the advantages of Isolation Forest algorithm in network traffic anomaly detection and big data processing capability of Spark technology. Meanwhile, we apply the idea of parallelization to the process of modeling and evaluation. In the calculation process, by assigning tasks to multiple compute nodes, Isolation Forest and Spark can efficiently perform anomaly detection and evaluation process. By this way, we can also solve the problem of computation bottleneck on single machine. Extensive experiments on real world datasets show that our Isolation Forest and Spark is efficient and scales well for anomaly detection on large network traffic data.

Suggested Citation

  • Xiaoling Tao & Yang Peng & Feng Zhao & Peichao Zhao & Yong Wang, 2018. "A parallel algorithm for network traffic anomaly detection based on Isolation Forest," International Journal of Distributed Sensor Networks, , vol. 14(11), pages 15501477188, November.
  • Handle: RePEc:sae:intdis:v:14:y:2018:i:11:p:1550147718814471
    DOI: 10.1177/1550147718814471
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/1550147718814471
    Download Restriction: no

    File URL: https://libkey.io/10.1177/1550147718814471?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Li, Ming, 2017. "Record length requirement of long-range dependent teletraffic," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 472(C), pages 164-187.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dokuz, Ahmet Sakir, 2022. "Weighted spatio-temporal taxi trajectory big data mining for regional traffic estimation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 589(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Ming & Wang, Anqi, 2020. "Fractal teletraffic delay bounds in computer networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 557(C).
    2. Song, Wanqing & Li, Ming & Li, Yuanyuan & Cattani, Carlo & Chi, Chi-Hung, 2019. "Fractional Brownian motion: Difference iterative forecasting models," Chaos, Solitons & Fractals, Elsevier, vol. 123(C), pages 347-355.
    3. Li, Ming, 2020. "Multi-fractional generalized Cauchy process and its application to teletraffic," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 550(C).
    4. Liu, He & Song, Wanqing & Li, Ming & Kudreyko, Aleksey & Zio, Enrico, 2020. "Fractional Lévy stable motion: Finite difference iterative forecasting model," Chaos, Solitons & Fractals, Elsevier, vol. 133(C).
    5. Li, Ming, 2021. "Generalized fractional Gaussian noise and its application to traffic modeling," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 579(C).
    6. Li, Ming & Li, Jia-Yue, 2017. "Generalized Cauchy model of sea level fluctuations with long-range dependence," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 484(C), pages 309-335.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:intdis:v:14:y:2018:i:11:p:1550147718814471. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.