Machine learning based botnet identification traffic

The continued growth of the Internet has resulted in the increasing sophistication of toolkit and methods to conduct computer attacks and intrusions that are easy to use and publicly available to download, such as Zeus botnet toolkit. Botnets are responsible for many cyber-attacks, such as spam, distributed denial-of-service (DDoS), identity theft, and phishing. Most of existence botnet toolkits release updates for new features, development and support. This presents challenges in the detection and prevention of bots. Current botnet detection approaches mostly ineffective as botnets change their Command and Control (C&C) server structures, centralized (e.g., IRC, HTTP), distributed (e.g., P2P), and encryption deterrent. In this paper, based on real world data sets we present our preliminary research on predicting the new bots before they launch their attack. We propose a rich set of features of network traffic using Classification of Network Information Flow Analysis (CONIFA) framework to capture regularities in C&C communication channels and malicious traffic. We present a case study of applying the approach to a popular botnet toolkit, Zeus. The experimental evaluation suggest that it is possible to detect effectively botnets during the botnet C&C communication generated from new updated Zeus botnet toolkit by building the classifier using machine learning from an earlier version and before they launch their attacks using traffic behaviors. Also, show that there is similarity in C&C structures various Botnet toolkit versions and that the network characteristics of botnet C&C traffic is different from legitimate network traffic. Such methods could reduce many different resources needed to identify C&C communication channels and malicious traffic

In-depth comparative evaluation of supervised machine learning approaches for detection of cybersecurity threats

This paper describes the process and results of analyzing CICIDS2017, a modern, labeled data set for testing intrusion detection systems. The data set is divided into several days, each pertaining to different attack classes (Dos, DDoS, infiltration, botnet, etc.). A pipeline has been created that includes nine supervised learning algorithms. The goal was binary classification of benign versus attack traffic. Cross-validated parameter optimization, using a voting mechanism that includes five classification metrics, was employed to select optimal parameters. These results were interpreted to discover whether certain parameter choices were dominant for most (or all) of the attack classes. Ultimately, every algorithm was retested with optimal parameters to obtain the final classification scores. During the review of these results, execution time, both on consumerand corporate-grade equipment, was taken into account as an additional requirement. The work detailed in this paper establishes a novel supervised machine learning performance baseline for CICIDS2017

Machine learning based botnet identification traffic

The continued growth of the Internet has resulted in the increasing sophistication of toolkit and methods to conduct computer attacks and intrusions that are easy to use and publicly available to download, such as Zeus botnet toolkit. Botnets are responsible for many cyber-attacks, such as spam, distributed denial-of-service (DDoS), identity theft, and phishing. Most of existence botnet toolkits release updates for new features, development and support. This presents challenges in the detection and prevention of bots. Current botnet detection approaches mostly ineffective as botnets change their Command and Control (C&C) server structures, centralized (e.g., IRC, HTTP), distributed (e.g., P2P), and encryption deterrent. In this paper, based on real world data sets we present our preliminary research on predicting the new bots before they launch their attack. We propose a rich set of features of network traffic using Classification of Network Information Flow Analysis (CONIFA) framework to capture regularities in C&C communication channels and malicious traffic. We present a case study of applying the approach to a popular botnet toolkit, Zeus. The experimental evaluation suggest that it is possible to detect effectively botnets during the botnet C&C communication generated from new updated Zeus botnet toolkit by building the classifier using machine learning from an earlier version and before they launch their attacks using traffic behaviors. Also, show that there is similarity in C&C structures various Botnet toolkit versions and that the network characteristics of botnet C&C traffic is different from legitimate network traffic. Such methods could reduce many different resources needed to identify C&C communication channels and malicious traffic

Deep Learning for Network Traffic Monitoring and Analysis (NTMA): A Survey

Modern communication systems and networks, e.g., Internet of Things (IoT) and cellular networks, generate a massive and heterogeneous amount of traffic data. In such networks, the traditional network management techniques for monitoring and data analytics face some challenges and issues, e.g., accuracy, and effective processing of big data in a real-time fashion. Moreover, the pattern of network traffic, especially in cellular networks, shows very complex behavior because of various factors, such as device mobility and network heterogeneity. Deep learning has been efficiently employed to facilitate analytics and knowledge discovery in big data systems to recognize hidden and complex patterns. Motivated by these successes, researchers in the field of networking apply deep learning models for Network Traffic Monitoring and Analysis (NTMA) applications, e.g., traffic classification and prediction. This paper provides a comprehensive review on applications of deep learning in NTMA. We first provide fundamental background relevant to our review. Then, we give an insight into the confluence of deep learning and NTMA, and review deep learning techniques proposed for NTMA applications. Finally, we discuss key challenges, open issues, and future research directions for using deep learning in NTMA applications.publishedVersio

NetSentry: A deep learning approach to detecting incipient large-scale network attacks

Machine Learning (ML) techniques are increasingly adopted to tackle ever-evolving high-profile network attacks, including DDoS, botnet, and ransomware, due to their unique ability to extract complex patterns hidden in data streams. These approaches are however routinely validated with data collected in the same environment, and their performance degrades when deployed in different network topologies and/or applied on previously unseen traffic, as we uncover. This suggests malicious/benign behaviors are largely learned superficially and ML-based Network Intrusion Detection System (NIDS) need revisiting, to be effective in practice. In this paper we dive into the mechanics of large-scale network attacks, with a view to understanding how to use ML for Network Intrusion Detection (NID) in a principled way. We reveal that, although cyberattacks vary significantly in terms of payloads, vectors and targets, their early stages, which are critical to successful attack outcomes, share many similarities and exhibit important temporal correlations. Therefore, we treat NID as a time-sensitive task and propose NetSentry, perhaps the first of its kind NIDS that builds on Bidirectional Asymmetric LSTM (Bi-ALSTM), an original ensemble of sequential neural models, to detect network threats before they spread. We cross-evaluate NetSentry using two practical datasets, training on one and testing on the other, and demonstrate F1 score gains above 33% over the state-of-the-art, as well as up to 3 times higher rates of detecting attacks such as XSS and web bruteforce. Further, we put forward a novel data augmentation technique that boosts the generalization abilities of a broad range of supervised deep learning algorithms, leading to average F1 score gains above 35%