Multi-agent-based DDoS detection on big data systems

The Hadoop framework has become the most deployed platform for processing Big Data. Despite its advantages, Hadoop s infrastructure is still deployed within the secured network perimeter because the framework lacks adequate inherent security mechanisms against various security threats. However, this approach is not sufficient for providing adequate security layer against attacks such as Distributed Denial of Service. Furthermore, current work to secure Hadoop s infrastructure against DDoS attacks is unable to provide a distributed node-level detection mechanism. This thesis presents a software agent-based framework that allows distributed, real-time intelligent monitoring and detection of DDoS attack at Hadoop s node-level. The agent s cognitive system is ingrained with cumulative sum statistical technique to analyse network utilisation and average server load and detect attacks from these measurements. The framework is a multi-agent architecture with transducer agents that interface with each Hadoop node to provide real-time detection mechanism. Moreover, the agents contextualise their beliefs by training themselves with the contextual information of each node and monitor the activities of the node to differentiate between normal and anomalous behaviours. In the experiments, the framework was exposed to TCP SYN and UDP flooding attacks during a legitimate MapReduce job on the Hadoop testbed. The experimental results were evaluated regarding performance metrics such as false-positive ratio, false-negative ratio and response time to attack. The results show that UDP and TCP SYN flooding attacks can be detected and confirmed on multiple nodes in nineteen seconds with 5.56% false-positive ration, 7.70% false-negative ratio and 91.5% success rate of detection. The results represent an improvement compare to the state-of the-ar

Analisis Statistik Log Jaringan untuk Deteksi Serangan Ddos Berbasis Neural Network

Distributed denial-of-service (DDoS) merupakan jenis serangan dengan volume, intensitas, dan biaya mitigasi yang terus meningkat seiring berkembangnya skala organisasi. Penelitian ini memiliki tujuan untuk mengembangkan sebuah pendekatan baru untuk mendeteksi serangan DDoS, berdasarkan log jaringan yang dianalisis secara statistik dengan fungsi neural network sebagai metode deteksi. Data pelatihan dan pengujian diambil dari CAIDA DDoS Attack 2007 dan simulasi mandiri. Pengujian terhadap metode analisis statistik terhadap log jaringan dengan fungsi neural network sebagai metode deteksi menghasilkan prosentase rata-rata pengenalan terhadap tiga kondisi jaringan (normal, slow DDoS, dan DDoS) sebesar 90,52%. Adanya pendekatan baru dalam mendeteksi serangan DDoS, diharapkan bisa menjadi sebuah komplemen terhadap sistem Intrusion Detection System (IDS) dalam meramalkan terjadinya serangan DDo

A Survey on Big Data for Network Traffic Monitoring and Analysis

Network Traffic Monitoring and Analysis (NTMA) represents a key component for network management, especially to guarantee the correct operation of large-scale networks such as the Internet. As the complexity of Internet services and the volume of traffic continue to increase, it becomes difficult to design scalable NTMA applications. Applications such as traffic classification and policing require real-time and scalable approaches. Anomaly detection and security mechanisms require to quickly identify and react to unpredictable events while processing millions of heterogeneous events. At last, the system has to collect, store, and process massive sets of historical data for post-mortem analysis. Those are precisely the challenges faced by general big data approaches: Volume, Velocity, Variety, and Veracity. This survey brings together NTMA and big data. We catalog previous work on NTMA that adopt big data approaches to understand to what extent the potential of big data is being explored in NTMA. This survey mainly focuses on approaches and technologies to manage the big NTMA data, additionally briefly discussing big data analytics (e.g., machine learning) for the sake of NTMA. Finally, we provide guidelines for future work, discussing lessons learned, and research directions

Klasifikasi Paket Jaringan Berbasis Analisis Statistik dan Neural Network

Distributed Denial-of-Service (DDoS) is one of network attack technique which increased every year, especially in both of intensity and volume. DDoS attacks are still one of the world's major Internet threats and become a major problem of cyber-world security. Research in this paper aims to establish a new approach on network packets classification, which can be a basis for framework development on Distributed Denial-of-Service (DDoS) attack detection systems. The proposed approach to solving the problem on network packet classification is by combining statistical data quantification methods with neural network methods. Based on the test, it is found that the average percentage of neural network classification accuracy against network data packet is 92.99%

Real-time big data processing for anomaly detection : a survey

The advent of connected devices and omnipresence of Internet have paved way for intruders to attack networks, which leads to cyber-attack, financial loss, information theft in healthcare, and cyber war. Hence, network security analytics has become an important area of concern and has gained intensive attention among researchers, off late, specifically in the domain of anomaly detection in network, which is considered crucial for network security. However, preliminary investigations have revealed that the existing approaches to detect anomalies in network are not effective enough, particularly to detect them in real time. The reason for the inefficacy of current approaches is mainly due the amassment of massive volumes of data though the connected devices. Therefore, it is crucial to propose a framework that effectively handles real time big data processing and detect anomalies in networks. In this regard, this paper attempts to address the issue of detecting anomalies in real time. Respectively, this paper has surveyed the state-of-the-art real-time big data processing technologies related to anomaly detection and the vital characteristics of associated machine learning algorithms. This paper begins with the explanation of essential contexts and taxonomy of real-time big data processing, anomalous detection, and machine learning algorithms, followed by the review of big data processing technologies. Finally, the identified research challenges of real-time big data processing in anomaly detection are discussed. © 2018 Elsevier Lt

Towards Large-Scale, Heterogeneous Anomaly Detection Systems in Industrial Networks: A Survey of Current Trends

Industrial Networks (INs) are widespread environments where heterogeneous devices collaborate to control and monitor physical processes. Some of the controlled processes belong to Critical Infrastructures (CIs), and, as such, IN protection is an active research field. Among different types of security solutions, IN Anomaly Detection Systems (ADSs) have received wide attention from the scientific community.While INs have grown in size and in complexity, requiring the development of novel, Big Data solutions for data processing, IN ADSs have not evolved at the same pace. In parallel, the development of BigData frameworks such asHadoop or Spark has led the way for applying Big Data Analytics to the field of cyber-security,mainly focusing on the Information Technology (IT) domain. However, due to the particularities of INs, it is not feasible to directly apply IT security mechanisms in INs, as IN ADSs face unique characteristics. In this work we introduce three main contributions. First, we survey the area of Big Data ADSs that could be applicable to INs and compare the surveyed works. Second, we develop a novel taxonomy to classify existing INbased ADSs. And, finally, we present a discussion of open problems in the field of Big Data ADSs for INs that can lead to further development

Implemetasi Jaringan Saraf Tiruan Untuk Mendeteksi Serangan DDoS Pada Forensik Jaringan

Network attacks that are often carried out including using Distributed Denial of Service (DDoS) have caused significant financial losses and require very large recovery costs to reach double. Activities that damage, interfere with, steal data, and anything that harms the system owner of a computer network is illegal and can be legally sanctioned in court. Network forensics mechanism to find criminals in order to be ensnared by law. Investigators usually use network monitoring systems such as Intrusion Detection System (IDS) for forensics purposes. The use of IDS allows the detection of errors or changes in traffic and new types of attacks because attacks are carried out using syn packages, where the syn protocol is considered legal because it is needed in the authentication process of communication between devices in the Internet network. Signature-based detection and notification systems are also not strong enough to be used as evidence in the trial. An analysis mechanism is needed to test the accuracy of DDoS attacks that have been detected by the intrusion detection system. Testing the accuracy of DDoS attacks can be done using the neural network classification method using statistical calculations. Based on the results of the analysis and testing carried out found an accuracy value of 95.23%. These results can be used to support and strengthen the evidence of findings in the trial.Keywords: DDoS, IDS, network forensics, JST