LSTM 기반 언어 모델을 통한 침입 탐지 시스템

학위논문 (석사)-- 서울대학교 대학원 공과대학 전기·정보공학부, 2017. 8. 윤성로.컴퓨터 보안에서 견고한 침입 탐지 시스템을 설계하는 것은 가장 핵심적이고 중요한 문제 중의 하나이다. 본 논문에서는 비정상 기반 호스트 침입 탐지 시스템 설계를 위한 시스템 콜 시퀀스와 분기 시퀀스에 대한 언어 모델 방법을 제안한다. 기존의 방법에서 흔히 발생하는 높은 오탐율 문제를 해결하기 위해 여러 임계값 분류기를 혼합하여 정상적인 시퀀스들을 잘 모을 수 있는 새로운 앙상블 방법을 사용하였다. 본 언어 모델은 기존 방법들이 잘 하지 못했던 각 시스템 콜의 의미와 그들 간의 상호 작용을 학습 할 수 있다는 장점이 있다. 공개된 데이터들과 새롭게 생성한 데이터를 바탕으로 다양한 실험을 통해 제안 된 방법의 타당성과 유효성을 입증하였다. 또한, 본 모델이 높은 이식성을 갖고 있음을 보였다.국문초록 i Acknowledgement ii 1 Introduction 1 2 Language Model of System Call Sequences 6 2.1 Model Architecture 6 2.2 Baseline Classifiers 8 2.3 Performance Evaluation 9 3 Ensemble Method to Reduce False Alarms 14 3.1 Ensemble Method 14 3.2 Comparsion with Other Methods 15 4 Interpretation to Transfer Learning 19 4.1 Portability of Model 19 4.2 Visualization of Learned Representations 20 5 Generalization to Branch Sequences 23 5.1 Handling Open Vocabulary Problem 23 5.2 Experiments on Branch Sequences 24 5.3 Discussion on Branch Language Model 26 6 Future Work 28 6.1 Advanced Model Architecture 28 6.2 Finding Anomalous Segments 28 6.3 Adversarial Training 29 6.4 Online Learning Framework 30 7 Conclusion 31 References 32 Abstract 37Maste

Deep Learning Approach for Intrusion Detection System (IDS) in the Internet of Things (IoT) Network using Gated Recurrent Neural Networks (GRU)

The Internet of Things (IoT) is a complex paradigm where billions of devices are connected to a network. These connected devices form an intelligent system of systems that share the data without human-to-computer or human-to-human interaction. These systems extract meaningful data that can transform human lives, businesses, and the world in significant ways. However, the reality of IoT is prone to countless cyber-attacks in the extremely hostile environment like the internet. The recent hack of 2014 Jeep Cherokee, iStan pacemaker, and a German steel plant are a few notable security breaches. To secure an IoT system, the traditional high-end security solutions are not suitable, as IoT devices are of low storage capacity and less processing power. Moreover, the IoT devices are connected for longer time periods without human intervention. This raises a need to develop smart security solutions which are light-weight, distributed and have a high longevity of service. Rather than per-device security for numerous IoT devices, it is more feasible to implement security solutions for network data. The artificial intelligence theories like Machine Learning and Deep Learning have already proven their significance when dealing with heterogeneous data of various sizes. To substantiate this, in this research, we have applied concepts of Deep Learning and Transmission Control Protocol/Internet Protocol (TCP/IP) to build a light-weight distributed security solution with high durability for IoT network security. First, we have examined the ways of improving IoT architecture and proposed a light-weight and multi-layered design for an IoT network. Second, we have analyzed the existingapplications of Machine Learning and Deep Learning to the IoT and Cyber-Security. Third, we have evaluated deep learning\u27s Gated Recurrent Neural Networks (LSTM and GRU) on the DARPA/KDD Cup \u2799 intrusion detection data set for each layer in the designed architecture. Finally, from the evaluated metrics, we have proposed the best neural network design suitable for the IoT Intrusion Detection System. With an accuracy of 98.91% and False Alarm Rate of 0.76 %, this unique research outperformed the performance results of existing methods over the KDD Cup \u2799 dataset. For this first time in the IoT research, the concepts of Gated Recurrent Neural Networks are applied for the IoT security

Applying Machine Learning to Advance Cyber Security: Network Based Intrusion Detection Systems

Many new devices, such as phones and tablets as well as traditional computer systems, rely on wireless connections to the Internet and are susceptible to attacks. Two important types of attacks are the use of malware and exploiting Internet protocol vulnerabilities in devices and network systems. These attacks form a threat on many levels and therefore any approach to dealing with these nefarious attacks will take several methods to counter. In this research, we utilize machine learning to detect and classify malware, visualize, detect and classify worms, as well as detect deauthentication attacks, a form of Denial of Service (DoS). This work also includes two prevention mechanisms for DoS attacks, namely a one- time password (OTP) and through the use of machine learning. Furthermore, we focus on an exploit of the widely used IEEE 802.11 protocol for wireless local area networks (WLANs). The work proposed here presents a threefold approach for intrusion detection to remedy the effects of malware and an Internet protocol exploit employing machine learning as a primary tool. We conclude with a comparison of dimensionality reduction methods to a deep learning classifier to demonstrate the effectiveness of these methods without compromising the accuracy of classification

Enhanced Prediction of Network Attacks Using Incomplete Data

For years, intrusion detection has been considered a key component of many organizations’ network defense capabilities. Although a number of approaches to intrusion detection have been tried, few have been capable of providing security personnel responsible for the protection of a network with sufficient information to make adjustments and respond to attacks in real-time. Because intrusion detection systems rarely have complete information, false negatives and false positives are extremely common, and thus valuable resources are wasted responding to irrelevant events. In order to provide better actionable information for security personnel, a mechanism for quantifying the confidence level in predictions is needed. This work presents an approach which seeks to combine a primary prediction model with a novel secondary confidence level model which provides a measurement of the confidence in a given attack prediction being made. The ability to accurately identify an attack and quantify the confidence level in the prediction could serve as the basis for a new generation of intrusion detection devices, devices that provide earlier and better alerts for administrators and allow more proactive response to events as they are occurring

Anomaly Detection in Sequential Data: A Deep Learning-Based Approach

Anomaly Detection has been researched in various domains with several applications in intrusion detection, fraud detection, system health management, and bio-informatics. Conventional anomaly detection methods analyze each data instance independently (univariate or multivariate) and ignore the sequential characteristics of the data. Anomalies in the data can be detected by grouping the individual data instances into sequential data and hence conventional way of analyzing independent data instances cannot detect anomalies. Currently: (1) Deep learning-based algorithms are widely used for anomaly detection purposes. However, significant computational overhead time is incurred during the training process due to static constant batch size and learning rate parameters for each epoch, (2) the threshold to decide whether an event is normal or malicious is often set as static. This can drastically increase the false alarm rate if the threshold is set low or decrease the True Alarm rate if it is set to a remarkably high value, (3) Real-life data is messy. It is impossible to learn the data features by training just one algorithm. Therefore, several one-class-based algorithms need to be trained. The final output is the ensemble of the output from all the algorithms. The prediction accuracy can be increased by giving a proper weight to each algorithm\u27s output. By extending the state-of-the-art techniques in learning-based algorithms, this dissertation provides the following solutions: (i) To address (1), we propose a hybrid, dynamic batch size and learning rate tuning algorithm that reduces the overall training time of the neural network. (ii) As a solution for (2), we present an adaptive thresholding algorithm that reduces high false alarm rates. (iii) To overcome (3), we propose a multilevel hybrid ensemble anomaly detection framework that increases the anomaly detection rate of the high dimensional dataset

Scalable and Efficient Network Anomaly Detection on Connection Data Streams

Everyday, security experts and analysts must deal with and face the huge increase of cyber security threats that are propagating very fast on the Internet and threatening the security of hundreds of millions of users worldwide. The detection of such threats and attacks is of paramount importance to these experts in order to prevent these threats and mitigate their effects in the future. Thus, the need for security solutions that can prevent, detect, and mitigate such threats is imminent and must be addressed with scalable and efficient solutions. To this end, we propose a scalable framework, called Daedalus, to analyze streams of NIDS (network-based intrusion detection system) logs in near real-time and to extract useful threat security intelligence. The proposed system pre-processes massive amounts of connections stream logs received from different participating organizations and applies an elaborated anomaly detection technique in order to distinguish between normal and abnormal or anomalous network behaviors. As such, Daedalus detects network traffic anomalies by extracting a set of significant pre-defined features from the connection logs and then applying a time series-based technique in order to detect abnormal behavior in near real-time. Moreover, we correlate IP blocks extracted from the logs with some external security signature-based feeds that detect factual malicious activities (e.g., malware families and hashes, ransomware distribution, and command and control centers) in order to validate the proposed approach. Performed experiments demonstrate that Daedalus accurately identifies the malicious activities with an average F_1 score of 92.88\%. We further compare our proposed approach with existing K-Means and deep learning (LSTMs) approaches and demonstrate the accuracy and efficiency of our system

Symmetry-Adapted Machine Learning for Information Security

Symmetry-adapted machine learning has shown encouraging ability to mitigate the security risks in information and communication technology (ICT) systems. It is a subset of artificial intelligence (AI) that relies on the principles of processing future events by learning past events or historical data. The autonomous nature of symmetry-adapted machine learning supports effective data processing and analysis for security detection in ICT systems without the interference of human authorities. Many industries are developing machine-learning-adapted solutions to support security for smart hardware, distributed computing, and the cloud. In our Special Issue book, we focus on the deployment of symmetry-adapted machine learning for information security in various application areas. This security approach can support effective methods to handle the dynamic nature of security attacks by extraction and analysis of data to identify hidden patterns of data. The main topics of this Issue include malware classification, an intrusion detection system, image watermarking, color image watermarking, battlefield target aggregation behavior recognition model, IP camera, Internet of Things (IoT) security, service function chain, indoor positioning system, and crypto-analysis