Learning Fast and Slow: PROPEDEUTICA for Real-time Malware Detection

In this paper, we introduce and evaluate PROPEDEUTICA, a novel methodology and framework for efficient and effective real-time malware detection, leveraging the best of conventional machine learning (ML) and deep learning (DL) algorithms. In PROPEDEUTICA, all software processes in the system start execution subjected to a conventional ML detector for fast classification. If a piece of software receives a borderline classification, it is subjected to further analysis via more performance expensive and more accurate DL methods, via our newly proposed DL algorithm DEEPMALWARE. Further, we introduce delays to the execution of software subjected to deep learning analysis as a way to "buy time" for DL analysis and to rate-limit the impact of possible malware in the system. We evaluated PROPEDEUTICA with a set of 9,115 malware samples and 877 commonly used benign software samples from various categories for the Windows OS. Our results show that the false positive rate for conventional ML methods can reach 20%, and for modern DL methods it is usually below 6%. However, the classification time for DL can be 100X longer than conventional ML methods. PROPEDEUTICA improved the detection F1-score from 77.54% (conventional ML method) to 90.25%, and reduced the detection time by 54.86%. Further, the percentage of software subjected to DL analysis was approximately 40% on average. Further, the application of delays in software subjected to ML reduced the detection time by approximately 10%. Finally, we found and discussed a discrepancy between the detection accuracy offline (analysis after all traces are collected) and on-the-fly (analysis in tandem with trace collection). Our insights show that conventional ML and modern DL-based malware detectors in isolation cannot meet the needs of efficient and effective malware detection: high accuracy, low false positive rate, and short classification time.Comment: 17 pages, 7 figure

An Immune Inspired Approach to Anomaly Detection

The immune system provides a rich metaphor for computer security: anomaly detection that works in nature should work for machines. However, early artificial immune system approaches for computer security had only limited success. Arguably, this was due to these artificial systems being based on too simplistic a view of the immune system. We present here a second generation artificial immune system for process anomaly detection. It improves on earlier systems by having different artificial cell types that process information. Following detailed information about how to build such second generation systems, we find that communication between cells types is key to performance. Through realistic testing and validation we show that second generation artificial immune systems are capable of anomaly detection beyond generic system policies. The paper concludes with a discussion and outline of the next steps in this exciting area of computer security.Comment: 19 pages, 4 tables, 2 figures, Handbook of Research on Information Security and Assuranc

A Lightweight Intrusion Detection System for the Cluster Environment

As clusters of Linux workstations have gained in popularity, security in this environment has become increasingly important. While prevention methods such as access control can enhance the security level of a cluster system, intrusions are still possible and therefore intrusion detection and recovery methods are necessary. In this thesis, a system architecture for an intrusion detection system in a cluster environment is presented. A prototype system called pShield based on this architecture for a Linux cluster environment is described and its capability to detect unique attacks on MPI programs is demonstrated. The pShield system was implemented as a loadable kernel module that uses a neural network classifier to model normal behavior of processes. A new method for generating artificial anomalous data is described that uses a limited amount of attack data in training the neural network. Experimental results demonstrate that using this method rather than randomly generated anomalies reduces the false positive rate without compromising the ability to detect novel attacks. A neural network with a simple activation function is used in order to facilitate fast classification of new instances after training and to ease implementation in kernel space. Our goal is to classify the entire trace of a program¡¯s execution based on neural network classification of short sequences in the trace. Therefore, the effect of anomalous sequences in a trace must be accumulated. Several trace classification methods were compared. The results demonstrate that methods that use information about locality of anomalies are more effective than those that only look at the number of anomalies. The impact of pShield on system performance was evaluated on an 8-node cluster. Although pShield adds some overhead for each API for MPI communication, the experimental results show that a real world parallel computing benchmark was slowed only slightly by the intrusion detection system. The results demonstrate the effectiveness of pShield as a light-weight intrusion detection system in a cluster environment. This work is part of the Intelligent Intrusion Detection project of the Center for Computer Security Research at Mississippi State University

Intelligent multi-agent system for intrusion detection and countermeasures

Intelligent mobile agent systems offer a new approach to implementing intrusion detection systems (IDS). The prototype intrusion detection system, MAIDS, demonstrates the benefits of an agent-based IDS, including distributing the computational effort, reducing the amount of information sent over the network, platform independence, asynchronous operation, and modularity offering ease of updates. Anomaly detection agents use machine learning techniques to detect intrusions; one such agent processes streams of system calls from privileged processes. Misuse detection agents match known problems and correlate events to detect intrusions. Agents report intrusions to other agents and to the system administrator through the graphical user interface (GUI);A sound basis has been created for the intrusion detection system. Intrusions have been modeled using the Software Fault Tree Analysis (SFTA) technique; when augmented with constraint nodes describing trust, contextual, and temporal relationships, the SFTA forms a basis for stating the requirements of the intrusion detection system. Colored Petri Nets (CPN) have been created to model the design of the Intrusion Detection System. Algorithmic transformations are used to create CPN templates from augmented SFT and to create implementation templates from CPNs. The implementation maintains the CPN semantics in the distributed agent-based intrusion detection system

Identifying Native Applications with High Assurance

The work described in this paper investigates the problem of identifying and deterring stealthy malicious processes on a host. We point out the lack of strong application iden- tication in main stream operating systems. We solve the application identication problem by proposing a novel iden- tication model in which user-level applications are required to present identication proofs at run time to be authenti- cated by the kernel using an embedded secret key. The se- cret key of an application is registered with a trusted kernel using a key registrar and is used to uniquely authenticate and authorize the application. We present a protocol for secure authentication of applications. Additionally, we de- velop a system call monitoring architecture that uses our model to verify the identity of applications when making critical system calls. Our system call monitoring can be integrated with existing policy specication frameworks to enforce application-level access rights. We implement and evaluate a prototype of our monitoring architecture in Linux as device drivers with nearly no modication of the ker- nel. The results from our extensive performance evaluation shows that our prototype incurs low overhead, indicating the feasibility of our model