Malware Detection Using Dynamic Analysis

In this research, we explore the field of dynamic analysis which has shown promis- ing results in the field of malware detection. Here, we extract dynamic software birth- marks during malware execution and apply machine learning based detection tech- niques to the resulting feature set. Specifically, we consider Hidden Markov Models and Profile Hidden Markov Models. To determine the effectiveness of this dynamic analysis approach, we compare our detection results to the results obtained by using static analysis. We show that in some cases, significantly stronger results can be obtained using our dynamic approach

A Comparison of Clustering Techniques for Malware Analysis

In this research, we apply clustering techniques to the malware detection problem. Our goal is to classify malware as part of a fully automated detection strategy. We compute clusters using the well-known �-means and EM clustering algorithms, with scores obtained from Hidden Markov Models (HMM). The previous work in this area consists of using HMM and �-means clustering technique to achieve the same. The current effort aims to extend it to use EM clustering technique for detection and also compare this technique with the �-means clustering

Metamorphic Code Generation from LLVM IR Bytecode

Metamorphic software changes its internal structure across generations with its functionality remaining unchanged. Metamorphism has been employed by malware writers as a means of evading signature detection and other advanced detection strate- gies. However, code morphing also has potential security benefits, since it increases the “genetic diversity” of software. In this research, we have created a metamorphic code generator within the LLVM compiler framework. LLVM is a three-phase compiler that supports multiple source languages and target architectures. It uses a common intermediate representation (IR) bytecode in its optimizer. Consequently, any supported high-level programming language can be transformed to this IR bytecode as part of the LLVM compila- tion process. Our metamorphic generator functions at the IR bytecode level, which provides many advantages over previously developed metamorphic generators. The morphing techniques that we employ include dead code insertion—where the dead code is actually executed within the morphed code—and subroutine permutation. We have tested the effectiveness of our code morphing using hidden Markov model analysis

Combining Dynamic and Static Analysis for Malware Detection

Well-designed malware can evade static detection techniques, such as signature scanning. Dynamic analysis strips away one layer of obfuscation and hence such an approach can potentially provide more accurate detection results. However, dynamic analysis is generally more costly than static analysis. In this research, we analyze the effectiveness of using dynamic analysis to enhance the training phase, while using only static techniques in the detection phase. Relative to a fully static approach, the additional overhead is minimal, since training is essentially one-time work

Detecting Botnets Using Hidden Markov Model, Profile Hidden Markov Model and Network Flow Analysis

Botnet is a network of infected computer systems called bots managed remotely by an attacker using bot controllers. Using distributed systems, botnets can be used for large-scale cyber attacks to execute unauthorized actions on the targeted system like phishing, distributed denial of service (DDoS), data theft, and crashing of servers. Common internet protocols used by normal systems for regular communication like hypertext transfer (HTTP) and internet relay chat (IRC) are also used by botnets. Thus, distinguishing botnet activity from normal activity can be challenging. To address this issue, this project proposes an approach to detect botnets using peculiar traits in the communication between command and control servers and bots. Patterns can be observed in botnet behavior like orchestrated attacks, heartbeat signals, or periodic distribution of commands. Hidden Markov Models (HMM) and Profile Hidden Markov Model (PHMM) are probabilistic models that can be trained on network traffic data to identify activity patterns that suggest botnet activity. In this project, HMM and PHMM are used to detect and classify botnets using publicly available datasets for real network data consisting of botnet traffic mixed with normal and background traffic. A comparative analysis of performance of HMM and PHMM is conducted in this project and the results show that HMM and PHMM can be useful in detecting botnets. PHMM outperforms HMM in terms of accuracy of botnet detection

Tiresias: Predicting Security Events Through Deep Learning

With the increased complexity of modern computer attacks, there is a need for defenders not only to detect malicious activity as it happens, but also to predict the specific steps that will be taken by an adversary when performing an attack. However this is still an open research problem, and previous research in predicting malicious events only looked at binary outcomes (e.g., whether an attack would happen or not), but not at the specific steps that an attacker would undertake. To fill this gap we present Tiresias, a system that leverages Recurrent Neural Networks (RNNs) to predict future events on a machine, based on previous observations. We test Tiresias on a dataset of 3.4 billion security events collected from a commercial intrusion prevention system, and show that our approach is effective in predicting the next event that will occur on a machine with a precision of up to 0.93. We also show that the models learned by Tiresias are reasonably stable over time, and provide a mechanism that can identify sudden drops in precision and trigger a retraining of the system. Finally, we show that the long-term memory typical of RNNs is key in performing event prediction, rendering simpler methods not up to the task