HIDDEN MARKOV MODELS FOR SOFTWARE PIRACY DETECTION

The unauthorized copying of software is often referred to as software piracy. Soft- ware piracy causes billions of dollars of annual losses for companies and governments worldwide. In this project, we analyze a method for detecting software piracy. A meta- morphic generator is used to create morphed copies of a base piece of software. A hidden Markov Model is trained on the opcode sequences extracted from these mor- phed copies. The trained model is then used to score suspect software to determine its similarity to the base software. A high score indicates that the suspect software may be a modified version of the base software and, therefore, further investigation is warranted. In contrast, a low score indicates that the suspect software differs sig- nificantly from the base software. We show that our approach is robust, in the sense that the base software must be extensively modified before it is not detected

Metamorphic Code Generation from LLVM IR Bytecode

Metamorphic software changes its internal structure across generations with its functionality remaining unchanged. Metamorphism has been employed by malware writers as a means of evading signature detection and other advanced detection strate- gies. However, code morphing also has potential security benefits, since it increases the “genetic diversity” of software. In this research, we have created a metamorphic code generator within the LLVM compiler framework. LLVM is a three-phase compiler that supports multiple source languages and target architectures. It uses a common intermediate representation (IR) bytecode in its optimizer. Consequently, any supported high-level programming language can be transformed to this IR bytecode as part of the LLVM compila- tion process. Our metamorphic generator functions at the IR bytecode level, which provides many advantages over previously developed metamorphic generators. The morphing techniques that we employ include dead code insertion—where the dead code is actually executed within the morphed code—and subroutine permutation. We have tested the effectiveness of our code morphing using hidden Markov model analysis

Malware Detection Using Dynamic Analysis

In this research, we explore the field of dynamic analysis which has shown promis- ing results in the field of malware detection. Here, we extract dynamic software birth- marks during malware execution and apply machine learning based detection tech- niques to the resulting feature set. Specifically, we consider Hidden Markov Models and Profile Hidden Markov Models. To determine the effectiveness of this dynamic analysis approach, we compare our detection results to the results obtained by using static analysis. We show that in some cases, significantly stronger results can be obtained using our dynamic approach

Metamorphic Detection Using Function Call Graph Analysis

Well-designed metamorphic malware can evade many commonly used malware detection techniques including signature scanning. In this research, we consider a score based on function call graph analysis. We test this score on several challenging classes of metamorphic malware and we show that the resulting detection rates yield an improvement over previous research

Structural Entropy and Metamorphic Malware

Metamorphic malware is capable of changing its internal structure without al- tering its functionality. A common signature is nonexistent in highly metamorphic malware. Consequently, such malware may remain undetected even under emulation and signature scanning combined. In this project, we use the concept of structural entropy to analyze variations in the complexity of data within a file. The process consists of two stages, namely, file segmentation and sequence comparison. In the file segmentation stage, we use entropy measurements and wavelet analysis to segment a file. The second stage measures the similarity of files by computing the edit distance between sequence segments. We apply this technique to the metamorphic detection problem and show that we can obtain strong results in certain challenging cases

Static Analysis of Malicious Java Applets

In this research, we consider the problem of detecting malicious Java applets, based on static analysis. In general, dynamic analysis is more informative, but static analysis is more efficient, and hence more practical. Consequently, static analysis is preferred, provided we can obtain results comparable to those obtained using dynamic analysis. We conducted experiments with the machine learning technique, Hidden Markov Model (HMM). We show that in some cases a static technique can detect malicious Java applets with greater accuracy than previously published research that relied on dynamic analysis

Detection and Analysis of Malware Evolution

Malware is a malicious software that causes disruption, allows access to unapproved resources, or performs other unauthorized activity. Developing effective malware detection techniques is a critical aspect of information security. One difficulty that arises is that malware often evolves over time, due to changing goals of malware developers, or to counter advances in detection. This evolution can occur through various modifications in malware code. To maintain effective malware detection, it is necessary to detect and analyze malware evolution so that appropriate countermeasures can be taken. We perform a variety of experiments to detect points in time where a malware family has likely evolved. We then conduct further experiments to confirm that such evolution has actually occurred. We validate our approach by considering a number of malware families, each of which includes a significant number of samples collected over an extended period of time. All of our experiments are based on machine learning models, and hence our techniques require minimal human intervention and can easily be automated

Classification of Malware Models

Automatically classifying similar malware families is a challenging problem. In this research, we attempt to classify malware families by applying machine learning to machine learning models. Specifically, we train hidden Markov models (HMM) for each malware family in our dataset. The resulting models are then compared in two ways. First, we treat the HMM matrices as images and experiment with convolutional neural networks (CNN) for image classification. Second, we apply support vector machines (SVM) to classify the HMMs. We analyze the results and discuss the relative advantages and disadvantages of each approach

Effective Detection of Vulnerable and Malicious Browser Extensions

Unsafely coded browser extensions can compromise the security of a browser, making them attractive targets for attackers as a primary vehicle for conducting cyber-attacks. Among others, the three factors making vulnerable extensions a high-risk security threat for browsers include: i) the wide popularity of browser extensions, ii) the similarity of browser extensions with web applications, and iii) the high privilege of browser extension scripts. Furthermore, mechanisms that specifically target to mitigate browser extension-related attacks have received less attention as opposed to solutions that have been deployed for common web security problems (such as SQL injection, XSS, logic flaws, client-side vulnerabilities, drive-by-download, etc.). To address these challenges, recently some techniques have been proposed to defend extension-related attacks. These techniques mainly focus on information flow analysis to capture suspicious data flows, impose privilege restriction on API calls by malicious extensions, apply digital signatures to monitor process and memory level activities, and allow browser users to specify policies in order to restrict the operations of extensions. This article presents a model-based approach to detect vulnerable and malicious browser extensions by widening and complementing the existing techniques. We observe and utilize various common and distinguishing characteristics of benign, vulnerable, and malicious browser extensions. These characteristics are then used to build our detection models, which are based on the Hidden Markov Model constructs. The models are well trained using a set of features extracted from a number of browser extensions together with user supplied specifications. Along the course of this study, one of the main challenges we encountered was the lack of vulnerable and malicious extension samples. To address this issue, based on our previous knowledge on testing web applications and heuristics obtained from available vulnerable and malicious extensions, we have defined rules to generate training samples. The approach is implemented in a prototype tool and evaluated using a number of Mozilla Firefox extensions. Our evaluation indicated that the approach not only detects known vulnerable and malicious extensions, but also identifies previously undetected extensions with a negligible performance overhead