COMPRESSION-BASED ANALYSIS OF METAMORPHIC MALWARE

Recent work has presented a technique based on structural entropy measurement as an effective way to detect metamorphic malware. The technique uses two steps, file segmentation and sequence comparison, to calculate file similarity. In another previous work, it was observed that similar malware have similar measures of Kolmogorov complexity. A proposed method of estimating Kolmogorov complexity was to calculate the compression ratio of a given malware which could then be used to cluster the malicious software. Malware detection has also been attempted through the use of adaptive data compression and showed promising results. In this paper, we attempt to combine these concepts and propose using compression ratios as an alternative measure of entropy with the purpose of segmenting files according to their structural characteristics. We then compare the segment-based sequences of two given files to determine file similarity. The idea is that even after malware is transformed using a metamorphic engine, the resulting variants still share identifiable structural similarities with the original. Using this proposed technique to identify metamorphic malware, we compare our results with previous work

Structural features with nonnegative matrix factorization for metamorphic malware detection

Metamorphic malware is well known for evading signature-based detection by exploiting various code obfuscation techniques. Current metamorphic malware detection approaches require some prior knowledge during feature engineering stage to extract patterns and behaviors from malware. In this paper, we attempt to complement and extend previous techniques by proposing a metamorphic malware detection approach based on structure analysis by using information theoretic measures and statistical metrics with machine learning model. In particular, compression ratio, entropy, Jaccard coefficient and Chi-square tests are used as feature representations to reveal the byte information existing in malware binary file. Furthermore, by using Nonnegative Matrix Factorization, feature dimension can be reduced. The experimental results show the Jaccard coefficient on hexadecimal byte as feature representation is effective for Windows metamorphic malware detection with an accuracy rate and F-score as high as 0.9972 and 0.9958, respectively. Whereas for Linux morphed malware detection, the Chi-square statistic test shows as effective feature representation with an accuracy rate and F-score as high as 0.9878 and 0.9901, respectively. Overall, the proposed feature representations and the technique of dimension reduction can be useful for detecting metamorphic malware

Nonnegative matrix factorization and metamorphic malware detection

Metamorphic malware change their internal code structure by adopting code obfuscation technique while maintaining their malicious functionality during each infection. This causes change of their signature pattern across each infection and makes signature based detection particularly difficult. In this paper, through static analysis, we use similarity score from matrix factorization technique called Nonnegative Matrix Factorization for detecting challenging metamorphic malware. We apply this technique using structural compression ratio and entropy features and compare our results with previous eigenvector-based techniques. Experimental results from three malware datasets show this is a promising technique as the accuracy detection is more than 95%

Hunting for Pirated Software Using Metamorphic Analysis

In this paper, we consider the problem of detecting software that has been pirated and modified. We analyze a variety of detection techniques that have been previously studied in the context of malware detection. For each technique, we empirically determine the detection rate as a function of the degree of modification of the original code. We show that the code must be greatly modified before we fail to reliably distinguish it, and we show that our results offer a significant improvement over previous related work. Our approach can be applied retroactively to any existing software and hence, it is both practical and effective

Unveiling metamorphism by abstract interpretation of code properties

Metamorphic code includes self-modifying semantics-preserving transformations to exploit code diversification. The impact of metamorphism is growing in security and code protection technologies, both for preventing malicious host attacks, e.g., in software diversification for IP and integrity protection, and in malicious software attacks, e.g., in metamorphic malware self-modifying their own code in order to foil detection systems based on signature matching. In this paper we consider the problem of automatically extracting metamorphic signatures from metamorphic code. We introduce a semantics for self-modifying code, later called phase semantics, and prove its correctness by showing that it is an abstract interpretation of the standard trace semantics. Phase semantics precisely models the metamorphic code behavior by providing a set of traces of programs which correspond to the possible evolutions of the metamorphic code during execution. We show that metamorphic signatures can be automatically extracted by abstract interpretation of the phase semantics. In particular, we introduce the notion of regular metamorphism, where the invariants of the phase semantics can be modeled as finite state automata representing the code structure of all possible metamorphic change of a metamorphic code, and we provide a static signature extraction algorithm for metamorphic code where metamorphic signatures are approximated in regular metamorphism

Learning metamorphic malware signatures from samples

Metamorphic malware are self-modifying programs which apply semantic preserving transformations to their own code in order to foil detection systems based on signaturematching. Metamorphism impacts both software security and code protection technologies: it is used by malware writers to evade detection systems based on pattern matching and by software developers for preventing malicious host attacks through software diversification. In this paper, we consider the problem of automatically extracting metamorphic signatures from the analysis of metamorphic malware variants. We define a metamorphic signature as an abstract program representation that ideally captures all the possible code variants that might be generated during the execution of a metamorphic program. For this purpose, we developed MetaSign: a tool that takes as input a collection of metamorphic code variants and produces, as output, a set of transformation rules that could have been used to generate the considered metamorphic variants. MetaSign starts from a control flow graph representation of the input variants and agglomerates them into an automaton which approximates the considered code variants. The upper approximation process is based on the concept of widening automata, while the semantic preserving transformation rules, used by the metamorphic program, can be viewed as rewriting rules and modeled as grammar productions. In this setting, the grammar recognizes the language of code variants, while the production rules model the metamorphic transformations. In particular, we formalize the language of code variants in terms of pure context-free grammars, which are similar to context-free grammars with no terminal symbols. After the widening process, we create a positive set of samples from which we extract the productions of the grammar by applying a learning grammar technique. This allows us to learn the transformation rules used by the metamorphic engine to generate the considered code variants. We validate the results of MetaSign on some case studies

Exhaustive Statistical Analysis for Detection of Metamorphic Malware

Malware is a serious threat to the security of the system. With the widespread use of the World Wide Web, there has been a tremendous increase in virus attacks, making computer security an essential for every personal computer. The rat-race between virus writers and detectors has led to improved viruses and detection techniques. In recent years, metamorphic malwares have posed serious challenge to anti-virus writers. Current signature based detection techniques, heuristic based techniques are not comprehensive solutions. A formidable solution to detection of metamorphic malware is void. This paper investigates the problem of malware detection, specifically metamorphic malwares. The paper proposes a statistical based detection technique as a viable candidate for comprehensive detection of metamorphic malwares. Related work, experimental results and analysis of the results are presented in this paper

Clustering versus SVM for Malware Detection

Previous work has shown that we can effectively cluster certain classes of mal- ware into their respective families. In this research, we extend this previous work to the problem of developing an automated malware detection system. We first compute clusters for a collection of malware families. Then we analyze the effectiveness of clas- sifying new samples based on these existing clusters. We compare results obtained using �-means and Expectation Maximization (EM) clustering to those obtained us- ing Support Vector Machines (SVM). Using clustering, we are able to detect some malware families with an accuracy comparable to that of SVMs. One advantage of the clustering approach is that there is no need to retrain for new malware families