Learning metamorphic malware signatures from samples

Metamorphic malware are self-modifying programs which apply semantic preserving transformations to their own code in order to foil detection systems based on signaturematching. Metamorphism impacts both software security and code protection technologies: it is used by malware writers to evade detection systems based on pattern matching and by software developers for preventing malicious host attacks through software diversification. In this paper, we consider the problem of automatically extracting metamorphic signatures from the analysis of metamorphic malware variants. We define a metamorphic signature as an abstract program representation that ideally captures all the possible code variants that might be generated during the execution of a metamorphic program. For this purpose, we developed MetaSign: a tool that takes as input a collection of metamorphic code variants and produces, as output, a set of transformation rules that could have been used to generate the considered metamorphic variants. MetaSign starts from a control flow graph representation of the input variants and agglomerates them into an automaton which approximates the considered code variants. The upper approximation process is based on the concept of widening automata, while the semantic preserving transformation rules, used by the metamorphic program, can be viewed as rewriting rules and modeled as grammar productions. In this setting, the grammar recognizes the language of code variants, while the production rules model the metamorphic transformations. In particular, we formalize the language of code variants in terms of pure context-free grammars, which are similar to context-free grammars with no terminal symbols. After the widening process, we create a positive set of samples from which we extract the productions of the grammar by applying a learning grammar technique. This allows us to learn the transformation rules used by the metamorphic engine to generate the considered code variants. We validate the results of MetaSign on some case studies

Sub-circuit Selection and Replacement Algorithms Modeled as Term Rewriting Systems

Intent protection is a model of software obfuscation which, among other criteria, prevents an adversary from understanding the program’s function for use with contextual information. Relating this framework for obfuscation to malware detection, if a malware detector can perfectly normalize a program P and any obfuscation (variant) of the program O(P), the program is not intent protected. The problem of intent protection on programs can also be modeled as intent protection on combinational logic circuits. If a malware detector can perfectly normalize a circuit C and any obfuscation (variant) O(C) of the circuit, the circuit is not intent protected. In this effort, the research group set the primary goal as determining if a malware detector based upon the mechanisms of term rewriting theory can perfectly normalize circuits transformed by a sub-circuit selection and replacement algorithm, even when the transformation algorithm is known. The research group set the secondary goal as relating this result on circuit transformations to the realm of software obfuscation. The transformation rules of the sub-circuit selection and replacement algorithm are identified and modeled as rewrite rules in a term rewriting system. These rewrite rules are examined for critical overlaps which cannot be resolved by a widely used completion algorithm known as Knuth-Bendix. The research group performs an analysis of the critical overlaps found within the rewrite rules and successfully relates these results to the instruction-substitution obfuscations of a software obfuscator

c) 2003 IEEE. Personal use of this material is permitted. However, permission to

Metamorphic malware --- including certain viruses and worms --- rewrite their code during propagation. This paper presents a method for normalizing multiple variants of metamorphic programs that perform their transformations using finite sets of instruction-sequence substitutions. The paper shows that the problem of constructing a normalizer can, in specific contexts, be formalized as a term rewriting problem. A general method is proposed for constructing normalizers. It involves modeling the metamorphic program's transformations as rewrite rules, and then modifying these rules to create a normalizing rule set. Casting the problem in terms of term rewriting exposes key challenges for constructing effective normalizers. In cases where the challenges cannot be met, approximations are proposed. The normalizer construction method is applied in a case study involving the virus called "W32.Evol". The results demonstrate that both the overall approach and the approximation schemes may have practical use on realistic malware, and may thus have the potential to improve signature-based malware scanners