Hunting for Undetectable Metamorphic Viruses

Commercial anti-virus scanners are generally signature based, that is, they scan for known patterns to determine whether a file is infected by a virus or not. To evade signature-based detection, virus writers have adopted code obfuscation techniques to create highly metamorphic computer viruses. Since metamorphic viruses change their appearance from generation to generation, signature-based scanners cannot detect all instances of such viruses. To combat metamorphic viruses, detection tools based on statistical analysis have been studied. A tool based on hidden Markov models (HMMs) was previously developed and the results are encouraging—it has been shown that metamorphic viruses created by a well-designed metamorphic engine can be detected using an HMM. In this project, we explore whether there are any exploitable weaknesses in this HMM-based detection approach. We create a highly metamorphic virus generating tool designed specifically to evade HMM-based detection. We then test our engine, showing that we can generate viral copies that cannot be detected using previously-developed HMM-based detection techniques. Finally, we consider possible defenses against our approach

Metamorphic Code Generation from LLVM IR Bytecode

Metamorphic software changes its internal structure across generations with its functionality remaining unchanged. Metamorphism has been employed by malware writers as a means of evading signature detection and other advanced detection strate- gies. However, code morphing also has potential security benefits, since it increases the “genetic diversity” of software. In this research, we have created a metamorphic code generator within the LLVM compiler framework. LLVM is a three-phase compiler that supports multiple source languages and target architectures. It uses a common intermediate representation (IR) bytecode in its optimizer. Consequently, any supported high-level programming language can be transformed to this IR bytecode as part of the LLVM compila- tion process. Our metamorphic generator functions at the IR bytecode level, which provides many advantages over previously developed metamorphic generators. The morphing techniques that we employ include dead code insertion—where the dead code is actually executed within the morphed code—and subroutine permutation. We have tested the effectiveness of our code morphing using hidden Markov model analysis

Robust Watermarking using Hidden Markov Models

Software piracy is the unauthorized copying or distribution of software. It is a growing problem that results in annual losses in the billions of dollars. Prevention is a difficult problem since digital documents are easy to copy and distribute. Watermarking is a possible defense against software piracy. A software watermark consists of information embedded in the software, which allows it to be identified. A watermark can act as a deterrent to unauthorized copying, since it can be used to provide evidence for legal action against those responsible for piracy.In this project, we present a novel software watermarking scheme that is inspired by the success of previous research focused on detecting metamorphic viruses. We use a trained hidden Markov model (HMM) to detect a specific copy of software. We give experimental results that show our scheme is robust. That is, we can identify the original software even after it has been extensively modified, as might occur as part of an attack on the watermarking scheme

HIDDEN MARKOV MODELS FOR SOFTWARE PIRACY DETECTION

The unauthorized copying of software is often referred to as software piracy. Soft- ware piracy causes billions of dollars of annual losses for companies and governments worldwide. In this project, we analyze a method for detecting software piracy. A meta- morphic generator is used to create morphed copies of a base piece of software. A hidden Markov Model is trained on the opcode sequences extracted from these mor- phed copies. The trained model is then used to score suspect software to determine its similarity to the base software. A high score indicates that the suspect software may be a modified version of the base software and, therefore, further investigation is warranted. In contrast, a low score indicates that the suspect software differs sig- nificantly from the base software. We show that our approach is robust, in the sense that the base software must be extensively modified before it is not detected

METAMORPHIC WORM THAT CARRIES ITS OWN MORPHING ENGINE

Metamorphic malware changes its internal structure across generations, but its functionality remains unchanged. Well-designed metamorphic malware will evade signature detection. Recent research has revealed techniques based on hidden Markov models (HMMs) for detecting many types of metamorphic malware, as well as techniques for evading such detection. A worm is a type of malware that actively spreads across a network to other host systems. In this project we design and implement a prototype metamorphic worm that carries its own morphing engine. This is challenging, since the morphing engine itself must be morphed across replications, which imposes significant restrictions on the structure of the worm. Our design also employs previously developed techniques to evade detection. We provide test results to confirm that this worm effectively evades signature and HMM-based detection, and we consider possible detection strategies. This worm provides a concrete example that should prove useful for additional malware detection research

Code Obfuscation and Virus Detection

Typically, computer viruses and other malware are detected by searching for a string of bits which is found in the virus or malware. Such a string can be viewed as a “fingerprint” of the virus. These “fingerprints” are not generally unique; however they can be used to make rapid malware scanning feasible. This fingerprint is often called a signature and the technique of detecting viruses using signatures is known as signaturebased detection [8]. Today, virus writers often camouflage their viruses by using code obfuscation techniques in an effort to defeat signature-based detection schemes. So-called metamorphic viruses are viruses in which each instance has the same functionality but differs in its internal structure. Metamorphic viruses differ from polymorphic viruses in the method they use to hide their signature. While polymorphic viruses primarily rely on encryption for signature obfuscation, metamorphic viruses hide their signature via “mutating” their own code [3]. The paper [1] provides a rigorous proof that metamorphic viruses can bypass any signature-based detection, provided the code obfuscation has been done carefully based on a set of specified rules. Specifically, according to [1], if dead code is added and the control flow is changed sufficiently by inserting jump statements, the virus cannot be detected. In this project we first developed a code obfuscation engine conforming to the rules in [1]. We then used this engine to create metamorphic variants of a seed virus (created using the PS-MPK virus creation kit [15]) and demonstrated the validity of the assertion in [1] about metamorphic viruses and signature based detectors. In the second phase of this project we validated another theory advanced in [2], namely, that machine learning based methods¾specifically ones based on Hidden Markov Model (HMM) ¾can detect metamorphic viruses. In other words, we show that a collection of metamorphic viruses which are (provably) undetectable via signature detection techniques can nevertheless be detected using an HMM approach

DETECTING UNDETECTABLE COMPUTER VIRUSES

Signature-based detection relies on patterns present in viruses and provides a relatively simple and efficient method for detecting known viruses. At present, most anti-virus systems rely primarily on signature detection. Metamorphic viruses are one of the most difficult types of viruses to detect. Such viruses change their internal structure, which provides an effective means of evading signature detection. Previous work has provided a rigorous proof that a fairly simple metamorphic engine can generate viruses that will evade any signature-based detection. In this project, we first implement a metamorphic engine that is provably undetectable—in the sense of signature-based detection. We then show that, as expected, the resulting viruses are not detected by popular commercial anti-virus scanners. Finally, we analyze the same set of viruses using a previously developed approach based on hidden Markov models (HMM). This HMM- based technique easily detects the viruses

Morphing engines classification by code histogram

Morphing engines or mutation engines are exploited by metamorphic virus to change the code appearance in every new generation. The purpose of these engines is to escape from the signature-based scanner, which employs a unique string signature to detect the virus. Although the obfuscation techniques try to convert the binary sequence of the code, in some techniques, the statistical feature of the code binaries will be still remain unchanged, relatively. Accordingly, this feature can be utilized to classify the engine and detect the morphed virus code. In this article, we are going to introduce a new idea to classify the obfuscation engines based on their code statistical feature using the histogram comparison

Metamorphic Viruses with Built-In Buffer Overflow

Metamorphic computer viruses change their structure—and thereby their signature—each time they infect a system. Metamorphic viruses are potentially one of the most dangerous types of computer viruses because they are difficult to detect using signature-based methods. Most anti-virus software today is based on signature detection techniques. In this project, we create and analyze a metamorphic virus toolkit which creates viruses with a built-in buffer overflow. The buffer overflow serves to obfuscate the entry point of the actual virus, thereby making detection more challenging. We show that the resulting viruses successfully evade detection by commercial virus scanners. Several modern operating systems (e.g., Windows Vista and Windows 7) employ address space layout randomization (ASLR), which is designed to prevent most buffer overflow attacks. We show that our proposed buffer overflow technique succeeds, even in the presence of ASLR. Finally, we consider possible defenses against our proposed technique