A New Generic Taxonomy on Hybrid Malware Detection Technique

Malware is a type of malicious program that replicate from host machine and propagate through network. It has been considered as one type of computer attack and intrusion that can do a variety of malicious activity on a computer. This paper addresses the current trend of malware detection techniques and identifies the significant criteria in each technique to improve malware detection in Intrusion Detection System (IDS). Several existing techniques are analyzing from 48 various researches and the capability criteria of malware detection technique have been reviewed. From the analysis, a new generic taxonomy of malware detection technique have been proposed named Hybrid-Malware Detection Technique (Hybrid-MDT) which consists of Hybrid- Signature and Anomaly detection technique and Hybrid-Specification based and Anomaly detection technique to complement the weaknesses of the existing malware detection technique in detecting known and unknown attack as well as reducing false alert before and during the intrusion occur

Simulating Windows-Based Cyber Attacks Using Live Virtual Machine Introspection

Static memory analysis has been proven a valuable technique for digital forensics. However, the memory capture technique halts the system causing the loss of important dynamic system data. As a result, live analysis techniques have emerged to complement static analysis. In this paper, a compiled memory analysis tool for virtualization (CMAT-V) is presented as a virtual machine introspection (VMI) utility to conduct live analysis during simulated cyber attacks. CMAT-V leverages static memory dump analysis techniques to provide live system state awareness. CMAT-V parses an arbitrary memory dump from a simulated guest operating system (OS) to extract user information, network usage, active process information and registry files. Unlike some VMI applications, CMAT-V bridges the semantic gap using derivation techniques. This provides increased operating system compatibility for current and future operating systems. This research demonstrates the usefulness of CMAT-V as a situational awareness tool during simulated cyber attacks and measures the overall performance of CMAT-V

Optimal remote access trojans detection based on network behavior

RAT is one of the most infected malware in the hyper-connected world. Data is being leaked or disclosed every day because new remote access Trojans are emerging and they are used to steal confidential data from target hosts. Network behavior-based detection has been used to provide an effective detection model for Remote Access Trojans. However, there is still short comings: to detect as early as possible, some False Negative Rate and accuracy that may vary depending on ratio of normal and malicious RAT sessions. As typical network contains large amount of normal traffic and small amount of malicious traffic, the detection model was built based on the different ratio of normal and malicious sessions in previous works. At that time false negative rate is less than 2%, and it varies depending on different ratio of normal and malicious instances. An unbalanced dataset will bias the prediction model towards the more common class. In this paper, each RAT is run many times in order to capture variant behavior of a Remote Access Trojan in the early stage, and balanced instances of normal applications and Remote Access Trojans are used for detection model. Our approach achieves 99 % accuracy and 0.3% False Negative Rate by Random Forest Algorithm

A Layered Architecture for Detecting Malicious Behaviors

We address the semantic gap problem in behavioral monitoring by using hierarchical behavior graphs to infer high-level behaviors from myriad low-level events that could be parts of many different kinds of behavior. Our experimental system traces the execution of a process, performing data-flow analysis to identify meaningful actions such as \u201cproxying\u201d, \u201ckeystroke logging\u201d, \u201cdata leaking\u201d, and \u201cdownloading and executing a program\u201d from complex combinations of rudimentary system calls. To preemptively address evasive malware behavior, our specifications are carefully crafted to detect alternate sequences of events that achieve the same high-level goal. We tested seven malicious bots and eleven benign programs and found that we were able to thoroughly identify high-level behaviors across this diverse code base. Moreover, we were able to distinguish malicious execution of high-level behaviors from benign by distinguishing remotely-initiated from locally-initiated actions

Rootkit Detection Using A Cross-View Clean Boot Method

In cyberspace, attackers commonly infect computer systems with malware to gain capabilities such as remote access, keylogging, and stealth. Many malware samples include rootkit functionality to hide attacker activities on the target system. After detection, users can remove the rootkit and associated malware from the system with commercial tools. This research describes, implements, and evaluates a clean boot method using two partitions to detect rootkits on a system. One partition is potentially infected with a rootkit while the other is clean. The method obtains directory listings of the potentially infected operating system from each partition and compares the lists to find hidden files. While the clean boot method is similar to other cross-view detection techniques, this method is unique because it uses a clean partition of the same system as the clean operating system, rather than external media. The method produces a 0% false positive rate and a 40.625% true positive rate. In operation, the true positive rate should increase because the experiment produces limitations that prevent many rootkits from working properly. Limitations such as incorrect rootkit setup and rootkits that detect VMware prevent the method from detecting rootkit behavior in this experiment. Vulnerabilities of the method include the assumption that the system restore folder is clean and the assumption that the clean partition is clean. This thesis provides recommendations for more effective rootkit detection

Cyber Situational Awareness Using Live Hypervisor-Based Virtual Machine Introspection

In this research, a compiled memory analysis tool for virtualization (CMAT-V) is developed as a virtual machine introspection (VMI) utility to conduct live analysis during cyber attacks. CMAT-V leverages static memory dump analysis techniques to provide live dynamic system state data. Unlike some VMI applications, CMAT-V bridges the semantic gap using derivation techniques. CMAT-V detects Windows-based operating systems and uses the Microsoft Symbol Server to provide this context to the user. This research demonstrates the usefulness of CMAT-V as a situational awareness tool during cyber attacks, tests the detection of CMAT-V from the guest system level and measures its impact on host performance. During experimental testing, live system state information was successfully extracted from two simultaneously executing virtual machines (VM’s) under four rootkit-based malware attack scenarios. For each malware attack scenario, CMAT-V was able to provide evidence of the attack. Furthermore, data from CMAT-V detection testing did not confirm detection of the presence of CMAT-V’s live memory analysis from the VM itself. This supports the conclusion that CMAT-V does not create uniquely identifiable interference in the VM. Finally, three different benchmark tests reveal an 8% to 12% decrease in the host VM performance while CMAT-V is executing