A Study of Rootkit Stealth Techniques and Associated Detection Methods

In today\u27s world of advanced computing power at the fingertips of any user, we must constantly think of computer security. Information is power and this power is had within our computer systems. If we cannot trust the information within our computer systems then we cannot properly wield the power that comes from such information. Rootkits are software programs that are designed to develop and maintain an environment in which malware may hide on a computer system after successful compromise of that computer system. Rootkits cut at the very foundation of the trust that we put in our information and subsequent power. This thesis seeks to understand rootkit hiding techniques, rootkit finding techniques and develops attack trees and defense trees in order to help us identify deficiencies in detection to further increase the trust in our information systems

Covert Android Rootkit Detection: Evaluating Linux Kernel Level Rootkits on the Android Operating System

This research developed kernel level rootkits for Android mobile devices designed to avoid traditional detection methods. The rootkits use system call hooking to insert new handler functions that remove the presence of infection data. The effectiveness of the rootkit is measured with respect to its stealth against detection methods and behavior performance benchmarks. Detection method testing confirms that while detectable with proven tools, system call hooking detection is not built-in or currently available in the Google Play Android App Store. Performance behavior benchmarking showed that system call hooking affects the completion time of the targeted system calls. However, this delay\u27s magnitude may not be noticeable by users. The rootkits implemented targets Android 4.0 on the emulator available from the Android Open Source Project (AOSP) and the Samsung Galaxy Nexus. The rootkits are compiled against both Linux kernel 2.6 and 3.0, respectively. This research shows the Android\u27s Linux kernel is vulnerable to system call hooking and additional measures should be implemented before handling sensitive data with Android

Rootkit Detection Using A Cross-View Clean Boot Method

In cyberspace, attackers commonly infect computer systems with malware to gain capabilities such as remote access, keylogging, and stealth. Many malware samples include rootkit functionality to hide attacker activities on the target system. After detection, users can remove the rootkit and associated malware from the system with commercial tools. This research describes, implements, and evaluates a clean boot method using two partitions to detect rootkits on a system. One partition is potentially infected with a rootkit while the other is clean. The method obtains directory listings of the potentially infected operating system from each partition and compares the lists to find hidden files. While the clean boot method is similar to other cross-view detection techniques, this method is unique because it uses a clean partition of the same system as the clean operating system, rather than external media. The method produces a 0% false positive rate and a 40.625% true positive rate. In operation, the true positive rate should increase because the experiment produces limitations that prevent many rootkits from working properly. Limitations such as incorrect rootkit setup and rootkits that detect VMware prevent the method from detecting rootkit behavior in this experiment. Vulnerabilities of the method include the assumption that the system restore folder is clean and the assumption that the clean partition is clean. This thesis provides recommendations for more effective rootkit detection

解析回避機能を持つマルウェアの解析手法: テイント伝播によるアプローチ

早大学位記番号:新8140早稲田大

Malware detection and analysis via layered annotative execution

Malicious software (i.e., malware) has become a severe threat to interconnected computer systems for decades and has caused billions of dollars damages each year. A large volume of new malware samples are discovered daily. Even worse, malware is rapidly evolving to be more sophisticated and evasive to strike against current malware analysis and defense systems. This dissertation takes a root-cause oriented approach to the problem of automatic malware detection and analysis. In this approach, we aim to capture the intrinsic natures of malicious behaviors, rather than the external symptoms of existing attacks. We propose a new architecture for binary code analysis, which is called whole-system out-of-the-box fine-grained dynamic binary analysis, to address the common challenges in malware detection and analysis. to realize this architecture, we build a unified and extensible analysis platform, codenamed TEMU. We propose a core technique for fine-grained dynamic binary analysis, called layered annotative execution, and implement this technique in TEMU. Then on the basis of TEMU, we have proposed and built a series of novel techniques for automatic malware detection and analysis. For postmortem malware analysis, we have developed Renovo, Panorama, HookFinder, and MineSweeper, for detecting and analyzing various aspects of malware. For proactive malware detection, we have built HookScout as a proactive hook detection system. These techniques capture intrinsic characteristics of malware and thus are well suited for dealing with new malware samples and attack mechanisms

SMM rootkit: a new breed of OS independent malware

The emergence of hardware virtualization technology has led to the development of OS independent malware such as the virtual machine-based rootkits (VMBRs). In this paper, we draw attention to a different but related threat that exists on many commodity systems in operation today: The system management Mode based rootkit (SMBR). System Management mode (SMM) is a relatively obscure mode on Intel processors used for low-level hardware control. It has its own private memory space and execution environment which is generally invisible to code running outside (e.g., the Operating System). Furthermore, SMM code is completely non-preemptible, lacks any concept of privilege level, and is immune to memory protection mechanisms. These features make it a potentially attractive home for stealthy rootkits used for high-profile targeted attacks. In this paper, we present our development of a proof of concept SMM rootkit. In it, we explore the potential of system management mode for malicious use by implementing a chipset level keylogger and a network backdoor capable of directly interacting with the network card to send logged keystrokes to a remote machine via UDP and receive remote command packets stealthily. By modifying and reflashing the BIOS, the SMM rootkit can install itself on a computer even if the computer has originally locked its SMM. The rootkit hides its memory footprint and requires no changes to the existing operating system. It is compared and contrasted with VMBRs. Finally, techniques to defend against these threats are explored. By taking an offensive perspective we hope to help security researchers better understand the depth and scope of the problems posed by an emerging class of OS independent malware

Applying Memory Forensics to Rootkit Detection

Volatile memory dump and its analysis is an essential part of digital forensics. Among a number of various software and hardware approaches for memory dumping there are authors who point out that some of these approaches are not resilient to various anti-forensic techniques, and others that require a reboot or are highly platform dependent. New resilient tools have certain disadvantages such as low speed or vulnerability to rootkits which directly manipulate kernel structures e.g. page tables. A new memory forensic system - Malware Analysis System for Hidden Knotty Anomalies (MASHKA) is described in this paper. It is resilient to popular anti-forensic techniques. The system can be used for doing a wide range of memory forensics tasks. This paper describes how to apply the system for research and detection of kernel mode rootkits and also presents analysis of the most popular anti-rootkit tools.Comment: 25 pages, 3 figures, 8 tables. Paper presented at the Proceedings of the 9th annual Conference on Digital Forensics, Security and Law (CDFSL), 115-141, Richmond, VA, USA. (2014, May 28-29