Automated Virtual Machine Introspection for Host-Based Intrusion Detection

This thesis examines techniques to automate configuration of an intrusion detection system utilizing hardware-assisted virtualization. These techniques are used to detect the version of a running guest operating system, automatically configure version-specific operating system information needed by the introspection library, and to locate and monitor important operating system data structures. This research simplifies introspection library configuration and is a step toward operating system independent introspection. An operating system detection algorithm and Windows virtual machine system service dispatch table monitor are implemented using the Xen hypervisor and a modified version of the XenAccess library. All detection and monitoring is implemented from the Xen management domain. Results of the operating system detection are used to initialize the XenAccess library. Library initialization time and kernel symbol retrieval are compared to the standard library. The algorithm is evaluated using nine versions of the Windows operating system. The system service dispatch table monitor is evaluated using the Agony and ProAgent rootkits. The automation techniques successfully detect the operating system and system service dispatch table hooks for the nine Windows versions tested. The modified XenAccess library exhibits an average initialization speedup of 1.9. Kernel symbol lookup is 10 times faster, on average. The hook detector is able to detect all hooks used by both rookits

Intrusion recovery for database-backed web applications

Warp is a system that helps users and administrators of web applications recover from intrusions such as SQL injection, cross-site scripting, and clickjacking attacks, while preserving legitimate user changes. Warp repairs from an intrusion by rolling back parts of the database to a version before the attack, and replaying subsequent legitimate actions. Warp allows administrators to retroactively patch security vulnerabilities---i.e., apply new security patches to past executions---to recover from intrusions without requiring the administrator to track down or even detect attacks. Warp's time-travel database allows fine-grained rollback of database rows, and enables repair to proceed concurrently with normal operation of a web application. Finally, Warp captures and replays user input at the level of a browser's DOM, to recover from attacks that involve a user's browser. For a web server running MediaWiki, Warp requires no application source code changes to recover from a range of common web application vulnerabilities with minimal user input at a cost of 24--27% in throughput and 2--3.2 GB/day in storage.United States. Defense Advanced Research Projects Agency. Clean-slate design of Resilient, Adaptive, Secure Hosts (Contract N66001-10-2-4089)National Science Foundation (U.S.) (Award CNS-1053143)Quanta Computer (Firm)Google (Firm)Samsung Scholarship Foundatio

Autoscopy: Detecting Pattern-Searching Rootkits via Control Flow Tracing

Traditional approaches to rootkit detection assume the execution of code at a privilege level below that of the operating system kernel, with the use of virtual machine technologies to enable the detection system itself to be immune from the virus or rootkit code. In this thesis, we approach the problem of rootkit detection from the standpoint of tracing and instrumentation techniques, which work from within the kernel and also modify the kernel\u27s run-time state to detect aberrant control flows. We wish to investigate the role of emerging tracing frameworks (Kprobes, DTrace etc.) in enforcing operating system security without the reliance on a full-blown virtual machine just for the purposes of such policing. We first build a novel rootkit prototype that uses pattern-searching techniques to hijack hooks embedded in dynamically allocated memory, which we present as a showcase of emerging attack techniques. We then build an intrusion detection system-- autoscopy, atop kprobes, that detects anomalous control flow patterns typically exhibited by rootkits within a running kernel. Furthermore, to validate our approach, we show that we were able to successfully detect 15 existing Linux rootkits. We also conduct performance analyses, which show the overhead of our system to range from 2% to 5% on a wide range of standard benchmarks. Thus by leveraging tracing frameworks within operating systems, we show that it is possible to introduce real-world security in devices where performance and resource constraints are tantamount to security considerations

APHID: Anomaly Processor in Hardware for Intrusion Detection

The Anomaly Processor in Hardware for Intrusion Detection (APHID) is a step forward in the field of co-processing intrusion detection mechanism. By using small, fast hardware primitives APHID relieves the production CPU from the burden of security processing. These primitives are tightly coupled to the CPU giving them access to critical state information such as the current instruction(s) in execution, the next instruction, registers, and processor state information. By monitoring these hardware elements, APHID is able to determine when an anomalous action occurs within one clock cycle. Upon detection, APHID can force the processor into a corrective state, or a halted state, depending on the required response. APHID primitives also harden the production system against attacks such as Distribute Denial of Service attack and buffer overflow attacks. APHID is designed to be fast and agile, with the ability to create multiple monitors that switch in and out of monitoring with the context switches of the production processor to highly focused coverage over multiple devices and sections of code