Dynamic and Transparent Analysis of Commodity Production Systems

We propose a framework that provides a programming interface to perform complex dynamic system-level analyses of deployed production systems. By leveraging hardware support for virtualization available nowadays on all commodity machines, our framework is completely transparent to the system under analysis and it guarantees isolation of the analysis tools running on its top. Thus, the internals of the kernel of the running system needs not to be modified and the whole platform runs unaware of the framework. Moreover, errors in the analysis tools do not affect the running system and the framework. This is accomplished by installing a minimalistic virtual machine monitor and migrating the system, as it runs, into a virtual machine. In order to demonstrate the potentials of our framework we developed an interactive kernel debugger, nicknamed HyperDbg. HyperDbg can be used to debug any critical kernel component, and even to single step the execution of exception and interrupt handlers.Comment: 10 pages, To appear in the 25th IEEE/ACM International Conference on Automated Software Engineering, Antwerp, Belgium, 20-24 September 201

Runtime Automated Detection of Out of Process Resource Management in the X Windowing System

Software applications typically allocate and deallocate resources during their lifetime. Resources can be categorized into two broad groups, in-process and out-of-process resources where in-process resources are local resources directly managed by a client, while out-of-process resources are remotely managed by a client which instructs a server to allocate and deallocate the resource on its behalf. Out-of-process resources do not reside in a clients address space which poses an extra layer of complexity in attempting to debug their misuse. This thesis presents an automatic run-time solution to the problem of detecting and reporting source code locations of application client mismanagement of out-of-process resources for a specific case-study of the X Windowing System which lends itself to use in the wider general case

Detecting kernel rootkits

Kernel rootkits are a special category of malware that are deployed directly in the kernel and hence have unmitigated reign over the functionalities of the kernel itself. We seek to detect such rootkits that are deployed in the real world by first observing how the majority of kernel rootkits operate. To this end, comparable to how rootkits function in the real world, we write our own kernel rootkit that manipulates the network driver, thus giving us control over all packets sent into the network. We then implement a mechanism to thwart the attacks of such rootkits by noticing that a large number of the rootkits deployed today rely heavily on the redirection of function pointers within the kernel. By overwriting the desired function pointer to its own function, a rootkit can perform a proverbial man-in-the-middle attack. Our goal is not just the detection of kernel rootkits, but also to levy as little an impact on system performance as possible. Hence our technique is to leverage existing kernel functionalities (in the case of Linux) such as kprobes to identify potential attack scenarios from within the sytem rather than from outside it (such as a VMM). We hope to introduce real-world security in devices where performance and resource constraints are tantamount to security considerations

Using a LowLevel Virtual Machine to improve dynamic aspect support in operating system kernels

ABSTRACT Current implementations of software providing dynamic aspect functionality in operating system (OS) kernels are quite restricted in the possible joinpoint types for native code they are able to support. Most of the projects implementing advice for native code use basic technologies adopted from instrumentation methods which allow to provide before, after and around joinpoints for functions. More elaborate joinpoints, however, are not available since support for monitoring native code execution in current CPUs is very restricted without extensive extensions of the compiler toolchain. To realize improved ways of aspect activation in OS kernels, we present an architecture that provides an efficient low-level virtual machine running on top of a microkernel system in cooperation with an aspect deployment service to provide novel ways of aspect activation in kernel environments

Evaluation of Dynamic Binary Instrumentation Approaches: Dynamic Binary Translation vs. Dynamic Probe Injection

From web browsing to bank transactions, to data analysis and robot automation, just about any task necessitates or benefits from the use of software. Ensuring a piece of software to be effective requires profiling the program’s behavior to evaluate its performance, debugging the program to fix incorrect behaviors, and examining the program to detect security flaws. These tasks are made possible by instrumentation---the method of inserting code into a program to collect data about its behavior. Dynamic binary instrumentation (DBI) enables programmers to understand and reason about program behavior by inserting code into a binary during run time to collect relevant data, and is more flexible than static or source-code instrumentation, but incurs run-time overhead. This thesis attempts to extend the preexisting characterization of the tradeoffs between dynamic binary translation (DBT) and dynamic probe injection (DPI), two popular DBI approaches, using Pin and LiteInst as sample frameworks. It also describes extensions to the LiteInst framework that enable it to instrument function exits correctly. This evaluation involved using the two frameworks to instrument a set of SPEC CPU 2006 benchmarks for counting function entries alone, counting both function entries and exits, or dynamically generating a call graph. On these instrumentation tasks, Pin performed close to native binary time while LiteInst performed significantly slower. Exceptions to this observation, and analysis of the probe types used by LiteInst, suggest that LiteInst incurs significant overhead when executing a large number of probes