Improved Kernel Security Through Code Validation, Diversification, and Minimization

The vast majority of hosts on the Internet, including mobile clients, are running one of three commodity, general-purpose operating system families. In such operating systems the kernel software executes at the highest processor privilege level. If an adversary is able to hijack the kernel software then by extension he has full control of the system. This control includes the ability to disable protection mechanisms and hide evidence of compromise. The lack of diversity in commodity, general-purpose operating systems enables attackers to craft a single kernel exploit that has the potential to infect millions of hosts. If enough variants of the vulnerable software exist, then mass exploitation is much more difficult to achieve. We introduce novel kernel diversification techniques to improve kernel security. Many modern kernels are self-patching; they modify themselves at run-time. Self-patching kernels must therefore allow kernel code to be modified at run-time. To prevent code injection attacks, some operating systems and security mechanisms enforce a W^X memory protection policy for kernel code. This protection policy prevents self-patching kernels from applying patches at run-time. We introduce a novel run-time kernel instruction-level validation technique to validate the integrity of patches at run-time. Kernels shipped with general-purpose operating systems often contain extraneous code. The code may contain exploitable vulnerabilities or may be pieced together using return/jump-oriented programming to attack the system. Code-injection prevention techniques do not prevent such attacks. We introduce a novel run-time kernel minimization technique to improve kernel security. We show that it is possible to strengthen the defenses of commodity general-purpose computer operating systems by increasing the diversity of, validating the integrity of, and ensuring the minimality of the included kernel components without modifying the kernel source code. Such protections can therefore be added to existing widely-used unmodified operating systems to prevent malicious software from executing in supervisor mode

Hardware Mechanisms for Efficient Memory System Security

The security of a computer system hinges on the trustworthiness of the operating system and the hardware, as applications rely on them to protect code and data. As a result, multiple protections for safeguarding the hardware and OS from attacks are being continuously proposed and deployed. These defenses, however, are far from ideal as they only provide partial protection, require complex hardware and software stacks, or incur high overheads. This dissertation presents hardware mechanisms for efficiently providing strong protections against an array of attacks on the memory hardware and the operating system’s code and data. In the first part of this dissertation, we analyze and optimize protections targeted at defending memory hardware from physical attacks. We begin by showing that, contrary to popular belief, current DDR3 and DDR4 memory systems that employ memory scrambling are still susceptible to cold boot attacks (where the DRAM is frozen to give it sufficient retention time and is then re-read by an attacker after reboot to extract sensitive data). We then describe how memory scramblers in modern memory controllers can be transparently replaced by strong stream ciphers without impacting performance. We also demonstrate how the large storage overheads associated with authenticated memory encryption schemes (which enable tamper-proof storage in off-chip memories) can be reduced by leveraging compact integer encodings and error-correcting code (ECC) DRAMs – without forgoing the error detection and correction capabilities of ECC DRAMs. The second part of this dissertation presents Neverland: a low-overhead, hardware-assisted, memory protection scheme that safeguards the operating system from rootkits and kernel-mode malware. Once the system is done booting, Neverland’s hardware takes away the operating system’s ability to overwrite certain configuration registers, as well as portions of its own physical address space that contain kernel code and security-critical data. Furthermore, it prohibits the CPU from fetching privileged code from any memory region lying outside the physical addresses assigned to the OS kernel and drivers. This combination of protections makes it extremely hard for an attacker to tamper with the kernel or introduce new privileged code into the system – even in the presence of software vulnerabilities. Neverland enables operating systems to reduce their attack surface without having to rely on complex integrity monitoring software or hardware. The hardware mechanisms we present in this dissertation provide building blocks for constructing a secure computing base while incurring lower overheads than existing protections.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147604/1/salessaf_1.pd

Detecting kernel rootkits

Kernel rootkits are a special category of malware that are deployed directly in the kernel and hence have unmitigated reign over the functionalities of the kernel itself. We seek to detect such rootkits that are deployed in the real world by first observing how the majority of kernel rootkits operate. To this end, comparable to how rootkits function in the real world, we write our own kernel rootkit that manipulates the network driver, thus giving us control over all packets sent into the network. We then implement a mechanism to thwart the attacks of such rootkits by noticing that a large number of the rootkits deployed today rely heavily on the redirection of function pointers within the kernel. By overwriting the desired function pointer to its own function, a rootkit can perform a proverbial man-in-the-middle attack. Our goal is not just the detection of kernel rootkits, but also to levy as little an impact on system performance as possible. Hence our technique is to leverage existing kernel functionalities (in the case of Linux) such as kprobes to identify potential attack scenarios from within the sytem rather than from outside it (such as a VMM). We hope to introduce real-world security in devices where performance and resource constraints are tantamount to security considerations

Infrastructural Security for Virtualized Grid Computing

The goal of the grid computing paradigm is to make computer power as easy to access as an electrical power grid. Unlike the power grid, the computer grid uses remote resources located at a service provider. Malicious users can abuse the provided resources, which not only affects their own systems but also those of the provider and others. Resources are utilized in an environment where sensitive programs and data from competitors are processed on shared resources, creating again the potential for misuse. This is one of the main security issues, since in a business environment competitors distrust each other, and the fear of industrial espionage is always present. Currently, human trust is the strategy used to deal with these threats. The relationship between grid users and resource providers ranges from highly trusted to highly untrusted. This wide trust relationship occurs because grid computing itself changed from a research topic with few users to a widely deployed product that included early commercial adoption. The traditional open research communities have very low security requirements, while in contrast, business customers often operate on sensitive data that represents intellectual property; thus, their security demands are very high. In traditional grid computing, most users share the same resources concurrently. Consequently, information regarding other users and their jobs can usually be acquired quite easily. This includes, for example, that a user can see which processes are running on another user´s system. For business users, this is unacceptable since even the meta-data of their jobs is classified. As a consequence, most commercial customers are not convinced that their intellectual property in the form of software and data is protected in the grid. This thesis proposes a novel infrastructural security solution that advances the concept of virtualized grid computing. The work started back in 2007 and led to the development of the XGE, a virtual grid management software. The XGE itself uses operating system virtualization to provide a virtualized landscape. Users’ jobs are no longer executed in a shared manner; they are executed within special sandboxed environments. To satisfy the requirements of a traditional grid setup, the solution can be coupled with an installed scheduler and grid middleware on the grid head node. To protect the prominent grid head node, a novel dual-laned demilitarized zone is introduced to make attacks more difficult. In a traditional grid setup, the head node and the computing nodes are installed in the same network, so a successful attack could also endanger the user´s software and data. While the zone complicates attacks, it is, as all security solutions, not a perfect solution. Therefore, a network intrusion detection system is enhanced with grid specific signatures. A novel software called Fence is introduced that supports end-to-end encryption, which means that all data remains encrypted until it reaches its final destination. It transfers data securely between the user´s computer, the head node and the nodes within the shielded, internal network. A lightweight kernel rootkit detection system assures that only trusted kernel modules can be loaded. It is no longer possible to load untrusted modules such as kernel rootkits. Furthermore, a malware scanner for virtualized grids scans for signs of malware in all running virtual machines. Using virtual machine introspection, that scanner remains invisible for most types of malware and has full access to all system calls on the monitored system. To speed up detection, the load is distributed to multiple detection engines simultaneously. To enable multi-site service-oriented grid applications, the novel concept of public virtual nodes is presented. This is a virtualized grid node with a public IP address shielded by a set of dynamic firewalls. It is possible to create a set of connected, public nodes, either present on one or more remote grid sites. A special web service allows users to modify their own rule set in both directions and in a controlled manner. The main contribution of this thesis is the presentation of solutions that convey the security of grid computing infrastructures. This includes the XGE, a software that transforms a traditional grid into a virtualized grid. Design and implementation details including experimental evaluations are given for all approaches. Nearly all parts of the software are available as open source software. A summary of the contributions and an outlook to future work conclude this thesis

Protecting Commodity Operating Systems through Strong Kernel Isolation

Today’s operating systems are large, complex, and plagued with vulnerabilities that allow perpetrators to exploit them for profit. The constant rise in the number of software weaknesses, coupled with the sophistication of modern adversaries, make the need for effective and adaptive defenses more critical than ever. In this dissertation, we develop a set of novel protection mechanisms, and introduce new concepts and techniques to secure commodity operating systems against attacks that exploit vulnerabilities in kernel code. Modern OSes opt for a shared process/kernel model to minimize the overhead of operations that cross protection domains. However, this design choice provides a unique vantage point to local attackers, as it allows them to control—both in terms of permissions and contents—part of the memory that is accessible by the kernel, easily circumventing protections like kernel-space ASLR and WˆX. Attacks that leverage the weak separation between user and kernel space, characterized as return-to-user (ret2usr) attacks, have been the de facto kernel exploitation technique in virtually every major OS, while they are not limited to the x86 platform, but have also targeted ARM and others. Given the multi-OS and cross-architecture nature of ret2usr threats, we propose kGuard: a kernel protection mechanism, realized as a cross-platform compiler extension, which can safeguard any 32- or 64-bit OS kernel from ret2usr attacks. kGuard enforces strong address space segregation by instrumenting exploitable control transfers with dynamic Control- Flow Assertions (CFAs). CFAs, a new confinement (inline monitoring) concept that we introduce, act as guards that prevent the unconstrained transition of privileged execution paths to user space. To thwart attacks against itself, kGuard also incorporates two novel code diversification techniques: code inflation and CFA motion. Both countermeasures randomize the location of the inline guards, creating a moving target for an attacker that tries to pinpoint their exact placement to evade kGuard. Evaluation results indicate that kGuard provides comprehensive ret2usr protection with negligible overhead (∼1%). Furthermore, we expose a set of additional kernel design practices that trade stronger isolation for performance, all of which can be harnessed to deconstruct kernel isolation. To demonstrate the significance of the problem, we introduce a new kernel exploitation technique, dubbed return-to-direct-mapped memory (ret2dir), which relies on inherent properties of the memory management (sub)system of modern OSes to bypass every ret2usr defense to date. To illustrate the effectiveness of ret2dir, we outline a principled methodology for constructing reliable exploits against hardened targets. We further apply it on real-world kernel exploits for x86, x86-64, and ARM Linux, transforming them into ret2dir-equivalents that bypass deployed ret2usr protections, like Intel SMEP and ARM PXN. Finally, we introduce the concept of eXclusive Page Frame Ownership (XPFO): a memory management approach that prevents the implicit sharing of page frames among user processes and the kernel, ensuring that user-controlled content can no longer be injected into kernel space using ret2dir. We built XPFO on Linux and implemented a set of optimizations, related to TLB handling and page frame content sanitization, to minimize its performance penalty. Evaluation results show that our proposed defense offers effective protection against ret2dir attacks with low runtime overhead (<3%)

Dynamic reliability and security monitoring: a virtual machine approach

While one always works to prevent attacks and failures, they are inevitable and situational awareness is key to taking appropriate action. Monitoring plays an integral role in ensuring reliability and security of computing systems. Infrastructure as a Service (IaaS) clouds significantly lower the barrier for obtaining scalable computing resources and allow users to focus on what is important to them. Can a similar service be offered to provide on-demand reliability and security monitoring? Cloud computing systems are typically built using virtual machines (VMs). VM monitoring takes advantage of this and uses the hypervisor that runs VMs for robust reliability and security monitoring. The hypervisor provides an environment that is isolated from failures and attacks inside customers’ VMs. Furthermore, as a low-level manager of computing resources, the hypervisor has full access to the infrastructure running above it. Hypervisor-based VM monitoring leverages that information to observe the VMs for failures and attacks. However, existing VM monitoring techniques fall short of “as-a-service” expectations because they require a priori VM modifications and require human interaction to obtain necessary information about the underlying guest system. The research presented in this dissertation closes those gaps by providing a flexible VM monitoring framework and automated analysis to support that framework. We have developed and tested a dynamic VM monitoring framework called Hypervisor Probes (hprobes). The hprobe framework allows us to monitor the execution of both the guest OS and applications from the hypervisor. To supplement this monitoring framework, we use dynamic analysis techniques to investigate the relationship between hardware events visible to the hyper-visor and OS constructs common across OS versions. We use the results of this analysis to parametrize the hprobe-based monitors without requiring any user input. Combining the dynamic VM monitoring framework and analysis frameworks allows us to provide on-demand hypervisor based monitors for cloud VMs

DEALING WITH NEXT-GENERATION MALWARE

Malicious programs are a serious problem that threatens the security of billions of Internet users. Today's malware authors are motivated by the easy financial gain they can obtain by selling on the underground market the information stolen from the infected hosts. To maximize their profit, miscreants continuously improve their creations to make them more and more resilient against anti-malware solutions. This increasing sophistication in malicious code led to next-generation malware, a new class of threats that exploit the limitations of state-of-the-art anti-malware products to bypass security protections and eventually evade detection. Unfortunately, current anti-malware technologies are inadequate to face next-generation malware. For this reason, in this dissertation we propose novel techniques to address the shortcomings of defensive technologies and to enhance current state-of-the-art security solutions. Dynamic behavior-based analysis is a very promising approach to automatically understand the behaviors a malicious program may exhibit at run-time. However, behavior-based solutions still present several limitations. First of all, these techniques may give incomplete results because the execution environments in which they are applied are synthetic and do not faithfully resemble the environments of end-users, the intended targets of the malicious activities. To overcome this problem, we present a new framework for improving behavior-based analysis of suspicious programs, that allows an end-user to delegate security labs the execution and the analysis of a program and to force the program to behave as if it were executed directly in the environment of the former. Our evaluation demonstrated that the proposed framework allows security labs to improve the completeness of the analysis, by analyzing a piece of malware on behalf of multiple end-users simultaneously, while performing a fine-grained analysis of the behavior of the program with no computational cost for the end-users. Another drawback of state-of-the-art defensive solutions is non-transparency: malicious programs are often able to determine that their execution is being monitored, and thus they can tamper with the analysis to avoid detection, or simply behave innocuously to mislead the anti-malware tool. At this aim, we propose a generic framework to perform complex dynamic system-level analyses of deployed production systems. By leveraging hardware support for virtualization available nowadays on all commodity machines, our framework is completely transparent to the system under analysis and it guarantees isolation of the analysis tools running on top of it. The internals of the kernel of the running system need not to be modified and the whole platform runs unaware of the framework. Once the framework has been installed, even kernel-level malware cannot detect it or affect its execution. This is accomplished by installing a minimalistic virtual machine monitor and migrating the system, as it runs, into a virtual machine. To demonstrate the potentials of our framework we developed an interactive kernel debugger, named HyperDbg. As HyperDbg can be used to monitor any critical system component, it is suitable to analyze even malicious programs that include kernel-level modules. Despite all the progress anti-malware technologies can make, perfect malware detection remains an undecidable problem. When it is not possible to prevent a malicious threat from infecting a system, post-infection remediation remains the only viable possibility. However, if the machine has already been compromised, the execution of the remediation tool could be tampered by the malware that is running on the system. To address this problem we present Conqueror, a software-based attestation scheme for tamper-proof code execution on untrusted legacy systems. Besides providing load-time attestation of a piece of code, Conqueror also ensures run-time integrity. Conqueror constitutes a valid alternative to trusted computing platforms, for systems lacking specialized hardware for attestation. We implemented a prototype, specific for the Intel x86 architecture, and evaluated the proposed scheme. Our evaluation showed that, compared to competitors, Conqueror is resistant to both static and dynamic attacks. We believe Conqueror and our transparent dynamic analysis framework constitute important building blocks for creating new security applications. To demonstrate this claim, we leverage the aforementioned solutions to realize HyperSleuth, an infrastructure to securely perform live forensic analysis of potentially compromised production systems. HyperSleuth provides a trusted execution environment that guarantees an attacker controlling the system cannot interfere with the analysis and cannot tamper with the results. The framework can be installed as the system runs, without a reboot and without loosing any volatile data. Moreover, the analysis can be periodically and safely interrupted to resume normal execution of the system. On top of HyperSleuth we implemented three forensic analysis tools: a lazy physical memory dumper, a lie detector, and a system call tracer. The experimental evaluation we conducted demonstrated that even time consuming analyses, such as the dump of the content of the physical memory, can be securely performed without interrupting the services offered by the system