ECONOMICALLY PROTECTING COMPLEX, LEGACY OPERATING SYSTEMS USING SECURE DESIGN PRINCIPLES

In modern computer systems, complex legacy operating systems, such as Linux, are deployed ubiquitously. Many design choices in these legacy operating systems predate a modern understanding of security risks. As a result, new attack opportunities are routinely discovered to subvert such systems, which reveal design flaws that spur new research about secure design principles and other security mechanisms to thwart these attacks. Most research falls into two categories: encapsulating the threat and redesigning the system from scratch. Each approach has its challenge. Encapsulation can only limit the exposure to the risk, but not entirely prevent it. Rewriting the huge codebase of these operating systems is impractical in terms of developer effort, but appealing inasmuch as it can comprehensively eliminate security risks. This thesis pursues a third, understudied option: retrofitting security design principles in the existing kernel design. Conventional wisdom discourages retrofitting security because retrofitting is a hard problem, may require the use of new abstractions or break backward compatibility, may have unforeseen consequences, and may be equivalent to redesigning the system from scratch in terms of effort. This thesis offers new evidence to challenge this conventional wisdom, indicating that one can economically retrofit a comprehensive security policy onto complex, legacy systems. To demonstrate this assertion, this thesis firstly surveys the alternative of encapsulating the threat to the complex, legacy system by adding a monitoring layer using a technique called Virtual Machine Introspection, and discusses the shortcomings of this technique. Secondly, this thesis shows how to enforce the principle of least privilege by removing the need to run setuid-to-root binaries with administrator privilege. Finally, this thesis takes the first steps to show how to economically retrofit secure design principles to the OS virtualization feature of the Linux kernel called containers without rewriting the whole system. This approach can be applied more generally to other legacy systems.Doctor of Philosoph

On the Efficient Design and Testing of Dependable Systems Software

Modern computing systems that enable increasingly smart and complex applications permeate our daily lives. We strive for a fully connected and automated world to simplify our lives and increase comfort by offloading tasks to smart devices and systems. We have become dependent on the complex and ever growing ecosystem of software that drives the innovations of our smart technologies. With this dependence on complex software systems arises the question whether these systems are dependable, i.e., whether we can actually trust them to perform their intended functions. As software is developed by human beings, it must be expected to contain faults, and we need strategies and techniques to minimize both their number and the severity of their impact that scale with the increase in software complexity. Common approaches to achieve dependable operation include fault acceptance and fault avoidance strategies. The former gracefully handle faults when they occur during operation, e.g., by isolating and restarting faulty components, whereas the latter try to remove faults before system deployment, e.g., by applying correctness testing and software fault injection (SFI) techniques. On this background, this thesis aims at improving the efficiency of fault isolation for operating system kernel components, which are especially critical for dependable operation, as well as at improving the efficiency of dynamic testing activities to cope with the increasing complexity of software. Using the widely used Linux kernel, we demonstrate that partial fault isolation techniques for kernel software components can be enhanced with dynamic runtime profiles to strike a balance between the expected overheads imposed by the isolation mechanism and the achieved degree of isolation according to user requirements. With the increase in software complexity, comprehensive correctness and robustness assessments using testing and SFI require a substantially increasing number of individual tests whose execution requires a considerable amount of time. We study, considering different levels of the software stack, if modern parallel hardware can be employed to mitigate this increase. In particular, we demonstrate that SFI tests can benefit from parallel execution if such tests are carefully designed and conducted. We furthermore introduce a novel SFI framework to efficiently conduct such experiments. Moreover, we investigate if existing test suites for correctness testing can already benefit from parallel execution and provide an approach that offers a migration path for test suites that have not originally been designed for parallel execution

Efficient, transparent, and comprehensive runtime code manipulation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.Includes bibliographical references (p. 293-306).This thesis addresses the challenges of building a software system for general-purpose runtime code manipulation. Modern applications, with dynamically-loaded modules and dynamically-generated code, are assembled at runtime. While it was once feasible at compile time to observe and manipulate every instruction--which is critical for program analysis, instrumentation, trace gathering, optimization, and similar tools--it can now only be done at runtime. Existing runtime tools are successful at inserting instrumentation calls, but no general framework has been developed for fine-grained and comprehensive code observation and modification without high overheads. This thesis demonstrates the feasibility of building such a system in software. We present DynamoRIO, a fully-implemented runtime code manipulation system that supports code transformations on any part of a program, while it executes. DynamoRIO uses code caching technology to provide efficient, transparent, and comprehensive manipulation of an unmodified application running on a stock operating system and commodity hardware. DynamoRIO executes large, complex, modern applications with dynamically-loaded, generated, or even modified code. Despite the formidable obstacles inherent in the IA-32 architecture, DynamoRIO provides these capabilities efficiently, with zero to thirty percent time and memory overhead on both Windows and Linux. DynamoRIO exports an interface for building custom runtime code manipulation tools of all types. It has been used by many researchers, with several hundred downloads of our public release, and is being commercialized in a product for protection against remote security exploits, one of numerous applications of runtime code manipulation.by Derek L. Bruening.Ph.D

Understanding and Improving Security of the Android Operating System

Successful realization of practical computer security improvements requires an understanding and insight into the system\u27s security architecture, combined with a consideration of end-users\u27 needs as well as the system\u27s design tenets. In the case of Android, a system with an open, modular architecture that emphasizes usability and performance, acquiring this knowledge and insight can be particularly challenging for several reasons. In spite of Android\u27s open source philosophy, the system is extremely large and complex, documentation and reference materials are scarce, and the code base is rapidly evolving with new features and fixes. To make matters worse, the vast majority of Android devices in use do not run the open source code, but rather proprietary versions that have been heavily customized by vendors for product differentiation. Proposing security improvements or making customizations without sufficient insight into the system typically leads to less-practical, less-efficient, or even vulnerable results. Point solutions to specific problems risk leaving other similar problems in the distributed security architecture unsolved. Far-reaching general-purpose approaches may further complicate an already complex system, and force end-users to endure significant performance and usability degradations regardless of their specific security and privacy needs. In the case of vendor customization, uninformed changes can introduce access control inconsistencies and new vulnerabilities. Hence, the lack of methodologies and resources available for gaining insight about Android security is hindering the development of practical security solutions, sound vendor customizations, and end-user awareness of the proprietary devices they are using. Addressing this deficiency is the subject of this dissertation. New approaches for analyzing, evaluating and understanding Android access controls are introduced and used to create an interactive database for use by security researchers as well as system designers and end-user product evaluators. Case studies using the new techniques are described, with results uncovering problems in Android\u27s multiuser framework and vendor-customized System Services. Finally, the new insights are used to develop and implement a novel virtualization-based security architecture that protects sensitive resources while preserving Android\u27s open architecture and expected levels of performance and usability