Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2003.Includes bibliographical references (p. 77-82).We present program shepherding, a method for monitoring control flow transfers during program execution in order to enforce a security policy. Program shepherding provides three basic techniques as building blocks for security policies. First, program shepherding can restrict execution privileges on the basis of code origins. This distinction can ensure that malicious code masquerading as data is never executed, thwarting a large class of security attacks. Second, shepherding can restrict control transfers based on instruction type, source, and target. Finally, shepherding guarantees that sandboxing checks around any program operation will never be bypassed. Security attacks use inevitable bugs in trusted binaries to coerce a program into performing actions that it was never intended to perform. We use static and dynamic analyses to automatically build a custom security policy for a target program, which specifies the program's execution model. An accurate execution model restricts control flow transfers only to the intended ones and can thwart attacker attempts to alter program execution. For example, shepherding will allow execution of shared library code only through declared entry points. Finer specifications can be extracted from high-level information present in programs' source code - for example, which values a function pointer may take. Program shepherding will allow indirect calls only to their known targets, and function returns only to known callers. These analyses build a strict enough policy to prevent all deviations from the program's control flow graph and nearly all violations of the calling convention. This technique renders most security vulnerabilities unexploitable and thwarts current and future security attacks. We present an efficient implementation of program shepherding's capabilities in the DynamoRIO [6, 7] runtime code modification system. The resulting system imposes minimal performance overhead, operates on unmodified binaries, and requires no special hardware or operating system support.by Vladimir L. Kiriansky.M.Eng
