Enforcing security policies with runtime monitors

Le monitorage (monitoring) est une approche pour la sécurisation du code qui permet l'exécution d’un code potentiellement malicieux en observant son exécution, et en intervenant au besoin pour éviter une violation d’une politique de sécurité. Cette méthode a plusieurs applications prometteuses, notamment en ce qui a trait à la sécurisation du code mobile. Les recherches académiques sur le monitorage se sont généralement concentrées sur deux questions. La première est celle de délimiter le champ des politiques de sécurité applicables par des moniteurs opérant sous différentes contraintes. La seconde question est de construire des méthodes permettant d’insérer un moniteur dans un programme, ce qui produit un nouveau programme instrumenté qui respecte la politique de sécurité appliquée par ce moniteur. Mais malgré le fait qu’une vaste gamme de moniteurs a été étudiée dans la littérature, les travaux sur l’insertion des moniteurs dans les programmes se sont limités à une classe particulière de moniteurs, qui sont parmi les plus simples et les plus restreint quant à leur champ de politiques applicables. Cette thèse étend les deux avenues de recherches mentionnées précédemment et apporte un éclairage nouveau à ces questions. Elle s’attarde en premier lieu à étendre le champ des politiques applicables par monitorage en développabt une nouvelle approche pour l’insertion d’un moniteur dans un programme. En donnant au moniteur accès à un modèle du comportement du programme, l’étude montre que le moniteur acquiert la capacité d’appliquer une plus vaste gamme de politiques de sécurité. De plus, les recherches ont aussi d´emontré qu’un moniteur capable de transformer l’exécution qu’il surveille est plus puissant qu’un moniteur qui ne possède pas cette capacité. Naturellement, des contraintes doivent être imposées sur cette capacité pour que l’application de la politique soit cohérente. Autrement, si aucune restriction n’est imposée au moniteur, n’importe quelle politique devient applicable, mais non d’une manière utile ou désirable. Dans cette étude, nous proposons deux nouveaux paradigmes d’application des politiques de sécurité qui permettent d’incorporer des restrictions raisonnables imposées sur la capacité des moniteurs de transformer les exécutions sous leur contrôle. Nous étudions le champ des politiques applicables avec ces paradigmes et donnons des exemples de politiques réelles qui peuvent être appliquées à l’aide de notre approche.Execution monitoring is an approach that seeks to allow an untrusted code to run safely by observing its execution and reacting if need be to prevent a potential violation of a user-supplied security policy. This method has many promising applications, particularly with respect to the safe execution of mobile code. Academic research on monitoring has generally focused on two questions. The first, relates to the set of policies that can be enforced by monitors under various constraints and the conditions under which this set can be extended. The second question deals with the way to inline a monitor into an untrusted or potentially malicious program in order to produce a new instrumented program that provably respects the desired security policy. This study builds on the two strands of research mentioned above and brings new insights to this study. It seeks, in the first place, to increase the scope of monitorable properties by suggesting a new approach of monitor inlining. By drawing on an a priori model of the program’s possible behavior, we develop a monitor that can enforce a strictly larger set of security properties. Furthermore, longstanding research has showed that a monitor that is allowed to transform its input is more powerful than one lacking this ability. Naturally, this ability must be constrained for the enforcement to be meaningful. Otherwise, if the monitor is given too broad a leeway to transform valid and invalid sequences, any property can be enforced, but not in a way that is useful or desirable. In this study, we propose two new enforcement paradigms which capture reasonable restrictions on a monitor’s ability to alter its input. We study the set of properties enforceable if these enforcement paradigms are used and give examples of real-life security policies that can be enforced using our approach

A Hybrid Framework for the Systematic Detection of Software Security Vulnerabilities in Source Code

In this thesis, we address the problem of detecting vulnerabilities in software where the source code is available, such as free-and-open-source software. In this, we rely on the use of security testing. Either static or dynamic analysis can be used for security testing approaches, yet both analyses have their advantages and drawbacks. In fact, while these analyses are different, they are complementary to each other in many ways. Consequently, approaches that would combine these analyses have the potential of becoming very advantageous to security testing and vulnerability detection. This has motivated the work presented in this thesis. For the purpose of security testing, security analysts need to specify the security properties that they wish to test software against for security violations. Accordingly, we firstly propose a security model called Team Edit Automata (TEA), which extends security automata. Using TEA, security analysts are capable of precisely specifying the security properties under concerns. Since various code instrumentations are needed at different program points for the purpose of profiling the software behavior at run-time, we secondly propose a code instrumentation profiler. Furthermore, we provide an extension to the GCC compiler to enable such instrumentations. The profiler is based on the pointcut model of Aspect-Oriented Programming (AOP) languages and accordingly it is capable of providing a large set of instrumentation capabilities to the analysts. We particularly explore the capabilities and the current limitations of AOP languages as tools for security testing code instrumentation, and propose extensions to these languages to allow them to be used for such purposes. Thirdly, we explore the potential of static analysis for vulnerability detection and illustrate its applicability and limitations. Fourthly, we propose a framework that reduces security vulnerability detection to a reachability problem. The framework combines three main techniques: static analysis, program slicing, and reachability analysis. This framework mainly targets software applications that are generally categorized as being safety/security critical, and are of relatively small sizes, such as embedded software. Finally, we propose a more comprehensive security testing and test-data generation framework that provides further advantages over the proposed reachability model. This framework combines the power of static and dynamic analyses, and is used to generate concrete data, with which the existence of a vulnerability is proven beyond doubt, hence mitigating major drawbacks of static analysis, namely false positives. We also illustrate the feasibility of the elaborated frameworks by developing case studies for test-data generation and vulnerability detection on various-size software

A Hybrid Framework for the Systematic Detection of Software Security Vulnerabilities in Source Code

In this thesis, we address the problem of detecting vulnerabilities in software where the source code is available, such as free-and-open-source software. In this, we rely on the use of security testing. Either static or dynamic analysis can be used for security testing approaches, yet both analyses have their advantages and drawbacks. In fact, while these analyses are different, they are complementary to each other in many ways. Consequently, approaches that would combine these analyses have the potential of becoming very advantageous to security testing and vulnerability detection. This has motivated the work presented in this thesis. For the purpose of security testing, security analysts need to specify the security properties that they wish to test software against for security violations. Accordingly, we firstly propose a security model called Team Edit Automata (TEA), which extends security automata. Using TEA, security analysts are capable of precisely specifying the security properties under concerns. Since various code instrumentations are needed at different program points for the purpose of profiling the software behavior at run-time, we secondly propose a code instrumentation profiler. Furthermore, we provide an extension to the GCC compiler to enable such instrumentations. The profiler is based on the pointcut model of Aspect-Oriented Programming (AOP) languages and accordingly it is capable of providing a large set of instrumentation capabilities to the analysts. We particularly explore the capabilities and the current limitations of AOP languages as tools for security testing code instrumentation, and propose extensions to these languages to allow them to be used for such purposes. Thirdly, we explore the potential of static analysis for vulnerability detection and illustrate its applicability and limitations. Fourthly, we propose a framework that reduces security vulnerability detection to a reachability problem. The framework combines three main techniques: static analysis, program slicing, and reachability analysis. This framework mainly targets software applications that are generally categorized as being safety/security critical, and are of relatively small sizes, such as embedded software. Finally, we propose a more comprehensive security testing and test-data generation framework that provides further advantages over the proposed reachability model. This framework combines the power of static and dynamic analyses, and is used to generate concrete data, with which the existence of a vulnerability is proven beyond doubt, hence mitigating major drawbacks of static analysis, namely false positives. We also illustrate the feasibility of the elaborated frameworks by developing case studies for test-data generation and vulnerability detection on various-size software