Grasp : visualizing the behavior of hierarchical multiprocessor real-time systems

Trace visualization is a viable approach for gaining insight into the behavior of complex distributed real-time systems. Grasp is a versatile trace visualization toolset. This paper presents its unique visualization capabilities for hierarchical multiprocessor systems, including partitioned and global multiprocessor scheduling with migrating tasks and jobs, communication between jobs via shared memory and message passing, and hierarchical scheduling in combination with multiprocessor scheduling. Its flexible plugin infrastructure allows for easy extension with custom visualization and analysis techniques for automatic trace verification. Grasp is freely available on the web

Autoscopy: Detecting Pattern-Searching Rootkits via Control Flow Tracing

Traditional approaches to rootkit detection assume the execution of code at a privilege level below that of the operating system kernel, with the use of virtual machine technologies to enable the detection system itself to be immune from the virus or rootkit code. In this thesis, we approach the problem of rootkit detection from the standpoint of tracing and instrumentation techniques, which work from within the kernel and also modify the kernel\u27s run-time state to detect aberrant control flows. We wish to investigate the role of emerging tracing frameworks (Kprobes, DTrace etc.) in enforcing operating system security without the reliance on a full-blown virtual machine just for the purposes of such policing. We first build a novel rootkit prototype that uses pattern-searching techniques to hijack hooks embedded in dynamically allocated memory, which we present as a showcase of emerging attack techniques. We then build an intrusion detection system-- autoscopy, atop kprobes, that detects anomalous control flow patterns typically exhibited by rootkits within a running kernel. Furthermore, to validate our approach, we show that we were able to successfully detect 15 existing Linux rootkits. We also conduct performance analyses, which show the overhead of our system to range from 2% to 5% on a wide range of standard benchmarks. Thus by leveraging tracing frameworks within operating systems, we show that it is possible to introduce real-world security in devices where performance and resource constraints are tantamount to security considerations

Towards A Verified Complex Protocol Stack in a Production Kernel: Methodology and Demonstration

Any useful computer system performs communication and any communication must be parsed before it is computed upon. Given their importance, one might expect parsers to receive a significant share of attention from the security community. This is, however, not the case: bugs in parsers continue to account for a surprising portion of reported and exploited vulnerabilities. In this thesis, I propose a methodology for supporting the development of software that depends on parsers---such as anything connected to the Internet---to safely support any reasonably designed protocol: data structures to describe protocol messages; validation routines that check that data received from the wire conforms to the rules of the protocol; systems that allow a defender to inject arbitrary, crafted input so as to explore the effectiveness of the parser; and systems that allow for the observation of the parser code while it is being explored. Then, I describe principled method of producing parsers that automatically generates the myriad parser-related software from a description of the protocol. This has many significant benefits: it makes implementing parsers simpler, easier, and faster; it reduces the trusted computing base to the description of the protocol and the program that compiles the description to runnable code; and it allows for easier formal verification of the generated code. I demonstrate the merits of the proposed methodology by creating a description of the USB protocol using a domain-specific language (DSL) embedded in Haskell and integrating it with the FreeBSD operating system. Using the industry-standard umap test-suite, I measure the performance and efficacy of the generated parser. I show that it is stable, that it is effective at protecting a system from both accidentally and maliciously malformed input, and that it does not incur unreasonable overhead

Runtime Verification of Real-Time Applications Using Trace Data and Model Requirements

RÉSUMÉ Surveiller les systèmes multi-cœurs est difficile en raison des processus s'exécutant en parallèle, et pouvant interférer les uns avec les autres lorsqu'il s'agit d'accéder aux ressources du système, ou simplement d'avoir du temps processeur. Un tel système peut avoir à suivre des principes temps réel, ajoutant des contraintes de temps qui invalident les résultats dès qu'une date limite est manquée. Sur ce genre de système, des données précises n'auront ainsi de valeur que si elles peuvent être produites en respectant un délai donné. Le traçage peut fournir une grande quantité d'informations d'exécution de haute précision, à la fois sur le système et les applications. Il est ainsi souvent l'outil le plus précis et fiable pour étudier et analyser des systèmes ainsi contraints. Cependant, les utilisateurs doivent disposer d'une grande expertise du système afin de comprendre les évènements du noyau du système et leur signification. De plus, il peut être très long d'analyser et étudier manuellement de volumineuses traces d'exécution de haute précision. Dans cette thèse, nous proposons les méthodes et algorithmes permettant d'automatiquement détecter et identifier l'origine de comportements inattendus dans des applications, et ce à l'aide de traces de leurs exécutions et de modèles des exigences. Nous décrivons la structure interne des modèles, la méthodologie pour suivre l'exécution d'une application à travers ses évènements de l'espace utilisateur, et les structures de données nécessaires pour vérifier les contraintes. Nous détaillons ensuite le processus utilisé pour détecter et finalement comprendre la source du comportement non désiré. Nous proposons aussi une approche pour construire automatiquement les modèles pour des applications temps réel courantes. L'hypothèse servant de point de départ pour ce travail est que les traces d'exécution du système et de l'application à analyser peuvent être utilisées pour automatiquement suivre l'exécution de cette application, y détecter les anomalies et trouver leurs sources. Les résultats de ce travail sont les concepts, méthodologies et structures de données utilisés pour suivre et contraindre des applications, ainsi que les méthodes et algorithmes qui permettent de détecter et identifier les comportements inattendus dans ces applications. Ces derniers ont été testés sur de réelles applications temps réel, et ont permis avec succès de détecter et identifier l'origine des irrégularités à leur exécution. De plus, nous avons pu automatiquement, et de façon précise, construire des modèles pour ces applications. Cette dernière étape rend l'utilisation des méthodes de traçage beaucoup plus accessible aux utilisateurs non-experts. Le résultat final est que la détection automatique et la localisation automatique de la source des comportements inattendus dans une application est une option viable et fonctionnelle, qui accélère et simplifie le travail des utilisateurs pour analyser les exécutions de leurs applications.----------ABSTRACT Monitoring multi-core systems is hard because of the concurrently running processes that can contend with each other to access resources of the system or CPU time. Such a system may have to follow real-time principles, adding time constraints that invalidate results as soon as a deadline is missed. This means that accurate data will only be valuable if it can be produced in a timely fashion. Tracing is often the most accurate and reliable tool to study and analyze those systems, as it can provide a lot of high precision runtime information about both the system and applications. Nevertheless, a deep level of expertise of the system is required in order for the users to understand the kernel events and their meaning. Moreover, it can be time consuming to manually analyze and study voluminous high precision execution traces. In this thesis, we propose methods and algorithms to automatically detect and identify the origin of unwanted behaviors in applications, by using traces of their execution and models of the requirements. We describe the internal structure of the models, the methodology to follow an application runtime through its userspace events, and the data structures needed to verify constraints. We then detail the process followed to detect and finally understand the root cause of the unwanted behavior. We also propose an approach to automatically build the models for common real-time applications. The hypothesis serving as starting point for this work is that execution traces of both the system and the application to analyze can be used to automatically follow this application's execution, detect its anomalies and find their root causes. The results of this work are the concepts, methodologies and data structures used to follow and constrain applications, as well as the methods and algorithms allowing to detect and identify unexpected behaviors in those applications. These have been applied on actual real-time applications and succeeded in detecting and identifying the root causes of the irregularities in their runtime. Moreover, we were able to automatically and accurately build models for those applications, making it even easier for non-expert users to take advantage of tracing methods. The final result is that automatically detecting and pinpointing the origin of unwanted behaviors, in an application, is a valid and interesting option, making it faster and easier for users to analyze executions of their applications

Analyse des performances de stockage, en mémoire et sur les périphériques d'entrée/sortie, à partir d'une trace d'exécution

Le stockage des données est vital pour l’industrie informatique. Les supports de stockage doivent être rapides et fiables pour répondre aux demandes croissantes des entreprises. Les technologies de stockage peuvent être classifiées en deux catégories principales : stockage de masse et stockage en mémoire. Le stockage de masse permet de sauvegarder une grande quantité de données à long terme. Les données sont enregistrées localement sur des périphériques d’entrée/sortie, comme les disques durs (HDD) et les Solid-State Drive (SSD), ou en ligne sur des systèmes de stockage distribué. Le stockage en mémoire permet de garder temporairement les données nécessaires pour les programmes en cours d’exécution. La mémoire vive est caractérisée par sa rapidité d’accès, indispensable pour fournir rapidement les données à l’unité de calcul du processeur. Les systèmes d’exploitation utilisent plusieurs mécanismes pour gérer les périphériques de stockage, par exemple les ordonnanceurs de disque et les allocateurs de mémoire. Le temps de traitement d’une requête de stockage est affecté par l’interaction entre plusieurs soussystèmes, ce qui complique la tâche de débogage. Les outils existants, comme les outils d’étalonnage, permettent de donner une vague idée sur la performance globale du système, mais ne permettent pas d’identifier précisément les causes d’une mauvaise performance. L’analyse dynamique par trace d’exécution est très utile pour l’étude de performance des systèmes. Le traçage permet de collecter des données précises sur le fonctionnement du système, ce qui permet de détecter des problèmes de performance difficilement identifiables. L’objectif de cette thèse est de fournir un outil permettant d’analyser les performances de stockage, en mémoire et sur les périphériques d’entrée/sortie, en se basant sur les traces d’exécution. Les défis relevés par cet outil sont : collecter les données nécessaires à l’analyse depuis le noyau et les programmes en mode utilisateur, limiter le surcoût du traçage et la taille des traces générées, synchroniser les différentes traces, fournir des analyses multiniveau couvrant plusieurs aspects de la performance et enfin proposer des abstractions permettant aux utilisateurs de facilement comprendre les traces.----------ABSTRACT: Data storage is an essential resource for the computer industry. Storage devices must be fast and reliable to meet the growing demands of the data-driven economy. Storage technologies can be classified into two main categories: mass storage and main memory storage. Mass storage can store large amounts of data persistently. Data is saved locally on input/output devices, such as Hard Disk Drives (HDD) and Solid-State Drives (SSD), or remotely on distributed storage systems. Main memory storage temporarily holds the necessary data for running programs. Main memory is characterized by its high access speed, essential to quickly provide data to the Central Processing Unit (CPU). Operating systems use several mechanisms to manage storage devices, such as disk schedulers and memory allocators. The processing time of a storage request is affected by the interaction between several subsystems, which complicates the debugging task. Existing tools, such as benchmarking tools, provide a general idea of the overall system performance, but do not accurately identify the causes of poor performance. Dynamic analysis through execution tracing is a solution for the detailed runtime analysis of storage systems. Tracing collects precise data about the internal behavior of the system, which helps in detecting performance problems that are difficult to identify. The goal of this thesis is to provide a tool to analyze storage performance based on lowlevel trace events. The main challenges addressed by this tool are: collecting the required data using kernel and userspace tracing, limiting the overhead of tracing and the size of the generated traces, synchronizing the traces collected from different sources, providing multi-level analyses covering several aspects of storage performance, and lastly proposing abstractions allowing users to easily understand the traces. We carefully designed and inserted the instrumentation needed for the analyses. The tracepoints provide full visibility into the system and track the lifecycle of storage requests, from creation to processing. The Linux Trace Toolkit Next Generation (LTTng), a free and low-overhead tracer, is used for data collection. This tracer is characterized by its stability, and efficiency with highly parallel applications, thanks to the lock-free synchronization mechanisms used to update the content of the trace buffers. We also contributed to the creation of a patch that allows LTTng to capture the call stacks of userspace events

Multi-resource management in embedded real-time systems

This thesis addresses the problem of online multi-resource management in embedded real-time systems. It focuses on three research questions. The first question concentrates on how to design an efficient hierarchical scheduling framework for supporting independent development and analysis of component based systems, to provide temporal isolation between components. The second question investigates how to change the mapping of resources to tasks and components during run-time efficiently and predictably, and how to analyze the latency of such a system mode change in systems comprised of several scalable components. The third question deals with the scheduling and analysis of a set of parallel-tasks with real-time constraints which require simultaneous access to several different resources. For providing temporal isolation we chose a reservation-based approach. We first focused on processor reservations, where timed events play an important role. Common examples are task deadlines, periodic release of tasks, budget replenishment and budget depletion. Efficient timer management is therefore essential. We investigated the overheads in traditional timer management techniques and presented a mechanism called Relative Timed Event Queues (RELTEQ), which provides an expressive set of primitives at a low processor and memory overhead. We then leveraged RELTEQ to create an efficient, modular and extensible design for enhancing a real-time operating system with periodic tasks, polling, idling periodic and deferrable servers, and a two-level fixed-priority Hierarchical Scheduling Framework (HSF). The HSF design provides temporal isolation and supports independent development of components by separating the global and local scheduling, and allowing each server to define a dedicated scheduler. Furthermore, the design addresses the system overheads inherent to an HSF and prevents undesirable interference between components. It limits the interference of inactive servers on the system level by means of wakeup events and a combination of inactive server queues with a stopwatch queue. Our implementation is modular and requires only a few modifications of the underlying operating system. We then investigated scalable components operating in a memory-constrained system. We first showed how to reduce the memory requirements in a streaming multimedia application, based on a particular priority assignment of the different components along the processing chain. Then we investigated adapting the resource provisions to tasks during runtime, referred to as mode changes. We presented a novel mode change protocol called Swift Mode Changes, which relies on Fixed Priority with Deferred preemption Scheduling to reduce the mode change latency bound compared to existing protocols based on Fixed Priority Preemptive Scheduling. We then presented a new partitioned parallel-task scheduling algorithm called Parallel-SRP (PSRP), which generalizes MSRP for multiprocessors, and the corresponding schedulability analysis for the problem of multi-resource scheduling of parallel tasks with real-time constraints. We showed that the algorithm is deadlock-free, derived a maximum bound on blocking, and used this bound as a basis for a schedulability test. We then demonstrated how PSRP can exploit the inherent parallelism of a platform comprised of multiple heterogeneous resources. Finally, we presented Grasp, which is a visualization toolset aiming to provide insight into the behavior of complex real-time systems. Its flexible plugin infrastructure allows for easy extension with custom visualization and analysis techniques for automatic trace verification. Its capabilities include the visualization of hierarchical multiprocessor systems, including partitioned and global multiprocessor scheduling with migrating tasks and jobs, communication between jobs via shared memory and message passing, and hierarchical scheduling in combination with multiprocessor scheduling. For tracing distributed systems with asynchronous local clocks Grasp also supports the synchronization of traces from different processors during the visualization and analysis