Test Data Generation for Exposing Interference Bugs in Multi-Threaded Systems

RÉSUMÉ Tester les systèmes multi-thread est difficile en raison du comportement non-déterministe de l'environnement (ordonnanceurs, cache, interruptions) dans lequel ils s'exécutent. Comme ils ne peuvent contrôler l'environnement, les testeurs doivent recourir à des moyens indirects pour augmenter le nombre d'ordonnancements testés. Une autre source de non-déterminisme pour les systèmes multi-thread est l'accès à la mémoire partagée. Lors de leur exécution, les systèmes multi-thread peuvent générer des conditions de courses aux données ou d'interférence dues aux données partagées. La génération de jeux de test en utilisant des techniques basées sur les recherches locales a déjà fourni des solutions aux problèmes des tests de systèmes mono et multi-thread. Mais il n'y a pas encore eu de travaux abordant la question des bugs d'interférence dans les systèmes multi-thread en utilisant les recherches locales. Dans cette thèse, nous étudions la possibilité d'utiliser ces approches afin de maximiser la possibilité d'exposer des bugs d'interférence dans les systèmes multi-thread. Nous formulons notre hypothèse de recherche comme suit : les techniques basées sur les recherches locales peuvent être utilisées efficacement pour générer des jeux de test pour maximiser les conditions d’obtention de bugs d'interférence dans les systèmes multi-thread. Après étude de la littérature, nous avons découvert trois défis majeurs concernant l'utilisation des approches basées sur les recherches locales : C1 : Formuler le problème initial comme un problème de recherche locale, C2 : Développer une fonction de coût adaptée, et C3 : Trouver la meilleure recherche locale (la plus adaptée). Nous procédons d'abord à une étude préliminaire sur la façon dont ces défis peuvent être relevés dans les systèmes mono-thread. Nous pensons que nos résultats pourraient être ensuite applicables aux systèmes multi-thread. Pour notre première étude, nous abordons le problème de la génération des jeux de test pour lever des exceptions de type division par zéro dans un système mono-thread en utilisant des approches basées sur les recherches locales, tout en tenant compte de C1, C2 et C3. Nous constatons que les trois défis sont importants et peuvent être traitées, et que les approches basées sur les recherches locales sont nettement plus efficaces qu'une approche aléatoire lorsque le nombre de paramètres d'entrées croît. Nous traitons ensuite notre principal problème de génération de jeux de test afin de révéler des bugs d'interférence dans les systèmes multi-thread en utilisant des approches basées sur les recherches locales, tout en répondant aux trois mêmes défis. Nous avons constaté que même dans les systèmes multi-thread, il est important de traiter les trois défis et que les approches basées sur les recherches locales surpassent une approche aléatoire lorsque le nombre de paramètres d'entrée devient grand. Nous validons ainsi notre thèse. Cependant, d'autres études sont nécessaires afin de généraliser nos résultats.----------ABSTRACT Testing multi-threaded systems is difficult due to the non-deterministic behaviour of the environment (schedulers, cache, interrupts) in which the multi-threaded system runs. As one cannot control the environment, testers must resort to indirect means to increase the number of schedules tested. Another source of non-determinism in multi-threaded systems is the shared memory access. When executed, multi-threaded systems can experience one of many possible interleavings of memory accesses to shared data, resulting in data race or interference conditions being raised. Test data generation using search-based techniques has provided solutions to the problem of testing single and multi-threaded systems over the years. But to the best of our knowledge, there has been no work addressing the issue of interference bugs in multi-threaded systems using search-based approaches. In this thesis, we perform a feasibility study of using search-based approaches to maximize the possibility of exposing interference bug pattern in multi-threaded systems. We frame our thesis hypothesis as: Search-based techniques can be used effectively to generate test data to expose interference condition in multi-threaded systems. From the related work we found out three major challenges of using search-based approaches: C1: Formulating the original problem as a search problem, C2: Developing the right fitness function for the problem formulation, and C3: Finding the right (scalable) search-based approach using scalability analysis. Before studying multi-threaded systems, we perform a preliminary study on how these challenges can be addressed in single-threaded systems, as we feel that our findings might be applicable to the more complex multi-threaded systems. In our first study, we address the problem of generating test data for raising divide-by-zero exception in single-threaded systems using search-based approaches, while addressing C1, C2, and C3. We find that the three challenges are important and can be addressed. Further, we found that search-based approaches scale significantly better than random when the input search space grows from very small to huge. Based on the knowledge obtained from the study, we address our main problem of generating test data to expose interference bugs in multi-threaded systems using search-based approaches, while addressing the same challenges C1, C2, and C3. We found that even in multi-threaded systems, it is important to address the three challenges and that search-based approaches outperforms random when the input search-space becomes large. Thus we confirm our thesis. However, further studies are necessary to generalize

ACRE: un générateur automatique d'aspect pour tester des logiciels écrits en C++

RÉSUMÉ La qualité d'un logiciel est une notion très importante de nos jours. Un des moyens de s'assurer de la bonne qualité d'un logiciel est de le tester afin d'en détecter, et de corriger, le plus grand nombre de bogues possible. L'approche proposée dans ce mémoire de maîtrise consiste à utiliser la programmation orientée aspect (AOP, pour Aspect Oriented Programming) afin de tester des programmes ecrit en C++. Elle consiste de plus en la création d'un langage dédie (DSL, pour Domain Specific Language) et d'un logiciel capable de le lire, ACRE (Automatic aspeCt cREator). Ces deux outils permettent la génération automatique d'aspects écrits en AspectC++, aspects permettant d'effectuer des tests de mémoire, d'invariants ou d’interférence dans des programmes écrits en C++. La programmation orientée aspect est un paradigme basé sur le principe de la séparation des préoccupations, qui veut que chaque module d'un programme s'occupe uniquement d'une préoccupation et ne s'implique pas dans les autres. L'AOP a déjà été utilisée afin de tester des logiciels, mais uniquement pour des programmes écrits en Java et pour des programmes embarques écrits en C++. De plus très peu de solutions ont été proposées afin d'automatiser la création des aspects et leur utilisation. Lorsque de telles solutions ont été proposées, elles l’étaient uniquement pour des programmes écrits en Java. Notre approche permet donc la génération automatique d'aspects pour des programmes écrits en C++. Ceci est fait grâce au DSL non-intrusif (dans le sens qu'il ne modifie pas le comportement du code source) introduit sous forme de commentaires dans le code source du programme à tester. L’écriture des lignes de DSL au plus près du code à tester est par ailleurs la seule modification requise à effectuer dans le code source du programme. Les aspects générés sont écrits dans des fichiers séparés et sont ensuite tisses au moment de la compilation avec le code source du programme. Notre approche permet ainsi aux développeurs de pouvoir tester un programme sans avoir à en modifier le comportement. Une première étude de cas pour notre approche est son application afin de tester NOMAD, logiciel d'optimisation de fonctions de type boite noire, utilisé tant en recherche qu'en industrie. Nous avons donc généré automatiquement des aspects permettant la découverte de bogues dans NOMAD. Un test de mémoire nous a permis de détecter une fuite mémoire importante, rapportée par un relevé de bogues d'un utilisateur, grâce à un aspect généré automatiquement. Une deuxième étude de cas nous a permis de rassurer l’équipe de développement de NOMAD. Nous avons en effet pu vérifier la validité d'un point mathématique crucial de l'algorithme implémenté. Ce test d'invariant a été là encore effectué grâce à un aspect généré automatiquement et sans modifier le comportement du code source de NOMAD. Une troisième étude de cas concerne la détection de bogues de type interférence dans des programmes parallèles Un bogue de type interférence se produit lorsque qu'au moins deux fils d’exécution d'un programme tentent d’écrire en même temps dans une même variable. Ce type de bogue fait partie des bogues affectant les programmes parallèles les plus difficiles à détecter et peu de logiciels actuels permettent de les découvrir à coup sûr. Notre approche permet l'obtention, grâce à des aspects, des temps d’accès en lecture et en écriture d'attributs du programme, ces temps étant ensuite utilisés par une approche mathématique qui aide les logiciels actuels dans leur détection de ce type de bogue. Il y a cependant des limites à l'utilisation de notre approche, notamment dues à des limitations inhérentes à AspectC++, limitations dont nous discutons.----------ABSTRACT In the modern era, software quality is very important. In order to have high software quality, we can test it to find and remove the maximum number of bugs. Our approach uses Aspect-Oriented Programming (AOP) to test software written in C++. We propose a Domain Specific Language (DSL) and ACRE, an Automatic aspeCt cREator that can read our DSL. From these two tools we can automatically generate aspects written in AspectC++ to perform memory, invariant and interference testing for software written in C++. Aspect-Oriented Programming is a paradigm based on the concept of separation of concerns. In this concept, each module deals with one and only one concern, and does not deal with the concerns of other modules. AOP has already been used in testing, but only for software written in Java or embedded software written in C++. Few of these approaches allow an automatic generation of aspects, and to date, it was only to test software written in Java. Our approach allows automatic aspect generation for software written in C++ thanks to our non-intrusive DSL (in the sense that it does not change the behavior or the source code). Our DSL is inserted in the source code of the tested software as comments and this is the only modification made in the source code. Generated aspects are then written to dedicated files and woven within the source code at compile time. Our approach allows developers to test software without having to modify its behavior. Our first case study is to apply our approach with the intent of testing NOMAD, a black box optimization software used both in research and industry. Using automatically generated aspects, we are able to do memory testing in NOMAD and locate an important memory leak mentioned in a bug report. Our second case study is to do invariant testing in NOMAD. We are able, still using a generated aspect, to verify a crucial mathematical point of the implemented algorithm without modifying the behavior of the source code of NOMAD. The third case study concerns interference testing in multi-threaded programs. An interference bug occurs when at least two threads try to write to the same shared variable at the same time. This kind of bug is one of the most difficult to detect and no software is able to them with 100% accuracy. Our approach is used to get both read and write access times for the attribute of the tested software using aspects. These times are then used in a mathematical approach that helps the actual software find interference bug patterns. However there are some limitations to our approach, due to inherent limitations of AspectC++. These limitations are discussed in this thesis

Productive Programming Systems for Heterogeneous Supercomputers

The majority of today's scientific and data analytics workloads are still run on relatively energy inefficient, heavyweight, general-purpose processing cores, often referred to in the literature as latency-oriented architectures. The flexibility of these architectures and the programmer aids included (e.g. large and deep cache hierarchies, branch prediction logic, pre-fetch logic) makes them flexible enough to run a wide range of applications fast. However, we have started to see growth in the use of lightweight, simpler, energy-efficient, and functionally constrained cores. These architectures are commonly referred to as throughput-oriented. Within each shared memory node, the computational backbone of future throughput-oriented HPC machines will consist of large pools of lightweight cores. The first wave of throughput-oriented computing came in the mid 2000's with the use of GPUs for general-purpose and scientific computing. Today we are entering the second wave of throughput-oriented computing, with the introduction of NVIDIA Pascal GPUs, Intel Knights Landing Xeon Phi processors, the Epiphany Co-Processor, the Sunway MPP, and other throughput-oriented architectures that enable pre-exascale computing. However, while the majority of the FLOPS in designs for future HPC systems come from throughput-oriented architectures, they are still commonly paired with latency-oriented cores which handle management functions and lightweight/un-parallelizable computational kernels. Hence, most future HPC machines will be heterogeneous in their processing cores. However, the heterogeneity of future machines will not be limited to the processing elements. Indeed, heterogeneity will also exist in the storage, networking, memory, and software stacks of future supercomputers. As a result, it will be necessary to combine many different programming models and libraries in a single application. How to do so in a programmable and well-performing manner is an open research question. This thesis addresses this question using two approaches. First, we explore using managed runtimes on HPC platforms. As a result of their high-level programming models, these managed runtimes have a long history of supporting data analytics workloads on commodity hardware, but often come with overheads which make them less common in the HPC domain. Managed runtimes are also not supported natively on throughput-oriented architectures. Second, we explore the use of a modular programming model and work-stealing runtime to compose the programming and scheduling of multiple third-party HPC libraries. This approach leverages existing investment in HPC libraries, unifies the scheduling of work on a platform, and is designed to quickly support new programming model and runtime extensions. In support of these two approaches, this thesis also makes novel contributions in tooling for future supercomputers. We demonstrate the value of checkpoints as a software development tool on current and future HPC machines, and present novel techniques in performance prediction across heterogeneous cores

Compiler-managed memory system for software-exposed architectures

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2000.Includes bibliographical references (p. 155-161).Microprocessors must exploit both instruction-level parallelism (ILP) and memory parallelism for high performance. Sophisticated techniques for ILP have boosted the ability of modern-day microprocessors to exploit ILP when available. Unfortunately, improvements in memory parallelism in microprocessors have lagged behind. This thesis explains why memory parallelism is hard to exploit in microprocessors and advocate bank-exposed architectures as an effective way to exploit more memory parallelism. Bank exposed architectures are a kind of software-exposed architecture: one in which the low level details of the hardware are visible to the software. In a bank-exposed architecture, the memory banks are visible to the software, enabling the compiler to exploit a high degree of memory parallelism in addition to ILP. Bank-exposed architectures can be employed by general-purpose processors, and by embedded chips, such as those used for digital-signal processing. This thesis presents Maps, an enabling compiler technology for bank-exposed architectures. Maps solves the problem of bank-disambiguation, i.e., how to distribute data in sequential programs among several banks to best exploit memory parallelism, while retaining the ability to disambiguate each data reference to a particular bank. Two methods for bank disambiguation are presented: equivalence-class unification and modulo unrolling. Taking a sequential program as input, a bank-disambiguation method produces two outputs: first, a distribution of each program object among the memory banks; and second, a bank number for every reference that can be proven to access a single, known bank for that data distribution. Finally, the thesis shows why non-disambiguated accesses are sometimes desirable. Dependences between disambiguated and non-disambiguated accesses are enforced through explicit synchronization and software serial ordering. The MIT Raw machine is an example of a software-exposed architecture. Raw exposes its ILP, memory and communication mechanisms. The Maps system has been implemented in the Raw compiler. Results on Raw using sequential codes demonstrate that using bank disambiguation in addition to ILP improves performance by a factor of 3 to 5 over using ILP alone.by Rajeev Barua.Ph.D

Optimizing Threads Schedule Alignments to Expose the Interference Bug Pattern

Abstract. Managing and controlling interference conditions in multi-threaded programs has been an issue of worry for application developers for a long time. Typically, when write events from two concurrent threads to the same shared variable are not properly protected, an occurrence of the interference bug pattern could be exposed. We propose a mathematical formulation and its resolution to maximize the possibility of exposing occurrences of the interference bug pattern. We formulate and solve the issue as an optimization problem that gives us (1) the optimal position to inject a delay in the execution flow of a thread and (2) the optimal duration for this delay to align at least two different write events in a multi-threaded program. To run the injected threads and calculate the thread execution times for validating the results, we use a virtual platform modelling a perfectly parallel system. All the effects due to the operating system’s scheduler or the latencies of hardware components are reduced to zero, exposing only the interactions between threads. To the best of our knowledge, no previous work has formalized the alignment of memory access events to expose occurrences of th

Programming Languages and Systems

This open access book constitutes the proceedings of the 31st European Symposium on Programming, ESOP 2022, which was held during April 5-7, 2022, in Munich, Germany, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022. The 21 regular papers presented in this volume were carefully reviewed and selected from 64 submissions. They deal with fundamental issues in the specification, design, analysis, and implementation of programming languages and systems

Evolutionary genomics : statistical and computational methods

This open access book addresses the challenge of analyzing and understanding the evolutionary dynamics of complex biological systems at the genomic level, and elaborates on some promising strategies that would bring us closer to uncovering of the vital relationships between genotype and phenotype. After a few educational primers, the book continues with sections on sequence homology and alignment, phylogenetic methods to study genome evolution, methodologies for evaluating selective pressures on genomic sequences as well as genomic evolution in light of protein domain architecture and transposable elements, population genomics and other omics, and discussions of current bottlenecks in handling and analyzing genomic data. Written for the highly successful Methods in Molecular Biology series, chapters include the kind of detail and expert implementation advice that lead to the best results. Authoritative and comprehensive, Evolutionary Genomics: Statistical and Computational Methods, Second Edition aims to serve both novices in biology with strong statistics and computational skills, and molecular biologists with a good grasp of standard mathematical concepts, in moving this important field of study forward

Programming Languages and Systems

This open access book constitutes the proceedings of the 31st European Symposium on Programming, ESOP 2022, which was held during April 5-7, 2022, in Munich, Germany, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022. The 21 regular papers presented in this volume were carefully reviewed and selected from 64 submissions. They deal with fundamental issues in the specification, design, analysis, and implementation of programming languages and systems

The 31st Aerospace Mechanisms Symposium

The proceedings of the 31st Aerospace Mechanisms Symposium are reported. Topics covered include: robotics, deployment mechanisms, bearings, actuators, scanners, boom and antenna release, and test equipment. A major focus is the reporting of problems and solutions associated with the development and flight certification of new mechanisms