Achievements, open problems and challenges for search based software testing

Search Based Software Testing (SBST) formulates testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). We present an analysis of the SBST research agenda, focusing on the open problems and challenges of testing non-functional properties, in particular a topic we call 'Search Based Energy Testing' (SBET), Multi-objective SBST and SBST for Test Strategy Identification. We conclude with a vision of FIFIVERIFY tools, which would automatically find faults, fix them and verify the fixes. We explain why we think such FIFIVERIFY tools constitute an exciting challenge for the SBSE community that already could be within its reach

Evaluation of a Patient-Specific, Low-Cost, 3-Dimensional–Printed Transesophageal Echocardiography Human Heart Phantom

Simulation based education has been shown to increase the task-specific capability of medical trainees. Transesophageal echocardiography training greatly benefits from the use of simulators. They allow real time scanning of a beating heart and generation of ultrasound images side by side with anatomically accurate virtual model. These simulators are costly and have many limitations. 3D printing technologies have enabled the creation of bespoke phantoms capable of being used as task-trainers. This study aims to compare the ease of use and accuracy of a low-cost patient-specific, Computer-tomography based, 3D printed, echogenic TEE phantom compared to a commercially available echocardiography training mannequin. We hypothesized that a low-cost, 3D printed custom-made, cardiac phantom has comparable image quality, accuracy and usability as existing commercially available echocardiographic phantoms. After Institutional Ethic Research Board approval, we recruited ten American Board – Certified cardiac anesthesiologists and conducted a blinded comparative study divided into two stages. Stage one consisted of image assessment. A set of basic TEE views obtained from the 3D printed and commercial phantom were presented to the participants on a computer screen in random order. For each image, participants will be asked to identify the view, identify the quality of the image on a 1-5 Likert scale compared to the corresponding human view and guess with which phantom it was acquired (1 not at all realistic to patients view and 5 realistic to patients view). Stage two, participants will be asked to use the 3D printed and the commercially available phantom to obtain basic TEE views. In a maximum of 30 minutes. Each view was recorded and assessed for accuracy by two certified echocardiographers. Time needed to acquire each basic view and number of correct views was recorded. Overall usability of the phantoms was assessed through a questionnaire. For all continuous variables, we will calculate mean, median and standard deviation. We use Wilcoxon Signed-Rank test to assess significant differences in the rating of each phantom. All ten participants completed all part of the study. All participants could recognize all of the standard views. The average Likert scale was 3.2 for the 3D printed and 2.9 for the commercial Phantom with no significant difference. The average time to obtain views was 24.5 and 30 sec for the 3D printed and the commercial phantoms respectively statistically significantly in favor of the 3D printed phantom. The qualitative user assessment for ease to obtain the views, probe manipulation, image quality and overall experience were in great favor of the 3D printed phantom. Our Study suggest that the quality of TEE images obtained on the 3D printed phantom are not significantly different from those obtained on the commercial Phantom. The ease of use and time required to complete a basic TEE exam were in favor of the 3D Printed phantom.:Table of Content 1. Bibliographic Description 3 2. Introduction 4 2.1. Perioperative transesophageal echocardiography 4 2.2. Transesophageal echocardiography training 5 2.3. Transesophageal echocardiography simulation 6 2.4. 3D Heart Printing 13 2.5. 3D Segmentation 16 2.6. Development of the study phantom 17 2.7. Study Rationale 18 3. Publication 22 4. Summary 30 5. References 33 6. Appendices 37 6.1. Darstellung des eigenes Beitrags 38 6.2. Erklärung über die eigenständige Abfassung der Arbeit 39 6.3. Lebenslauf 40 6.4. Publikationen und Vorträge 44 6.5. Danksagung 61

Reflections on Software Failure Analysis

Failure studies are important in revealing the root causes, behaviors, and life cycle of defects in software systems. These studies either focus on understanding the characteristics of defects in specific classes of systems or the characteristics of a specific type of defect in the systems it manifests in. Failure studies have influenced various software engineering research directions, especially in the area of software evolution, defect detection, and program repair. In this paper, we reflect on the conduct of failure studies in software engineering. We reviewed a sample of 52 failure study papers. We identified several recurring problems in these studies, some of which hinder the ability of the engineering community to trust or replicate the results. Based on our findings, we suggest future research directions, including identifying and analyzing failure causal chains, standardizing the conduct of failure studies, and tool support for faster defect analysis

Information Retrieval and Spectrum Based Bug Localization: Better Together

Debugging often takes much effort and resources. To help developers debug, numerous information retrieval (IR)-based and spectrum-based bug localization techniques have been proposed. IR-based techniques process textual infor-mation in bug reports, while spectrum-based techniques pro-cess program spectra (i.e., a record of which program el-ements are executed for each test case). Both eventually generate a ranked list of program elements that are likely to contain the bug. However, these techniques only con-sider one source of information, either bug reports or pro-gram spectra, which is not optimal. To deal with the limita-tion of existing techniques, in this work, we propose a new multi-modal technique that considers both bug reports and program spectra to localize bugs. Our approach adaptively creates a bug-specific model to map a particular bug to its possible location, and introduces a novel idea of suspicious words that are highly associated to a bug. We evaluate our approach on 157 real bugs from four software systems, and compare it with a state-of-the-art IR-based bug localization method, a state-of-the-art spectrum-based bug localization method, and three state-of-the-art multi-modal feature loca-tion methods that are adapted for bug localization. Experi-ments show that our approach can outperform the baselines by at least 47.62%, 31.48%, 27.78%, and 28.80 % in terms of number of bugs successfully localized when a developer in

Automatic Verification of Message-Based Device Drivers

We develop a practical solution to the problem of automatic verification of the interface between device drivers and the OS. Our solution relies on a combination of improved driver architecture and verification tools. It supports drivers written in C and can be implemented in any existing OS, which sets it apart from previous proposals for verification-friendly drivers. Our Linux-based evaluation shows that this methodology amplifies the power of existing verification tools in detecting driver bugs, making it possible to verify properties beyond the reach of traditional techniques.Comment: In Proceedings SSV 2012, arXiv:1211.587

WCET of OCaml Bytecode on Microcontrollers: An Automated Method and Its Formalisation

Considering the bytecode representation of a program written in a high-level programming language enables portability of its execution as well as a factorisation of various possible analyses of this program. In this article, we present a method for computing the worst-case execution time (WCET) of an embedded bytecode program fit to run on a microcontroller. Due to the simple memory model of such a device, this automated WCET computation relies only on a control-flow analysis of the program, and can be adapted to multiple models of microcontrollers. This method evaluates the bytecode program using concrete as well as partially unknown values, in order to estimate its longest execution time. We present a software tool, based on this method, that computes the WCET of a synchronous embedded OCaml program. One key contribution of this article is a mechanically checked formalisation of the aforementioned method over an idealised bytecode language, as well as its proof of correctness

Using Constraint Programming to Verify DOPLER Variability Models

Software product lines are typically developed using model-based approaches. Models are used to guide and automate key activities such as the derivation of products. The verification of product line models is thus essential to ensure the consistency of the derived products. While many authors have proposed approaches for verifying feature models there is so far no such approach for decision models. We discuss challenges of analyzing and verifying decision-oriented DOPLER variability models. The manual verification of these models is an error-prone, tedious, and sometimes infeasible task. We present a preliminary approach that converts DOPLER variability models into constraint programs to support their verification. We assess the feasibility of our approach by identifying defects in two existing variability models