Exhaustive testing of safety critical Java

With traditional testing, the test case has no control over non-deterministic scheduling decisions, and thus errors dependent on scheduling are only found by pure chance. Java Path Finder (JPF) is a specialized Java virtual machine that can systematically explore execution paths for all possible schedulings, and thus catch these errors. Unfortunately, execution-based model checkers, including JPF, cannot be easily adapted to support real-time programs. We propose a scheduling algorithm for JPF which allows testing of Safety Critical Java (SCJ) applications with periodic event handlers at SCJ levels 0 and 1 (without aperiodic event handlers). The algorithm requires that deadlines are not missed and that there is an execution time model that can give best- and worst-case execution time estimates for a given program path and specific program inputs. Our implementation, named R SJ, allows to search for scheduling dependent memory access errors, certain invalid argument errors, priority ceiling emulation protocol violations, and failed assertions in application code in SCJ programs for levels 0 and 1. It uses the execution time model of the Java Optimized Processor (JOP). We test our tool wit

SCCharts: The Mindstorms Report

SCCharts are a visual language proposed in 2012 for specifying safety-critical reactive systems. This is the second SCCharts report towards the usability of the SCCharts visual language and its KIELER SCCharts implementation. KIELER is an open-source project which researches the pragmatics of model-based languages and related fields. Nine case-studies that were conducted between 2015 and 2019 evaluate the pros and cons in the context of small-scale Lego Mindstorms models and similar projects. Par-ticipants of the studies included undergraduate and graduate students from our local and also external facilities, as well as academics from the synchronous community. In the surveys, both the SCCharts language and the SCCharts tools are compared to other modeling and classical programming languages and tools

Software testing or the bugs’ nightmare

Software development is not error-free. For decades, bugs –including physical ones– have become a significant development problem requiring major maintenance efforts. Even in some cases, solving bugs led to increment them. One of the main reasons for bug’s prominence is their ability to hide. Finding them is difficult and costly in terms of time and resources. However, software testing made significant progress identifying them by using different strategies that combine knowledge from every single part of the program. This paper humbly reviews some different approaches from software testing that discover bugs automatically and presents some different state-of-the-art methods and tools currently used in this area. It covers three testing strategies: search-based methods, symbolic execution, and fuzzers. It also provides some income about the application of diversity in these areas, and common and future challenges on automatic test generation that still need to be addressed

Memoized Symbolic Execution

This paper introduces memoized symbolic execution (Memoise), a novel approach for more efficient application of forward symbolic execution, which is a well-studied technique for systematic exploration of program behaviors based on bounded execution paths. Our key insight is that application of symbolic execution often requires several successive runs of the technique on largely similar underlying problems, e.g., running it once to check a program to find a bug, fixing the bug, and running it again to check the modified program. Memoise introduces a trie-based data structure that stores the key elements of a run of symbolic execution. Maintenance of the trie during successive runs allows re-use of previously computed results of symbolic execution without the need for re-computing them as is traditionally done. Experiments using our prototype embodiment of Memoise show the benefits it holds in various standard scenarios of using symbolic execution, e.g., with iterative deepening of exploration depth, to perform regression analysis, or to enhance coverage

Program Model Checking: A Practitioner's Guide

Program model checking is a verification technology that uses state-space exploration to evaluate large numbers of potential program executions. Program model checking provides improved coverage over testing by systematically evaluating all possible test inputs and all possible interleavings of threads in a multithreaded system. Model-checking algorithms use several classes of optimizations to reduce the time and memory requirements for analysis, as well as heuristics for meaningful analysis of partial areas of the state space Our goal in this guidebook is to assemble, distill, and demonstrate emerging best practices for applying program model checking. We offer it as a starting point and introduction for those who want to apply model checking to software verification and validation. The guidebook will not discuss any specific tool in great detail, but we provide references for specific tools

Automatic Derivation of Requirements for Components Used in Human-Intensive Systems

Human-intensive systems (HISs), where humans must coordinate with each other along with software and/or hardware components to achieve system missions, are increasingly prevalent in safety-critical domains (e.g., healthcare). Such systems are often complex, involving aspects such as concurrency and exceptional situations. For these systems, it is often difficult but important to determine requirements for the individual components that are necessary to ensure the system requirements are satisfied. In this thesis, we investigated an approach that employs interface synthesis methods developed for software systems to automatically derive such requirements for components used in HISs. In previous work, we investigated a requirement deriver that employs a regular language learning algorithm to iteratively refine the derived requirement based on counterexamples generated by model checking techniques. Since this learning-based requirement deriver often did not scale well, we investigated several learning and model checking optimizations. These optimizations significantly improved performance but affected the counterexample generation heuristics, often widely varying the permissiveness of the derived requirements. For comparison purposes, we investigated a direct requirement deriver that was purported to have poor performance but guarantees the derived requirements are adequately permissive, conceptually meaning the requirements are permissive as possible without violating the system requirements. For our evaluation, we applied these requirement derivers to case studies in two important domains, healthcare and election administration. Based on this evaluation, the direct requirement deriver with all optimizations applied had reasonable performance and ensures the derived requirements are adequately permissive. For the learning-based requirement deriver, many of the optimizations and heuristics have been presented previously, but we recommend how to selectively combine them to obtain reasonable performance while usually producing the adequately permissive derived requirements. Since such derived requirements often reflect the system complexity, these requirements can be easily misunderstood. Thus, we also investigated building views of the requirements that abstract away or highlight certain aspects to try to improve their understandability. Each single view appears to improve understandability and the multiple views seem to complement each other further improving understandability. Such derived requirements and their views can be used to safely develop and deploy the components used in HISs

PET: A Partial Evaluation-based Test Case Generation Tool for Java Bytecode

PET is a prototype Partial Evaluation-based Test case generation tool for a subset of Java bytecode programs. It performs white-box test generation by means of two consecutive Partial Evaluations (PE). The first PE decompiles the Java bytecode program into an equivalent CLP (Constraint Logic Programming) counterpart. The second PE generates a test-case generator from the CLP program. This generator captures interesting test coverage criteria and it is able to generate further test cases on demand. For the first PE, PET incorporates an existing tool which decompiles bytecode to CLP. The main contribution of this work is the implementation of the second PE and the proof of concept of the approach. This has required the development of a partial evaluator for CLP with appropriate control strategies to ensure the required coverage criteria and to generate test-case generators. PET can be downloaded as free software from its web site, where a repository of examples and a web interface are also provided. Though PET has to be extended to be applicable to larger programs, we argue that it provides some evidence that the approach can be of practical interest

Checking Primitive Component Behavior

Software model checking je metoda ověřování vlastností programů a tedy zajišťování jejich vyšší spolehlivosti. Je však stále nutné dělit programy na části ověřované nezávisle, Nebot' úplné programy tvoří často příliš velký stavový prostor, který není možné prozkoumat v rozumném čase. Softwarové komponenty dělí aplikace na čáasti vhodným způsobem. Ale vzhledem k tomu, jak fungují model checkery, je nutné jim poskytnout vhodné prostředí pro ověření každé komponenty. Pro popis chování komponent vznikly behavior protokoly. Umožňují ověřovat kompatibilitu a shodu chování komponent - to se používá již při návrhu aplikací pro odhalení možných nevhodných použití komponent. Protokoly jsou také vhodné jako popis chování komponenty, který může být srovnán s chováním konkrétní implementace komponenty. Cílem této pr¶ace je poskytnout nástroj pro srovnání chování primitivní komponenty oproti její specifikaci. V rámci toho bude implementován generátor prostředí komponent podle frame protokolu, které umožní rozpoznat porušení specifikace chování. Bude použit Java PathFinder a Fractal component model.Software model checking is a process of checking for properties of a software application and thus assuring the software reliability. It is still necessary to divide the software application into pieces that are checked separately; the whole application yields an enormous state space that is impossible to traverse in a reasonable time. Therefore, the use of components is a straightforward approach for dividing the entire application. Furthermore, as model checker usually works with a close code only, a suitable component environment need to be provided for each component. Behavior protocols are a method for component behavior specification. They allow for checking for components' behavior compatibility and compliance, used at the design time of an application to find possible architectural component misplacements. They are also suitable to be compared with the behavior of the component implementation. The goal of the thesis is to provide a tool for comparing a primitive component behavior with its specification. Furthermore, a component environment generator using the component frame protocol will be implemented that would enable for checking for violation of the component behavior specification. The Java PathFinder model checker and the Fractal component model are to be used as a model checking platform.Department of Software EngineeringKatedra softwarového inženýrstvíMatematicko-fyzikální fakultaFaculty of Mathematics and Physic