Formal Specification and Verification for Automated Production Systems

Complex industrial control software often drives safety- and mission-critical systems, like automated production plants or control units embedded into devices in automotive systems. Such controllers have in common that they are reactive systems, i.e., that they periodically read sensor stimuli and cyclically execute the same program to produce actuator signals. The correctness of software for automated production is rarely verified using formal techniques. Although, due to the Industrial Revolution 4.0 (IR4.0), the impact and importance of software have become an important role in industrial automation. What is used instead in industrial practice today is testing and simulation, where individual test cases are used to validate an automated production system. Three reasons why formal methods are not popular are: (a) It is difficult to adequately formulate the desired temporal properties. (b) There is a lack of specification languages for reactive systems that are both sufficiently expressive and comprehensible for practitioners. (c) Due to the lack of an environment model the obtained results are imprecise. Nonetheless, formal methods for automated production systems are well studied academically---mainly on the verification of safety properties via model checking. In this doctoral thesis we present the concept of (1) generalized test tables (GTTs), a new specification language for functional properties, and their extension (2) relational test tables (RTTs) for relational properties. The concept includes the syntactical notion, designed for the intuition of engineers, and the semantics, which are based on game theory. We use RTTs for a novel confidential property on reactive systems, the provably forgetting of information. Moreover, for regression verification, an important relational property, we are able to achieve performance improvements by (3) creating a decomposing rule which splits large proofs into small sub-task. We implemented the verification procedures and evaluated them against realistic case studies, e.g., the Pick-and-Place-Unit from the Technical University of Munich. The presented contribution follows the idea of lowering the obstacle of verifying the dependability of reactive systems in general, and automated production systems in particular for the engineer either by introducing a new specification language (GTTs), by exploiting existing programs for the specification (RTTs, regression verification), or by improving the verification performance

Development of a framework for automated systematic testing of safety-critical embedded systems

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”In this paper we introduce the development of a framework for testing safety-critical embedded systems based on the concepts of model-based testing. In model-based testing the test cases are derived from a model of the system under test. In our approach the model is an automaton model that is automatically extracted from the C-source code of the system under test. Beside random test data generation the test case generation uses formal methods, in detail model checking techniques. To find appropriate test cases we use the requirements defined in the system specification. To cover further execution paths we developed an additional, to our best knowledge, novel method based on special structural coverage criteria. We present preliminary results on the model extraction using a concrete industrial case study from the automotive domain

Incremental bounded model checking for embedded software

Program analysis is on the brink of mainstream usage in embedded systems development. Formal verification of behavioural requirements, finding runtime errors and test case generation are some of the most common applications of automated verification tools based on bounded model checking (BMC). Existing industrial tools for embedded software use an off-the-shelf bounded model checker and apply it iteratively to verify the program with an increasing number of unwindings. This approach unnecessarily wastes time repeating work that has already been done and fails to exploit the power of incremental SAT solving. This article reports on the extension of the software model checker CBMC to support incremental BMC and its successful integration with the industrial embedded software verification tool BTC EMBEDDED TESTER. We present an extensive evaluation over large industrial embedded programs, mainly from the automotive industry. We show that incremental BMC cuts runtimes by one order of magnitude in comparison to the standard non-incremental approach, enabling the application of formal verification to large and complex embedded software. We furthermore report promising results on analysing programs with arbitrary loop structure using incremental BMC, demonstrating its applicability and potential to verify general software beyond the embedded domain

Validation & Verification of an EDA automated synthesis tool

Reliability and correctness are two mandatory features for automated synthesis tools. To reach the goals several campaigns of Validation and Verification (V&V) are needed. The paper presents the extensive efforts set up to prove the correctness of a newly developed EDA automated synthesis tool. The target tool, MarciaTesta, is a multi-platform automatic generator of test programs for microprocessors' caches. Getting in input the selected March Test and some architectural details about the target cache memory, the tool automatically generates the assembly level program to be run as Software Based Self-Testing (SBST). The equivalence between the original March Test, the automatically generated Assembly program, and the intermediate C/C++ program have been proved resorting to sophisticated logging mechanisms. A set of proved libraries has been generated and extensively used during the tool development. A detailed analysis of the lessons learned is reporte

A Historical Perspective on Runtime Assertion Checking in Software Development

This report presents initial results in the area of software testing and analysis produced as part of the Software Engineering Impact Project. The report describes the historical development of runtime assertion checking, including a description of the origins of and significant features associated with assertion checking mechanisms, and initial findings about current industrial use. A future report will provide a more comprehensive assessment of development practice, for which we invite readers of this report to contribute information

FORTEST: Formal methods and testing

Formal methods have traditionally been used for specification and development of software. However there are potential benefits for the testing stage as well. The panel session associated with this paper explores the usefulness or otherwise of formal methods in various contexts for improving software testing. A number of different possibilities for the use of formal methods are explored and questions raised. The contributors are all members of the UK FORTEST Network on formal methods and testing. Although the authors generally believe that formal methods are useful in aiding the testing process, this paper is intended to provoke discussion. Dissenters are encouraged to put their views to the panel or individually to the authors