Automata-theoretic and bounded model checking for linear temporal logic

In this work we study methods for model checking the temporal logic LTL. The focus is on the automata-theoretic approach to model checking and bounded model checking. We begin by examining automata-theoretic methods to model check LTL safety properties. The model checking problem can be reduced to checking whether the language of a finite state automaton on finite words is empty. We describe an efficient algorithm for generating small finite state automata for so called non-pathological safety properties. The presented implementation is the first tool able to decide whether a formula is non-pathological. The experimental results show that treating safety properties can benefit model checking at very little cost. In addition, we find supporting evidence for the view that minimising the automaton representing the property does not always lead to a small product state space. A deterministic property automaton can result in a smaller product state space even though it might have a larger number states. Next we investigate modular analysis. Modular analysis is a state space reduction method for modular Petri nets. The method can be used to construct a reduced state space called the synchronisation graph. We devise an on-the-fly automata-theoretic method for model checking the behaviour of a modular Petri net from the synchronisation graph. The solution is based on reducing the model checking problem to an instance of verification with testers. We analyse the tester verification problem and present an efficient on-the-fly algorithm, the first complete solution to tester verification problem, based on generalised nested depth-first search. We have also studied propositional encodings for bounded model checking LTL. A new simple linear sized encoding is developed and experimentally evaluated. The implementation in the NuSMV2 model checker is competitive with previously presented encodings. We show how to generalise the LTL encoding to a more succint logic: LTL with past operators. The generalised encoding compares favourably with previous encodings for LTL with past operators. Links between bounded model checking and the automata-theoretic approach are also explored.reviewe

Challenges in aligning requirements engineering and verification in a large-scale industrial context

[Context and motivation] When developing software, coordination between different organizational units is essential in order to develop a good quality product, on time and within budget. Particularly, the synchronization between requirements and verification processes is crucial in order to assure that the developed software product satisfies customer requirements. [Question/problem] Our research question is: what are the current challenges in aligning the requirements and verification processes? [Principal ideas/results] We conducted an interview study at a large software development company. This paper presents preliminary findings of these interviews that identify key challenges in aligning requirements and verification processes. [Contribution] The result of this study includes a range of challenges faced by the studied organization grouped into the categories: organization and processes, people, tools, requirements process, testing process, change management, traceability, and measurement. The findings of this study can be used by practitioners as a basis for investigating alignment in their organizations, and by scientists in developing approaches for more efficient and effective management of the alignment between requirements and verification.Comment: Requirements Engineering: Foundation for Software Quality: 16th International Working Conference, REFSQ 2010, Essen, Germany, June 30-July 2, 2010. Proceedings 16 (pp. 128-142). Springer Berlin Heidelber

Test Automation Case Study

Viimase kahe aasta jooksul on LHV panga ühe arendustiimi testimisprotsess läbinud mitu parendustsüklit. Samas pole seda arengut metoodiliselt valideeritud. Selle töö eesmärk on analüüsida kolme võtmetähtsusega perioodi ning mõõta nende testimisprotsessi maksumust ja efektiivsust. Seejärel viiakse läbi mitmetasandiline analüüs, et tuvastada problemaatilised ja kasulikud mustrid ning nendega seotud tegurid. Analüüsi tulemusel seatakse eesmärgiks muuta testimisprotsess automatiseeritumaks. Sellest tulenevalt tegeleb ülejäänud lõputöö erinevate meetodite kombineerimisega, et muuta selline lähenemine läbi testide automatiseerimise protsessi teatud osade endi automatiseerimise saavutatavaks. Nendest tehnikatest moodustatakse pakutav lahendus, mis seejärel implementeeritakse ja selle lõputöö kontekstis valideeritakse.Over the last two years, the testing process of one of the software development teams at LHV bank went through several development stages. However, there hasn't been any methodical approach towards validating that evolution. The aim of this thesis is to conduct an investigation of three key periods, and measure the cost and effectiveness of the testing process during each period. A multilevel analysis is then performed in order to identify problematic, as well as improvement patterns, and the factors associated with them. The analysis is concluded with setting the goal of shifting the testing process to a more automated model. Subsequently, the remainder of the thesis undertakes the task of combining a multiplicity of techniques that try to make such model achievable, by automating certain aspects of the test automation process itself. These techniques are articulated as a proposed solution, which is then implemented and validated in the context of this thesis

Flight control system design factors for applying automated testing techniques

Automated validation of flight-critical embedded systems is being done at ARC Dryden Flight Research Facility. The automated testing techniques are being used to perform closed-loop validation of man-rated flight control systems. The principal design features and operational experiences of the X-29 forward-swept-wing aircraft and F-18 High Alpha Research Vehicle (HARV) automated test systems are discussed. Operationally applying automated testing techniques has accentuated flight control system features that either help or hinder the application of these techniques. The paper also discusses flight control system features which foster the use of automated testing techniques

AFFTC overview of orbiter-reentry flight-test results

The Air Force Flight Test Center (AFFTC) has been participating in the flight testing of the Space Shuttle since 1976. An independent assessment of the reentry and landing capabilities of the Orbiter was conducted with respect to Department of Defense (DOD) missions. This activity is on-going and reports have been published after each flight. AFFTC participation in this conference is not directly related to the DOD assessment activity, however, and the views presented by myself and other AFFTC authors discuss the technical aspects of testing and the technology emanating from these tests

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Tailoring the Statistical Experimental Design Process for LVC Experiments

The use of Live, Virtual and Constructive (LVC) Simulation environments are increasingly being examined for potential analytical use particularly in test and evaluation. The LVC simulation environments provide a mechanism for conducting joint mission testing and system of systems testing when scale and resource limitations prevent the accumulation of the necessary density and diversity of assets required for these complex and comprehensive tests. The statistical experimental design process is re-examined for potential application to LVC experiments and several additional considerations are identified to augment the experimental design process for use with LVC. This augmented statistical experimental design process is demonstrated by a case study involving a series of tests on an experimental data link for strike aircraft using LVC simulation for the test environment. The goal of these tests is to assess the usefulness of information being presented to aircrew members via different datalink capabilities. The statistical experimental design process is used to structure the experiment leading to the discovery of faulty assumptions and planning mistakes that could potentially wreck the results of the experiment. Lastly, an aggressive sequential experimentation strategy is presented for LVC experiments when test resources are limited. This strategy depends on a foldover algorithm that we developed for nearly orthogonal arrays to rescue LVC experiments when important factor effects are confounded