1,438 research outputs found
Recommended from our members
Using formal methods to support testing
Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent
years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing
Complete Model-Based Testing Applied to the Railway Domain
Testing is the most important verification technique to assert the correctness of an embedded system. Model-based testing (MBT) is a popular approach that generates test cases from models automatically. For the verification of safety-critical systems, complete MBT strategies are most promising. Complete testing strategies can guarantee that all errors of a certain kind are revealed by the generated test suite, given that the system-under-test fulfils several hypotheses. This work presents a complete testing strategy which is based on equivalence class abstraction. Using this approach, reactive systems, with a potentially infinite input domain but finitely many internal states, can be abstracted to finite-state machines. This allows for the generation of finite test suites providing completeness. However, for a system-under-test, it is hard to prove the validity of the hypotheses which justify the completeness of the applied testing strategy. Therefore, we experimentally evaluate the fault-detection capabilities of our equivalence class testing strategy in this work. We use a novel mutation-analysis strategy which introduces artificial errors to a SystemC model to mimic typical HW/SW integration errors. We provide experimental results that show the adequacy of our approach considering case studies from the railway domain (i.e., a speed-monitoring function and an interlocking-system controller) and from the automotive domain (i.e., an airbag controller). Furthermore, we present extensions to the equivalence class testing strategy. We show that a combination with randomisation and boundary-value selection is able to significantly increase the probability to detect HW/SW integration errors
TOOL-ASSISTED VALIDATION AND VERIFICATION TECHNIQUES FOR STATE-BASED FORMAL METHODS
To tackle the growing complexity of developing modern software systems that usually have embedded and distributed nature, and more and more involve safety critical aspects, formal methods (FMs) have been affirmed as an efficient approach to ensure the quality and correctness of the design, that permits to discover errors yet at the early stages of the system development. Among the several FMs available, some of them can be described as state-based, since they describe systems by using the notions of state and transitions between states. State-based FMs are sometimes preferred since they produce specifications that are more intuitive, being the notions of state and transition close to the notions of program state and program execution that are familiar to any developer. Moreover, state-based FMs are usually executable and permit to be simulated, so having an abstraction of the execution of the system under development.
The aim of the thesis is to provide tool-assisted techniques that help the adoption of state-based FMs. In particular we address four main goals:
1) identifying a process for the development of an integrated framework around a formal method. The adoption of a formal method is often prevented by the lack of tools to support the user in the different development activities, as model editing, validation, verification, etc. Moreover, also when tools are available, they have usually been developed to target only one aspect of the system development process. So, having a well-engineered process that helps in the development of concrete notations and tools for a FM can make FMs of practical application.
2) promoting the integration of different FMs. Indeed, having only one formal notation, for doing different formal activities during the development of the system, is preferable than having a different notation for each formal activity. Moreover such notation should be high-level: working with high level notations is definitely easier than working with low-level ones, and the produced specifications are usually more readable. This goal can be seen as a sub-goal of the first goal; indeed, in a framework around a formal method, it should also be possible to integrate other formal methods that better address some particular formal activities.
3) helping the user in writing correct specifications. The basic assumption of any formal technique is that the specification, representing the desired properties of the system or the model of the system, is correct. However, in case the specification is not correct, all the verification activities based on the specification produce results that are meaningless. So, validation techniques should assure that the specification reflects the intended requirements; besides traditional simulation (user-guided or scenario-based), also model review techniques, checking for common quality attributes that any specification should have, are a viable solution.
4) reducing the distance between the formal specification and the actual implementation of the system. Several FMs work on a formal description of the system which is assumed to reflect the actual implementation; however, in practice, the formal specification and the actual implementation could be not conformant. A solution is to obtain the implementation, through refinements steps, from the formal specification, and proving that the refinements steps are correct. A different viable solution is to link the implementation with its formal specification and check, during the program execution, if they are conformant
Recommended from our members
Software engineering: Testing real-time embedded systems using timed automata based approaches
This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Real-time Embedded Systems (RTESs) have an increasing role in controlling society infrastructures that we use on a day-to-day basis. RTES behaviour is not based solely on the interactions it might have with its surrounding environment, but also on the timing requirements it induces. As a result, ensuring that an RTES behaves correctly is non-trivial, especially after adding time as a new dimension to the complexity of the testing process. This research addresses the problem of testing RTESs from Timed Automata (TA) specification by the following. First, a new Priority-based Approach (PA) for testing RTES modelled formally as UPPAAL timed automata (TA variant) is introduced. Test cases generated according to a proposed timed adequacy criterion (clock region coverage) are divided into three sets of priorities, namely boundary, out-boundary and in-boundary. The selection of which set is most appropriate for a System Under Test (SUT) can be decided by the tester according to the system type, time specified for the testing process and its budget. Second, PA is validated in comparison with four well-known timed testing approaches based on TA using Specification Mutation Analysis (SMA). To enable the validation, a set of timed and functional mutation operators based on TA is introduced. Three case studies are used to run SMA. The effectiveness of timed testing approaches are determined and contrasted according to the mutation score which shows that our PA achieves high mutation adequacy score compared with others. Third, to enhance the applicability of PA, a new testing tool (GeTeX) that deploys PA is introduced. In its current version, GeTeX supports Control Area Network (CAN) applications. GeTeX is validated by developing a prototype for that purpose. Using GeTeX, PA is also empirically validated in comparison with some TA testing approaches using a complete industrial-strength test bed. The assessment is based on fault coverage, structural coverage, the length of generated test cases and a proposed assessment factor. The assessment is based on fault coverage, structural coverage, the length of generated test cases and a proposed assessment factor. The assessment results confirmed the superiority of PA over the other test approaches. The overall assessment factor showed that structural and fault coverage scores of PA with respect to the length of its tests were better than the others proving the applicability of PA. Finally, an Analytical Hierarchy Process (AHP) decision-making framework for our PA is developed. The framework can provide testers with a systematic approach by which they can prioritise the available PA test sets that best fulfils their testing requirements. The AHP framework developed is based on the data collected heuristically from the test bed and data collected by interviewing testing experts. The framework is then validated using two testing scenarios. The decision outcomes of the AHP framework were significantly correlated to those of testing experts which demonstrated the soundness and validity of the framework.This study is funded by Damascus University, Syri
Recommended from our members
Improving fault coverage and minimising the cost of fault identification when testing from finite state machines
This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Software needs to be adequately tested in order to increase the confidence that the system being developed is reliable. However, testing is a complicated and expensive process. Formal specification based models such as finite state machines have been widely used in system modelling and testing. In this PhD thesis, we primarily investigate fault detection and identification when testing from finite state machines. The research in this thesis is mainly comprised of three topics - construction of multiple Unique Input/Output (UIO) sequences using Metaheuristic Optimisation Techniques (MOTs), the improved fault
coverage by using robust Unique Input/Output Circuit (UIOC) sequences, and fault diagnosis when testing from finite state machines. In the studies of the construction of UIOs, a model is proposed where a fitness function is defined to guide the search for input sequences that are potentially UIOs. In the studies of the improved fault coverage, a new type of UIOCs is defined. Based upon the Rural Chinese Postman Algorithm (RCPA), a new approach is proposed for the construction of more robust test sequences. In the studies of fault diagnosis, heuristics are defined that attempt to lead to failures being observed in some shorter test sequences, which helps to reduce the
cost of fault isolation and identification. The proposed approaches and techniques were evaluated with regard to a set of case studies, which provides experimental evidence for their efficacy.Brunel Research Initiative and Enterprise Fund (BRIEF) Award from Brunel University and Departmental bursary from Department of Information Systems and Computing, Brunel Universit
SAVCBS 2004 Specification and Verification of Component-Based Systems: Workshop Proceedings
This is the proceedings of the 2004 SAVCBS workshop. The workshop is concerned with how formal (i.e., mathematical) techniques can be or should be used to establish a suitable foundation for the specification and verification of component-based systems. Component-based systems are a growing concern for the software engineering community. Specification and reasoning techniques are urgently needed to permit composition of systems from components. Component-based specification and verification is also vital for scaling advanced verification techniques such as extended static analysis and model checking to the size of real systems. The workshop considers formalization of both functional and non-functional behavior, such as performance or reliability
Model Checking and Model-Based Testing : Improving Their Feasibility by Lazy Techniques, Parallelization, and Other Optimizations
This thesis focuses on the lightweight formal method of model-based testing for checking safety properties, and derives a new and more feasible approach.
For liveness properties, dynamic testing is impossible, so feasibility is increased by specializing on an important class of properties, livelock freedom, and deriving a more feasible model checking algorithm for it.
All mentioned improvements are substantiated by experiments
- …