Software reliability through fault-avoidance and fault-tolerance

Strategies and tools for the testing, risk assessment and risk control of dependable software-based systems were developed. Part of this project consists of studies to enable the transfer of technology to industry, for example the risk management techniques for safety-concious systems. Theoretical investigations of Boolean and Relational Operator (BRO) testing strategy were conducted for condition-based testing. The Basic Graph Generation and Analysis tool (BGG) was extended to fully incorporate several variants of the BRO metric. Single- and multi-phase risk, coverage and time-based models are being developed to provide additional theoretical and empirical basis for estimation of the reliability and availability of large, highly dependable software. A model for software process and risk management was developed. The use of cause-effect graphing for software specification and validation was investigated. Lastly, advanced software fault-tolerance models were studied to provide alternatives and improvements in situations where simple software fault-tolerance strategies break down

Constraints T-Way Testing Strategy With Modified Condition /Decision Coverage (MC/DC)

Modern society in today’s digital era depends heavily on software in almost every aspect of daily life. In fact, whenever possible, most hardware implementation is now being replaced by the software counterparts. From the washing machine controllers, mobile phone applications to the sophisticated airplane control systems, the growing dependency on software can be attributed to a number of factors. Unlike hardware, software does not wear out. Thus, the use of software can also help to control maintenance costs. Additionally, software is also malleable and can easily be changed and customized as the need arises. With the advent of advancement in computer hardware technology, software applications grow drastically in terms of lines of codes, that is, to keep up with ever increasing customer demands for new functionalities and innovations. As such, ensuring software quality can be a daunting task. Exhaustive testing is practically infeasible given the large domain of inputs and possibly too many possible execution paths. Over the years, many sampling techniques (or strategies) have been proposed to select subsets of test cases for testing consideration. In many applications, sampling strategies based on boundary value analysis, equivalence partioning, cause and effect analysis, and decision tables are sufficiently useful but they are not designed to address faults due to interaction. In other applications particularly involving structural (predicate) testing (e.g. in avionic industry), sampling strategies based on coverage criteria such as statements, decisions, and path coverage are deemed necessary, however, they often suffer from the effect of masking (i.e. due to the resulting AND and OR operations). Currently, researchers in combinatorial testing have already developed strategies based on interaction testing (termed t-way testing) in order to detect bugs due to interaction. Here, depending on the value of interaction strength (t), all desired t-way interactions are faithfuly covered in the resulting test cases.Although useful, much existing work t-way testing has not sufficiently considered modified conditions/decision coverage (MC/DC) as the criteria for test generation. In many critical applications particularly involving the airborne system, compliants to MC/DC are required by law [1]. Proposed by NASA in 1992, the MC/DC is a white box testing criterion ensuring each condition within a predicate can independently influence the outcome of the decision - while the outcome of all other conditions remains constant. In this manner, MC/DC criterion subsumes other well known coverage such as statements, decisions, and path [2]. Addressing some of the aforementioned issues, this research discusses the design of a new constraints based t-way strategy with MC/DC criterion for structural (predicate) testing. In doing so, this paper also highlights the possible implementations

Rule-based Test Generation with Mind Maps

This paper introduces basic concepts of rule based test generation with mind maps, and reports experiences learned from industrial application of this technique in the domain of smart card testing by Giesecke & Devrient GmbH over the last years. It describes the formalization of test selection criteria used by our test generator, our test generation architecture and test generation framework.Comment: In Proceedings MBT 2012, arXiv:1202.582

Functional Requirements-Based Automated Testing for Avionics

We propose and demonstrate a method for the reduction of testing effort in safety-critical software development using DO-178 guidance. We achieve this through the application of Bounded Model Checking (BMC) to formal low-level requirements, in order to generate tests automatically that are good enough to replace existing labor-intensive test writing procedures while maintaining independence from implementation artefacts. Given that existing manual processes are often empirical and subjective, we begin by formally defining a metric, which extends recognized best practice from code coverage analysis strategies to generate tests that adequately cover the requirements. We then formulate the automated test generation procedure and apply its prototype in case studies with industrial partners. In review, the method developed here is demonstrated to significantly reduce the human effort for the qualification of software products under DO-178 guidance

Constraint-Based Heuristic On-line Test Generation from Non-deterministic I/O EFSMs

We are investigating on-line model-based test generation from non-deterministic output-observable Input/Output Extended Finite State Machine (I/O EFSM) models of Systems Under Test (SUTs). We propose a novel constraint-based heuristic approach (Heuristic Reactive Planning Tester (xRPT)) for on-line conformance testing non-deterministic SUTs. An indicative feature of xRPT is the capability of making reasonable decisions for achieving the test goals in the on-line testing process by using the results of off-line bounded static reachability analysis based on the SUT model and test goal specification. We present xRPT in detail and make performance comparison with other existing search strategies and approaches on examples with varying complexity.Comment: In Proceedings MBT 2012, arXiv:1202.582