Development of a framework for automated systematic testing of safety-critical embedded systems

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”In this paper we introduce the development of a framework for testing safety-critical embedded systems based on the concepts of model-based testing. In model-based testing the test cases are derived from a model of the system under test. In our approach the model is an automaton model that is automatically extracted from the C-source code of the system under test. Beside random test data generation the test case generation uses formal methods, in detail model checking techniques. To find appropriate test cases we use the requirements defined in the system specification. To cover further execution paths we developed an additional, to our best knowledge, novel method based on special structural coverage criteria. We present preliminary results on the model extraction using a concrete industrial case study from the automotive domain

A third generation personality test

The development of personality testing in the workplace has undergone three phases. The first generation of tests, such as Cattell’s 16 PF and the British test OPQ, was characterized by complex systems for the description of the personality. These systems were simplified in part by the following generation of the test, which was based on the five factor model but that model was simple only at the horizontal level. Beneath the five main factors were a large number of ancillary factors, usually 30-40 in number. No tests of the first and second generation could effectively handle the problem of impression management, nor did they take into account the effects of mood on the test responses. These and a number of other problems were solved to a great extent in the UPP test, which therefore is proposed to represent the third generation of personality tests. The test features focusing on “narrow” and work-relevant traits, inclusion of a few aggregated variables with the same focus, including two variables especially fitted to the requirement of any given application, an effective and validated method for correction for impression management, extensive treatment of quality of data from each tested person to yield a “warning signal” when results should not be trusted, measurement of current mood at the time of testing which can give another “warning signal”, measurement of attitude towards the test (“face validity”), two types of narrative reports both to the person taking the test and the recruiter/psychologist – one based on normative comparisons and the other on ipsative (within-person) comparisons, measures of work related attitudes which are of value in themselves but can also be used as proxy criteria, greatly facilitating validation work.personality test; impression management; mood; face vaility; data quality

Testing refinements by refining tests

One of the potential benefits of formal methods is that they offer the possibility of reducing the costs of testing. A specification acts as both the benchmark against which any implementation is tested, and also as the means by which tests are generated. There has therefore been interest in developing test generation techniques from formal specifications, and a number of different methods have been derived for state based languages such as Z, B and VDM. However, in addition to deriving tests from a formal specification, we might wish to refine the specification further before its implementation. The purpose of this paper is to explore the relationship between testing and refinement. As our model for test generation we use a DNF partition analysis for operations written in Z, which produces a number of disjoint test cases for each operation. In this paper we discuss how the partition analysis of an operation alters upon refinement, and we develop techniques that allow us to refine abstract tests in order to generate test cases for a refinement. To do so we use (and extend existing) methods for calculating the weakest data refinement of a specification

Automated Functional Testing based on the Navigation of Web Applications

Web applications are becoming more and more complex. Testing such applications is an intricate hard and time-consuming activity. Therefore, testing is often poorly performed or skipped by practitioners. Test automation can help to avoid this situation. Hence, this paper presents a novel approach to perform automated software testing for web applications based on its navigation. On the one hand, web navigation is the process of traversing a web application using a browser. On the other hand, functional requirements are actions that an application must do. Therefore, the evaluation of the correct navigation of web applications results in the assessment of the specified functional requirements. The proposed method to perform the automation is done in four levels: test case generation, test data derivation, test case execution, and test case reporting. This method is driven by three kinds of inputs: i) UML models; ii) Selenium scripts; iii) XML files. We have implemented our approach in an open-source testing framework named Automatic Testing Platform. The validation of this work has been carried out by means of a case study, in which the target is a real invoice management system developed using a model-driven approach.Comment: In Proceedings WWV 2011, arXiv:1108.208

Assessment of assimilating SMOS soil moisture information into a distributed hydrologic model

The role that soil moisture plays in terms of modulating hydrologic processes including infiltration and runoff generation makes it an essential component to capture for hydrologic modeling. This work aims to leverage information gained from SMOS to improve surface soil moisture simulations in the Russian River Basin (California, U.S.A). The basin's complex terrain offers a rigorous testing ground for SMOS soil moisture products. Data from seven in situ observation sites are used to assess model performance after assimilating SMOS-based soil saturation ratios. For a comparison of "best case" scenarios, the in situ observations themselves are assimilated. Results show that SMOS assimilated simulations shows modest improvement at most in situ locations. Despite the observed decrease in model performance at some locations, overall performance of simulations assimilated with SMOS-based saturation ratios remains high. Findings suggest that even in a complex environment, useful information may be extracted from SMOS estimates for hydrologic modeling

Automated metamorphic testing on the analyses of feature models

Copyright © 2010 Elsevier B.V. All rights reserved.Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem.Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach.Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively.Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency.Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.This work has been partially supported by the European Commission(FEDER)and Spanish Government under CICYT project SETI(TIN2009-07366)and the Andalusian Government project ISABEL(TIC-2533)