Evolutionary testing supported by slicing and transformation

Evolutionary testing is a search based approach to the automated generation of systematic test data, in which the search is guided by the test data adequacy criterion. Two problems for evolutionary testing are the large size of the search space and structural impediments in the implementation of the program which inhibit the formulation of a suitable fitness function to guide the search. In this paper we claim that slicing can be used to narrow the search space and transformation can be applied to the problem of structural impediments. The paper presents examples of how these two techniques have been successfully employed to make evolutionary testing both more efficient and more effective

Signed

I declare that this thesis has not been, nor is currently being, submitted for the award of any other degree or similar qualification

EuroSTAR’99 Testing the Temporal Behavior of Real-Time Tasks using Extended Evolutionary Algorithms Abstract

Many industrial products are based on the use of embedded computer systems. Usually, these systems have to fulfill real-time requirements. Correct system functionality depends on their logical correctness as well as on their temporal correctness. Therefore, the developed systems have to be tested not only with regard to their functional behavior but they also have to be thoroughly tested in order to detect existing deficiencies in temporal behavior, as well as to strengthen the confidence in temporal correctness. Existing test methods are not specialized in the examination of temporal correctness. For this reason, new test methods are required which concentrate on determining whether the system violates its specified timing constraints. Within the past couple of years DaimlerChrysler Research and Technology has developed a new testing procedure for testing the temporal behavior of real-time systems: namely the evolutionary test. The test is interpreted as a problem of optimization, and employs evolutionary computation to find the test data with extreme execution times. Evolutionary operators used for the evolutionary test have been constantly improved and upgraded during this period of time. During the initial work phases test data generation was still based on genetic algorithms. Meanwhile, extended evolutionary algorithms have been established specifically as adequate operators for testing temporal behavior. This paper introduces approved extended evolutionary algorithms and illustrates their application for testing the temporal behavior of a bubblesort procedure. Further, an account will be given of the first industrial application of the evolutionary test during the development of a motor control system.

Using evolutionary testing to improve efficiency and quality in software testing

The development of embedded systems is an essential industrial activity. More than 90 % of all electronic components manufactured are used in embedded systems, e.g. in aerospace technology; railway and motor vehicle technology; process and automation technology; communication technology; process and power engineering, as well as in defense electronics. Embedded systems are also used regularly in safety relevant applications. Therefore, the occurrence of errors may endanger human lives or cause costly recalls, for example in the automotive industry. Accordingly, the development of embedded systems must comply with the highest quality requirements and standards. Analytical quality assurance is of central importance to achieving high quality development of embedded systems. In practice, the most important analytical quality assurance measure is dynamic testing. The thorough testing of developed systems is therefore essential for product quality. The aim of testing is to detect errors in the system under test and to convey confidence in the correct functioning of the system if no errors are found during comprehensive testing. The effectiveness and efficiency of the test process can be clearly improved by evolutionary testing. This has been successfully proved in several case studies and industrial applications. Evolutionary Tests thus contribute to quality improvement as well as to the reduction of development costs.- 1-

14th International Internet & Software Quality Week 2001 Evolutionary Testing of Embedded Systems

The development of embedded systems is an essential industrial activity. More than 90 % of all produced electronic components are used in embedded systems. Testing of embedded systems is considerably more complex than testing of conventional software systems. This is due on the one hand to the technical features of embedded systems, and on the other hand to the special requirements made on these kinds of systems: embedded systems usually have to fulfill functional as well as temporal requirements. Very often embedded systems are safety-relevant. In addition due to high costs resulting from errors occurring during the operation of embedded systems, high quality requirements apply. Dynamic testing is the most important method for testing such quality requirements. However, test case design is difficult to automate, therefore, most test cases have to be defined manually. A promising approach to automate test case design is the Evolutionary Test. It can be applied to testing the temporal behavior of systems, to structural testing as well as to safety testing. Effectiveness and efficiency of the test process can be clearly improved by Evolutionary Tests. This has been successfully proved in several case studies. Evolutionary Tests thus contribute to quality improvement as well as to the reduction of development costs.

EuroSTAR’96 Systematic Testing of Real-Time Systems

The development of embedded systems is an essential industrial activity whose importance is in-creasing. Commonly, embedded software systems have to fulfil real-time requirements. The most important analytical method to assure the quality of real-time systems is dynamic testing. Testing is the only method which examines the actual run-time behaviour of embedded software systems, based on an execution in the real application environment. Dynamic aspects like the duration of computations, the memory actually needed, or the synchronisation of parallel processes are of ma-jor importance for the correct function of real-time systems, and have to be tested. The most important prerequisite for a thorough test is the design of relevant test cases. The classifi-cation-tree method was developed to overcome the shortcomings of current functional testing prac-tice. It turns functional test case design into a process comprising several structured and system-atised parts- making it easy to handle, to understand, and also to document. Nevertheless, the classification-tree method alone is not sufficient for a comprehensive test of em-bedded systems. It is not well-suited for an examination of temporal correctness which is addition-ally essential to real-time systems. Already very small systems show a large variety of different exe