A cost-effective software testing strategy employing online feedback information

An online partitioning strategy is presented, in which test cases are selected based on feedback information collected during the testing process. The strategy differs from con- ventional approaches because the partitioning is performed online rather than off-line and because the partitioning is not based on program code or specifications. It can, therefore, be implemented in the absence of the source code or specification of the program under test. The cost-effectiveness of the proposed strategy has been empirically investigated with a set of subject programs, namely, SPACE, SED, GREP, and the Siemens Suite of Programs. The results demonstrate that the proposed strategy constantly achieves large savings in terms of the total number of test case executions needed to detect all faults

Integration and test strategies for complex manufacturing machines

Complex manufacturing machines, like ASML wafer scanners, consist of thousands of components like electronic boards, software, mechanical parts and optics. These components of multiple disciplines are assembled or integrated into modules. The modules are integrated into sub-systems forming the system, according to an integration plan. Components as well as modules, sub-systems, and systems, can be tested, diagnosed and ??xed, according to a test-diagnose- fix plan. An increase in the number of components results in an increase of the number of tasks in these plans. Moreover, the effort required to obtain a sequence that describes in which order the tasks should be executed also increases. The duration and the cost of a sequence depends on the quality of the system. In this project we introduce a method to analyze the duration and the cost of sequences of integration and test-diagnose-fix tasks. The method uses test-diagnose-fixed models to analyze the performance of sequences. The basic elements in such a model are: a) test, diagnose and fix tasks with their costs and durations, b) fault states, c) the coverage of test tasks on fault states, d) failure probabilities of fault states. These elements can be obtained for components, modules or sub-systems of multiple disciplines. Three case studies have been performed using this method. The outcome of the analysis indicates that choosing a di??erent test sequence can reduce the test duration by 30% to 70%. In addition, three techniques have been developed to improve integration and test-diagnose-fix sequences: ¿ To reduce the execution time of test-diagnose-fix sequences an algorithm has been developed to determine a new test task with an optimal coverage w.r.t. the fault states. The algorithm selects the new test task based on the maximum information gain. A test sequence, including the new test case, improves the test duration of the test-diagnose-fix task, because faults can be detected earlier. ¿ To reduce the execution time of test-diagnose-fix sequences an adapted hypergraph partitioning algorithm has been developed. The algorithm partitions a test-diagnose-??x task into smaller tasks which can be executed in parallel. The result of a case study is a reduction of the test duration by 30% with a concomitant increase of 30% in the test cost. ¿ The impact of the choice of the system architecture on the execution time and planning effort of integration and test-diagnose-fix sequences is investigate