Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Ensuring behavioural equivalence in test-driven porting

In this paper we present a test-driven approach to porting code from one object-oriented language to another. We derive an order for the porting of the code, along with a testing strategy to verify the behaviour of the ported system at intra and inter-class level. We utilise the recently defined methodology for porting C++ applications, eXtreme porting, as a framework for porting. This defines a systematic routine based upon porting and unit-testing classes in turn. We augment this approach by using Object Relation Diagrams to define an order for porting that minimises class stubbing. Since our strategy is class-oriented and test-driven, we can ensure the structural equivalence of the ported system, along with the limited behavioural equivalence of each class. In order to extend this to integration-level equivalence, we exploit aspect-oriented programming to generate UML sequence diagrams, and we present a technique to compare such automatically-generated diagrams for equivalence. We demonstrate and evaluate our approach using a case study that involves porting an application from C++ to Java

Ensuring behavioural equivalence in test-driven porting

In this paper we present a test-driven approach to porting code from one object-oriented language to another. We derive an order for the porting of the code, along with a testing strategy to verify the behaviour of the ported system at intra and inter-class level. We utilise the recently defined methodology for porting C++ applications, eXtreme porting, as a framework for porting. This defines a systematic routine based upon porting and unit-testing classes in turn. We augment this approach by using Object Relation Diagrams to define an order for porting that minimises class stubbing. Since our strategy is class-oriented and test-driven, we can ensure the structural equivalence of the ported system, along with the limited behavioural equivalence of each class. In order to extend this to integration-level equivalence, we exploit aspect-oriented programming to generate UML sequence diagrams, and we present a technique to compare such automatically-generated diagrams for equivalence. We demonstrate and evaluate our approach using a case study that involves porting an application from C++ to Java

Time-Space Efficient Regression Testing for Configurable Systems

Configurable systems are those that can be adapted from a set of options. They are prevalent and testing them is important and challenging. Existing approaches for testing configurable systems are either unsound (i.e., they can miss fault-revealing configurations) or do not scale. This paper proposes EvoSPLat, a regression testing technique for configurable systems. EvoSPLat builds on our previously-developed technique, SPLat, which explores all dynamically reachable configurations from a test. EvoSPLat is tuned for two scenarios of use in regression testing: Regression Configuration Selection (RCS) and Regression Test Selection (RTS). EvoSPLat for RCS prunes configurations (not tests) that are not impacted by changes whereas EvoSPLat for RTS prunes tests (not configurations) which are not impacted by changes. Handling both scenarios in the context of evolution is important. Experimental results show that EvoSPLat is promising. We observed a substantial reduction in time (22%) and in the number of configurations (45%) for configurable Java programs. In a case study on a large real-world configurable system (GCC), EvoSPLat reduced 35% of the running time. Comparing EvoSPLat with sampling techniques, 2-wise was the most efficient technique, but it missed two bugs whereas EvoSPLat detected all bugs four times faster than 6-wise, on average.Comment: 14 page

Early aspects: aspect-oriented requirements engineering and architecture design

This paper reports on the third Early Aspects: Aspect-Oriented Requirements Engineering and Architecture Design Workshop, which has been held in Lancaster, UK, on March 21, 2004. The workshop included a presentation session and working sessions in which the particular topics on early aspects were discussed. The primary goal of the workshop was to focus on challenges to defining methodical software development processes for aspects from early on in the software life cycle and explore the potential of proposed methods and techniques to scale up to industrial applications

Towards quality programming in the automated testing of distributed applications

PhD ThesisSoftware testing is a very time-consuming and tedious activity and accounts for over 25% of the cost of software development. In addition to its high cost, manual testing is unpopular and often inconsistently executed. Software Testing Environments (STEs) overcome the deficiencies of manual testing through automating the test process and integrating testing tools to support a wide range of test capabilities. Most prior work on testing is in single-thread applications. This thesis is a contribution to testing of distributed applications, which has not been well explored. To address two crucial issues in testing, when to stop testing and how good the software is after testing, a statistics-based integrated test environment which is an extension of the testing concept in Quality Programming for distributed applications is presented. It provides automatic support for test execution by the Test Driver, test development by the SMAD Tree Editor and the Test Data Generator, test failure analysis by the Test Results Validator and the Test Paths Tracer, test measurement by the Quality Analyst, test management by the Test Manager and test planning by the Modeller. These tools are integrated around a public, shared data model describing the data entities and relationships which are manipulable by these tools. It enables early entry of the test process into the life cycle due to the definition of the quality planning and message-flow routings in the modelling. After well-prepared modelling and requirements specification are undertaken, the test process and the software design and implementation can proceed concurrently. A simple banking application written using Java Remote Method Invocation (RMI) and Java DataBase Connectivity (JDBC) shows the testing process of fitting it into the integrated test environment. The concept of the automated test execution through mobile agents across multiple platforms is also illustrated on this 3-tier client/server application.The National Science Council, Taiwan: The Ministry of National Defense, Taiwan

Software integration testing based on communication coverage criteria and partial model generation

This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send- and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences are derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences could suffer from a combinatorial explosion, the effort required to generate the partial model is polynomial in the number of test sequences and their length. Thus, where it is not feasible to produce test sequences that achieve a given type of coverage it is still possible to produce a partial model on the basis of test sequences generated to achieve some other criterion. As a result, the process of generating a partial model has the potential to scale to large industrial software systems. While a particular model checker, UPPAAL, was used, it should be relatively straightforward to adapt the approach for use with other CTL based model checkers. A potential additional benefit of the approach is that it provides a visual description of the state-based testing of distributed systems, which may be beneficial in other contexts such as education and comprehension

Proceedings of the 11th Overture Workshop

The 11th Overture Workshop was held in Aarhus, Denmark on Wed/Thu 28–29th Au- gust 2013. It was the 11th workshop in the current series focusing on the Vienna De- velopment Method (VDM) and particularly its community-based tools development project, Overture (http://www.overturetool.org/), and related projects such as COMPASS(http://www.compass-research.eu/) and DESTECS (http://www.destecs.org). Invited talks were given by Yves Ledru and Joe Kiniry. The workshop attracted 25 participants representing 10 nationalities. The goal of the workshop was to provide a forum to present new ideas, to identify and encourage new collaborative research, and to foster current strands of work towards publication in the mainstream conferences and journals. The Overture initiative held its first workshop at FM’05. Workshops were held subsequently at FM’06, FM’08 and FM’09, FM’11, FM’12 and in between