Tracing Execution of Software for Design Coverage

Test suites are designed to validate the operation of a system against requirements. One important aspect of a test suite design is to ensure that system operation logic is tested completely. A test suite should drive a system through all abstract states to exercise all possible cases of its operation. This is a difficult task. Code coverage tools support test suite designers by providing the information about which parts of source code are covered during system execution. Unfortunately, code coverage tools produce only source code coverage information. For a test engineer it is often hard to understand what the noncovered parts of the source code do and how they relate to requirements. We propose a generic approach that provides design coverage of the executed software simplifying the development of new test suites. We demonstrate our approach on common design abstractions such as statecharts, activity diagrams, message sequence charts and structure diagrams. We implement the design coverage using Third Eye tracing and trace analysis framework. Using design coverage, test suites could be created faster by focussing on untested design elements.Comment: Short version of this paper to be published in Proceedings of 16th IEEE International Conference on Automated Software Engineering (ASE 2001). 13 pages, 9 figure

A generic operational memory model specification framework for multithreaded program verification

technical reportGiven the complicated nature of modern architectural and language level memory model designs, it is vital to have a systematic ap- proach for specifying memory consistency requirements that can support verification and promote understanding. In this paper, we develop a spec- ification methodology that defines a memory model operationally using a generic transition system with integrated model checking capability to enable formal reasoning about program correctness in a multithreaded environment. Based on a simple abstract machine, our system can be configured to define a variety of consistency models in a uniform nota- tion. We then apply this framework as a taxonomy to formalize several well known memory models. We also provide an alternative specification for the Java memory model based on a proposal from Manson and Pugh and demonstrate how to conduct computer aided analysis for Java thread semantics. Finally, we compare this operational approach with axiomatic approaches and discuss a method to convert a memory model definition from one style to the other

Mutating database queries

A set of mutation operators for SQL queries that retrieve information from a database is developed and tested against a set of queries drawn from the NIST SQL Conformance Test Suite. The mutation operators cover a wide spectrum of SQL features, including the handling of null values. Additional experiments are performed to explore whether the cost of executing mutants can be reduced using selective mutation or the test suite size can be reduced by using an appropriate ordering of the mutants. The SQL mutation approach can be helpful in assessing the adequacy of database test cases and their development, and as a tool for systematically injecting faults in order to compare different database testing technique

A Formal Approach to Practical Software Verification

Interpretation and Partition Refinement for Model Checking. Ph. D. Thesis, Eindhoven University of Technology, 1996. [DDR00] J. Katoen D. Distefano and A. Rensink. On a temporal logic for object-based systems. Proceedings of Formal Methods for Open Object-Based Distributed Systems, 2000. [DH99] M. Dwyer and J. Hatcliff. Slicing software for model construction. Proceedings of ACM SIGPLAN Partial Evaluation and Program Manipulation, January 1999. [DP98] M. Dwyer and C. Pasareanu. Filter-based model checking of partial systems. Software Engineering Notes, Volume 23, pp. 189 -- 202, November 1998. [DPD00] J. Skakkebaek D. Park, U. Stern and D. Dill. Java model checking. Proceedings of the First International Workshop on Automated Program Analysis, Testing and Verification, pp 74--82, June 2000. [Dwy98] Matthew Dwyer. Property pattern mappings for ltl . http://www.cis.ksu.edu/santos/spec-patterns/ltl.html, 1998. [EC98] G. Reggio E. Coscia. A proposal for a semantics of a subset of m..