JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction

Popular software testing tools, such as JUnit, allow frequent retesting of modified code; yet the manually created test scripts are often seriously incomplete. A unit-testing tool called JWalk has therefore been developed to address the need for systematic unit testing within the context of agile methods. The tool operates directly on the compiled code for Java classes and uses a new lazy method for inducing the changing design of a class on the fly. This is achieved partly through introspection, using Java’s reflection capability, and partly through interaction with the user, constructing and saving test oracles on the fly. Predictive rules reduce the number of oracle values that must be confirmed by the tester. Without human intervention, JWalk performs bounded exhaustive exploration of the class’s method protocols and may be directed to explore the space of algebraic constructions, or the intended design state-space of the tested class. With some human interaction, JWalk performs up to the equivalent of fully automated state-based testing, from a specification that was acquired incrementally

A Survey of Languages for Specifying Dynamics: A Knowledge Engineering Perspective

A number of formal specification languages for knowledge-based systems has been developed. Characteristics for knowledge-based systems are a complex knowledge base and an inference engine which uses this knowledge to solve a given problem. Specification languages for knowledge-based systems have to cover both aspects. They have to provide the means to specify a complex and large amount of knowledge and they have to provide the means to specify the dynamic reasoning behavior of a knowledge-based system. We focus on the second aspect. For this purpose, we survey existing approaches for specifying dynamic behavior in related areas of research. In fact, we have taken approaches for the specification of information systems (Language for Conceptual Modeling and TROLL), approaches for the specification of database updates and logic programming (Transaction Logic and Dynamic Database Logic) and the generic specification framework of abstract state machine

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Testing data types implementations from algebraic specifications

Algebraic specifications of data types provide a natural basis for testing data types implementations. In this framework, the conformance relation is based on the satisfaction of axioms. This makes it possible to formally state the fundamental concepts of testing: exhaustive test set, testability hypotheses, oracle. Various criteria for selecting finite test sets have been proposed. They depend on the form of the axioms, and on the possibilities of observation of the implementation under test. This last point is related to the well-known oracle problem. As the main interest of algebraic specifications is data type abstraction, testing a concrete implementation raises the issue of the gap between the abstract description and the concrete representation. The observational semantics of algebraic specifications bring solutions on the basis of the so-called observable contexts. After a description of testing methods based on algebraic specifications, the chapter gives a brief presentation of some tools and case studies, and presents some applications to other formal methods involving datatypes

Testing object-oriented industrial software without precise oracles or results

Software testing such as object-oriented software Testing At the Class and Cluster LEvels, or TACCLE, can be achieved by defining the test objectives, selecting and executing test cases, and checking results. The software specifications are extracted from technical drawings of mechanical and electronic hardware designed by process engineers using an in-house technique. TACCLE enables software engineers to test each individual class independently, then test the interaction among classes. The TACCLE methodology is described in three sections that include class-level testing, cluster-level testing, and test case generation and test script translation. The notion of testing observational equivalence and bypassing the need for oracles is important in test automation for industrial projects where it is impractical to define a precise relationship between specification and the software under test.postprin

Equality to equals and unequals: a revisit of the equivalence and nonequivalence criteria in object-oriented software testing

published_or_final_versio

The construction of oracles for software testing

Software testing is important throughout the software life cycle. Testing is the part of the software development process where a computer program is subject to specific conditions to show that the problem meets its intended design. Building a testing oracle is one part of software testing. An oracle is an external mechanism which can be used to check test output for correctness. The characteristics of available oracles have a dominating influence on the cost and quality of software testing. In this thesis, methods of constructing oracles are investigated and classified. There are three kinds of method of constructing oracles: the pseudo-oracle approach, oracles using attributed grammars and oracles based on formal specification. This thesis develops a method for constructing an oracle, based on the Z specification language. A specification language can describe the correct syntax and semantics of software. The contextual part of a specification describes all the legal input to the program and the semantics part describes the meaning of the given input data. Based on this idea, an oracle is constructed and a prototype is implemented according to the method proposed in the thesis

The Oracle Problem in Software Testing: A Survey

Testing involves examining the behaviour of a system in order to discover potential faults. Given an input for a system, the challenge of distinguishing the corresponding desired, correct behaviour from potentially incorrect behavior is called the “test oracle problem”. Test oracle automation is important to remove a current bottleneck that inhibits greater overall test automation. Without test oracle automation, the human has to determine whether observed behaviour is correct. The literature on test oracles has introduced techniques for oracle automation, including modelling, specifications, contract-driven development and metamorphic testing. When none of these is completely adequate, the final source of test oracle information remains the human, who may be aware of informal specifications, expectations, norms and domain specific information that provide informal oracle guidance. All forms of test oracles, even the humble human, involve challenges of reducing cost and increasing benefit. This paper provides a comprehensive survey of current approaches to the test oracle problem and an analysis of trends in this important area of software testing research and practice

Visual Contracts as Test Oracle in AGG 2.0

A test oracle predicts expected outcomes for a set of test cases, often based on a formal, executable specification. Visual contracts are graph transformation rules describing pre- and post-conditions of a service’s operations. To obtain an oracle based on visual contracts, we use the Attributed Graph Grammar System (AGG) to execute the rules, creating a simulation of the behaviour expected of the system under test.The paper discusses the basic idea, illustrates it by an example, describes the challenges and solutions of its implementation and draws conclusions for the use of graph transformation and AGG in test oracles