Assessing, Comparing, and Combining Statechart- based testing and Structural testing: An Experiment

Although models have been proven to be helpful in a number of software engineering activities there is still significant resistance to model-driven development. This paper investigates one specific aspect of this larger problem. It addresses the impact of using statecharts for testing class clusters that exhibit a state-dependent behavior. More precisely, it reports on a controlled experiment that investigates their impact on testing fault-detection effectiveness. Code-based, structural testing is compared to statechart-based testing and their combination is investigated to determine whether they are complementary. Results show that there is no significant difference between the fault detection effectiveness of the two test strategies but that they are significantly more effective when combined. This implies that a cost-effective strategy would specify statechart-based test cases early on, execute them once the source code is available, and then complete them with test cases based on code coverage analysis

Instrumenting contracts with aspect-oriented programming to increase observability and support debugging

In this paper we report on how Aspect-Oriented Programming (AOP), using AspectJ, can be employed to automatically and efficiently instrument contracts and invariants in Java, The paper focuses on the templates to instrument preconditions, postconditions, and class invariants, and the necessary instrumentation for compliance-checking to the Liskov Substitution Principle

A UML-based quantitative framework for early prediction of resource usage and load in distributed real-time systems

This paper presents a quantitative framework for early prediction of resource usage and load in distributed real-time systems (DRTS). The prediction is based on an analysis of UML 2.0 sequence diagrams, augmented with timing information, to extract timed-control flow information. It is aimed at improving the early predictability of a DRTS by offering a systematic approach to predict, at the design phase, system behavior in each time instant during its execution. Since behavioral models such as sequence diagrams are available in early design phases of the software life cycle, the framework enables resource analysis at a stage when design decisions are still easy to change. Though we provide a gener

Enabling the runtime assertion checking of concurrent contracts for the Java modeling language

Though there exists ample support for Design by Contract (DbC) for sequential programs, applying DbC to concurrent programs presents several challenges. In previous work, we extended the Java Modeling Language (JML) with constructs to specify concurrent contracts for Java programs. We present a runtime assertion checker (RAC) for the expanded JML capable of verifying assertions for concurrent Java programs. We systematically evaluate the validity of system testing results obtained via runtime assertion checking using actual concurrent and functional faults on a highly concurrent industrial system from the telecommunications domain

Toward automatic generation of Intrusion Detection verification rules

An Intrusion Detection System (IDS) is a crucial element of a network security posture. One class of IDS, called signature-based network IDSs, monitors network traffic, looking for evidence of malicious behavior as specified in attack descriptions (referred to as signatures). Many studies have reported that IDSs can generate thousands of alarms a day, many of which are false alarms. The problem often lies in the low accuracy of IDS signatures. It is therefore important to have more accurate signatures in order to reduce the number of false alarms. One part of the false alarm problem is the inability of IDSs to verify attacks (i.e. distinguish between successful and failed attacks). If IDSs were able to accurately verify attacks, this would reduce the number of false alarms a network administrator has to investigate. In this paper, we demonstrate the feasibility of using a data mining algorithm to automatically generate IDS verification rules. We show that this automated approach is effective in reducing the number of false alarms

Multi-objective genetic algorithm to support class responsibility assignment

Class responsibility assignment is not an easy skill to acquire. Though there are many methodologies for assigning responsibilities to classes, they all rely on human judgment and decision making. Our objective is to provide decision-making help to re-assign methods and attributes to classes in a class diagram. Our solution is based on a multi-objective genetic algorithm (MOGA) and uses class coupling and cohesion measurement. Our MOGA takes as input a class diagram to be optimized and suggests possible improvements to it. The choice of a MOGA stems from the fact that there are typically many evaluation criteria that cannot be easily combined into one objective, and several alternative solutions are acceptable for a given OO domain model. This article presents our approach in detail, our decisions regarding the multi-objective genetic algorithm, and reports on a case study. Our results suggest that the MOGA can help correct suboptimal class responsibility assignment decisions

Revisiting strategies for ordering class integration testing in the presence of dependency cycles

The issue of ordering class integration in the context of integration testing has been discussed by a number of researchers. More specifically, strategies have been proposed to generate a test order while minimizing stubbing. Recent papers have addressed the problem of deriving an integration order in the presence of dependency cycles in the class diagram. Such dependencies represent a practical problem as they make any topological ordering of classes impossible. This paper proposes a strategy that integrates two existing methods aimed at "breaking" cycles so as to allow a topological order of classes. The first one was proposed by Tai et al and is based on assigning a higher-level order according to aggregation and inheritance relationships and a lower-level order according to associations. The second one was proposed by Le Traon et al and is based on identifying strongly connected components in the dependency graph. Among other things, the former approach may result into unnecessary stubbing whereas the latter may lead to br

An investigation of graph-based class integration test order strategies

The issue of ordering class integration in the context of integration testing has been discussed by a number of researchers. More specifically, strategies have been proposed to generate a test order while minimizing stubbing. Recent papers have addressed the problem of deriving an integration order in the presence of dependency cycles in the class diagram. Such dependencies represent a practical problem as they make any topological ordering of classes impossible. Three main approaches, aimed at "breaking" cycles, have been proposed. The first one was proposed by Tai and Daniels [22] and is based on assigning a higher-level order according to aggregation and inheritance relationships and a lower-level order according to associations. The second one was proposed by Le Traon et al. [16] and is based on identifying strongly connected components in the dependency graph. The third one was propos

Using machine learning to refine black-box test specifications and test suites

In the context of open source development or software evolution, developers often face test suites which have been developed with no apparent rationale and which may need to be augmented or refined to ensure sufficient dependability, or even reduced to meet tight deadlines. We refer to this process as the re-engineering of test suites. It is important to provide both methodological and tool support to help people understand the limitations of test suites and their possible redundancies, so as to be able to refine them in a cost effective manner. To address this problem in the case of black-box testing, we propose a methodology based on machine learning that has shown promising results on a case study

On the effectiveness of contracts as test oracles in the detection and diagnosis of race conditions and deadlocks in concurrent object-oriented software

The idea behind Design by Contract (DbC) is that a method defines a contract stating the requirements a client needs to fulfill to use it, the precondition, and the properties it ensures after its execution, the postcondition. Though there exists ample support for DbC for sequential programs, applying DbC to concurrent programs presents several challenges. We have proposed a solution to these challenges in the context of Java as programming language and the Java Modeling language as specification language. This paper presents our findings when applying our DbC technique on an industrial case study to evaluate the ability of contract-based, runtime assertion checking code at detecting and diagnosing race conditions and deadlocks during system testing. The case study is a highly concurrent industrial system from the telecommunications domain, with actual faults. It is the first work to systematically investigate the impact of contract assertions for the detection of race conditions and deadlocks, along with functional properties, in an industrial system