A Case Study in Testing Distributed Systems

This paper describes a case study in the testing of distributed systems. The software under test is a middleware system developed in Java. The full test lifecycle is examined including unit testing, integration testing, and system testing. Where possible, traditional tools and techniques are used to carry out the testing. One aspect where this is not possible is the testing of the low-level concurrency, which is often overlooked when testing commercial distributed systems, since the middleware or application server is already developed by a third-party and is assumed to operate correctly. This paper examines testing the middleware system itself, and therefore, a method for testing the concurrency properties of the system is used. The testing revealed a number of faults and design weaknesses, and showed that, with some adaptation, traditional tools and techniques go a long way in the testing of distributed applications

Conformance testing of peer-to-peer systems using message traffic analysis

Peer-to-Peer architectures are used by a large number of distributed systems; however, the challenges such as maintaining a reliable and stable peer-to-peer network can make such networks undesirable for distributed systems. Peer-to-peer architectures are designed to be executed on systems with diverse hardware configurations, distant geographic locations, and varied, unpredictable Internet connectivity that make the software testing process difficult. This research defines a method for conformance testing peer-to-peer content distribution systems called “Method for Conformance Testing by Analyzing Message Activity” (MCTAMA). MCTAMA uses a common representation for describing the behavior of nodes during both design and deployment. ATAMA generates, evaluates and filters test cases that help determine variation between the expected and observed behaviors. The focus on message traffic allows MCTAMA to be used at multiple stages of development and deployment while not being affected by the variations in the operating environment, availability of source code or the capabilities of a monitoring mechanism. As a part of MCTAMA, this research includes a method for combining sequence diagrams to create a description of the expected behavior of nodes in the system

Software integration testing based on communication coverage criteria and partial model generation

This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send- and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences are derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences could suffer from a combinatorial explosion, the effort required to generate the partial model is polynomial in the number of test sequences and their length. Thus, where it is not feasible to produce test sequences that achieve a given type of coverage it is still possible to produce a partial model on the basis of test sequences generated to achieve some other criterion. As a result, the process of generating a partial model has the potential to scale to large industrial software systems. While a particular model checker, UPPAAL, was used, it should be relatively straightforward to adapt the approach for use with other CTL based model checkers. A potential additional benefit of the approach is that it provides a visual description of the state-based testing of distributed systems, which may be beneficial in other contexts such as education and comprehension

Distributed Fault Diagnosis using Sensor Networks and Consensus-based Filters

This paper considers the problem of designing distributed fault diagnosis algorithms for dynamic systems using sensor networks. A network of distributed estimation agents is designed where a bank of local Kalman filters is embedded into each sensor. The diagnosis decision is performed by a distributed hypothesis testing method that relies on a belief consensus algorithm. Under certain assumptions, both the distributed estimation and the diagnosis algorithms are derived from their centralized counterparts thanks to dynamic average-consensus techniques. Simulation results are provided to demonstrate the effectiveness of the proposed architecture and algorithm

Advanced operator-splitting-based semi-implicit spectral method to solve the binary phase-field crystal equations with variable coefficients

We present an efficient method to solve numerically the equations of dissipative dynamics of the binary phase-field crystal model proposed by Elder et al. [Phys. Rev. B 75, 064107 (2007)] characterized by variable coefficients. Using the operator splitting method, the problem has been decomposed into sub-problems that can be solved more efficiently. A combination of non-trivial splitting with spectral semi-implicit solution leads to sets of algebraic equations of diagonal matrix form. Extensive testing of the method has been carried out to find the optimum balance among errors associated with time integration, spatial discretization, and splitting. We show that our method speeds up the computations by orders of magnitude relative to the conventional explicit finite difference scheme, while the costs of the pointwise implicit solution per timestep remains low. Also we show that due to its numerical dissipation, finite differencing can not compete with spectral differencing in terms of accuracy. In addition, we demonstrate that our method can efficiently be parallelized for distributed memory systems, where an excellent scalability with the number of CPUs is observed