A Case Study in Testing Distributed Systems

This paper describes a case study in the testing of distributed systems. The software under test is a middleware system developed in Java. The full test lifecycle is examined including unit testing, integration testing, and system testing. Where possible, traditional tools and techniques are used to carry out the testing. One aspect where this is not possible is the testing of the low-level concurrency, which is often overlooked when testing commercial distributed systems, since the middleware or application server is already developed by a third-party and is assumed to operate correctly. This paper examines testing the middleware system itself, and therefore, a method for testing the concurrency properties of the system is used. The testing revealed a number of faults and design weaknesses, and showed that, with some adaptation, traditional tools and techniques go a long way in the testing of distributed applications

A Multi-Agent Framework for Testing Distributed Systems

Software testing is a very expensive and time consuming process. It can account for up to 50% of the total cost of the software development. Distributed systems make software testing a daunting task. The research described in this paper investigates a novel multi-agent framework for testing 3-tier distributed systems. This paper describes the framework architecture as well as the communication mechanism among agents in the architecture. Web-based application is examined as a case study to validate the proposed framework. The framework is considered as a step forward to automate testing for distributed systems in order to enhance their reliability within an acceptable range of cost and time

Timed Implementation Relations for the Distributed Test Architecture

In order to test systems that have physically distributed interfaces, called ports, we might use a distributed approach in which there is a separate tester at each port. If the testers do not synchronise during testing then we cannot always determine the relative order of events observed at different ports and this leads to new notions of correctness that have been described using corresponding implementation relations. We study the situation in which each tester has a local clock and timestamps its observations. If we know nothing about how the local clocks relate then this does not affect the implementation relation while if the local clocks agree exactly then we can reconstruct the sequence of observations made. In practice, however, we are likely to be between these extremes: the local clocks will not agree exactly but we have some information regarding how they can differ. We start by assuming that a local tester interacts synchronously with the corresponding port of the system under test and then extend this to the case where communications can be asynchronous, considering both the first-in-first-out (FIFO) case and the non-FIFO case. The new implementation relations are stronger than implementation relations for distributed testing that do not use timestamps but still reflect the distributed nature of observations. This paper explores these alternatives and derives corresponding implementation relations

Formal Specification and Testing of a Management Architecture

The importance of network and distributed systems management to supply and maintain services required by users has led to a demand for management facilities. Open network management is assisted by representing the system resources to be managed as objects, and providing standard services and protocols for interrogating and manipulating these objects. This paper examines the application of formal description techniques to the specification of managed objects by presenting a case study in the specification and testing of a management architecture. We describe a formal specification of a management architecture suitable for scheduling and distributing services across nodes in a distributed system. In addition, we show how formal specifications can be used to generate conformance tests for the management architecture

Artificial Intelligence and Systems Theory: Applied to Cooperative Robots

This paper describes an approach to the design of a population of cooperative robots based on concepts borrowed from Systems Theory and Artificial Intelligence. The research has been developed under the SocRob project, carried out by the Intelligent Systems Laboratory at the Institute for Systems and Robotics - Instituto Superior Tecnico (ISR/IST) in Lisbon. The acronym of the project stands both for "Society of Robots" and "Soccer Robots", the case study where we are testing our population of robots. Designing soccer robots is a very challenging problem, where the robots must act not only to shoot a ball towards the goal, but also to detect and avoid static (walls, stopped robots) and dynamic (moving robots) obstacles. Furthermore, they must cooperate to defeat an opposing team. Our past and current research in soccer robotics includes cooperative sensor fusion for world modeling, object recognition and tracking, robot navigation, multi-robot distributed task planning and coordination, including cooperative reinforcement learning in cooperative and adversarial environments, and behavior-based architectures for real time task execution of cooperating robot teams

Scenarios-based testing of systems with distributed ports

Copyright @ 2011 John Wiley & SonsDistributed systems are usually composed of several distributed components that communicate with their environment through specific ports. When testing such a system we separately observe sequences of inputs and outputs at each port rather than a global sequence and potentially cannot reconstruct the global sequence that occurred. Typically, the users of such a system cannot synchronise their actions during use or testing. However, the use of the system might correspond to a sequence of scenarios, where each scenario involves a sequence of interactions with the system that, for example, achieves a particular objective. When this is the case there is the potential for there to be a significant delay between two scenarios and this effectively allows the users of the system to synchronise between scenarios. If we represent the specification of the global system by using a state-based notation, we say that a scenario is any sequence of events that happens between two of these operations. We can encode scenarios in two different ways. The first approach consists of marking some of the states of the specification to denote these synchronisation points. It transpires that there are two ways to interpret such models and these lead to two implementation relations. The second approach consists of adding a set of traces to the specification to represent the traces that correspond to scenarios. We show that these two approaches have similar expressive power by providing an encoding from marked states to sets of traces. In order to assess the appropriateness of our new framework, we show that it represents a conservative extension of previous implementation relations defined in the context of the distributed test architecture: if we onsider that all the states are marked then we simply obtain ioco (the classical relation for single-port systems) while if no state is marked then we obtain dioco (our previous relation for multi-port systems). Finally, we concentrate on the study of controllable test cases, that is, test cases such that each local tester knows exactly when to apply inputs. We give two notions of controllable test cases, define an implementation relation for each of these notions, and relate them. We also show how we can decide whether a test case satisfies these conditions.Research partially supported by the Spanish MEC project TESIS (TIN2009-14312-C02-01), the UK EPSRC project Testing of Probabilistic and Stochastic Systems (EP/G032572/1), and the UCM-BSCH programme to fund research groups (GR58/08 - group number 910606)

Generating Test Cases for LearningZone

The increment of web-based applications in all over the internet increase the concern about quality of these web-applications, which requires testing the designs (both black and white boxes) of these systems and generating new methodologies and tools for testing. This research is concern about the applications that are related to Learning Management Systems (LMS) and precisely Modular Objected Oriented Dynamic/Distributed/Development Learning Environment (Moodle), an open source software. Universiti Utara Malaysia (UUM) has named its Moodle as LearningZone which still lacks of testing. Therefore, a suitable testing approach for deriving test cases to evaluate this open source application has been identified. This application represents the case study of this research which focuses mainly on student functionalities of LearningZone. Such an objective can be achieved through identifying the suitable UML diagram for the research's case study. Later on, applying the generation of the test cases from the chosen UML diagram. Finally, the generated test cases have to be evaluated in terms of accuracy and system's errors. This will ensure the suitability of the usage of that diagram for LearningZone as a result of this research

Cyber-physical framework for emulating distributed control systems in smart grids

This paper proposes a cyber-physical framework for investigating distributed control systems operating in the context of smart-grid applications. At the moment, the literature focuses almost exclusively on the theoretical aspects of distributed intelligence in the smart-grid, meanwhile, approaches for testing and validating such systems are either missing or are very limited in their scope. Three aspects need to be taken into account while considering these applications: (1) the physical system, (2) the distributed computation platform, and (3) the communication system. In most of the previous works either the communication system is neglected or oversimplified, either the distributed computation aspect is disregarded, either both elements are missing. In order to cover all these aspects, we propose a framework which is built around a fleet of low-cost single board computers coupled with a real-time simulator. Additionally, using traffic control and network emulation, the flow of data between different controllers is shaped so that it replicates various quality of service (QoS) conditions. The versatility of the proposed framework is shown on a study case in which 27 controllers self-coordinate in order to solve the distributed optimal power flow (OPF) algorithm in a dc network

Dynamic symbolic execution for testing distributed objects

Abstract. This paper extends dynamic symbolic execution to distributed and concurrent systems. Dynamic symbolic execution can be used in software testing to systematically identify equivalence classes of input values and has been shown to scale well to large systems. Although mainly applied to sequential programs, this scalability makes it interesting to consider the technique in the distributed and concurrent setting as well. In order to extend the technique to concurrent systems, it is necessary to obtain sufficient control over the scheduling of concurrent activities to avoid race conditions. Creol, a modeling language for distributed concurrent objects, solves this problem by abstracting from a particular scheduling policy but explicitly defining scheduling points. This provides sufficient control to apply the technique of dynamic symbolic execution for model based testing of interleaved processes. The technique has been formalized in rewriting logic, executes in Maude, and applied to nontrivial examples, including an industrial case study