Applying adaptive test cases to nondeterministic implementations

The testing of a state-based system involves the application of sequences of inputs and the observation of the resultant input/output sequences (traces). These traces can result from preset input sequences or adaptive test cases in which the choice of the next input depends on the trace that has observed up to that input. Adaptive test cases are used in a number of areas including protocol conformance testing and adaptivity forms the basis of the standardised test language TTCN. Suppose that we apply adaptive test case ° to the system under test (SUT) and observe the trace ¯¾. If the SUT is deterministic and we apply ° again, after resetting the SUT, then we will observe ¯¾ again. Further, if we have another adaptive test case °0 where a prefix ¯¾0 of ¯¾ is a possible response to °0 then we know that the application of °0 must lead to ¯¾0. Thus, for a deterministic SUT the response of the SUT to an adaptive test case °0 might be deduced from the response of the SUT to another adaptive test case. This observation can be used to reduce the cost of testing: we only apply adaptive test case °0 if we cannot deduce the response to °0 from the set of observations. While many systems are deterministic, nondeterminism is becoming increasingly common. Nondeterminism in the SUT is typically a consequence of limits in the ability to observe the SUT. For example, it could be a result of information hiding, real time properties, or of different possible interleavings in a concurrent system (see, for example. This paper investigates the case where the SUT is nondeterministic. We consider the situation in which a set O of traces has been observed in testing and we are considering applying an adaptive test case °. In general we cannot expect to be able to deduce the response of a nondeterministic SUT to an adaptive test case ° since there may be more than one possible response. Instead we consider the question of how we can decide whether the application of ° could lead to a trace that has not been observed. A solution to this would allow us to reduce the cost of testing: if all possible responses of the SUT to ° have already been observed then we do not have to apply ° in testing and thus reduce the cost of test execution. This paper considers three cases. Section 3 considers the case where we can apply a fairness assumption. Section 4 weakens this assumption to us having a lower bound p on the probability of observing alternative responses of the SUT to any input and in any state. Section 5 then considers the general case

Reducing the cost of applying adaptive test cases

The testing of a state-based system may involve the application of a number of adaptive test cases. Where the implementation under test (IUT) is deterministic, the response of the IUT to some adaptive test case $\gamma_1$ could be capable of determining the response of the IUT to another adaptive test case $\gamma_2". Thus, the expected cost of applying a set of adaptive test cases depends upon the order in which they are applied. This paper explores properties of adaptive test cases and considers the problem of finding an order of application of the elements from some set of adaptive test cases, which minimises the expected cost of testing

Environment Behavior Models for Scenario Generation and Testing Automation

In Proceedings of the First International Workshop on Advances in Model-Based Software Testing (A-MOST'05), the 27th International Conference on Software Engineering ICSE’05, May 15-16, 2005, St. Louis, USAThis paper suggests an approach to automatic scenario generation from environment models for testing of real-time reactive systems. The behavior of the system is defined as a set of events (event trace) with two basic relations: precedence and inclusion. The attributed event grammar (AEG) specifies possible event traces and provides a uniform approach for automatically generating, executing, and analyzing test cases. The environment model includes a description of hazardous states in which the system may arrive and makes it possible to gather statistics for system safety assessment. The approach is supported by a generator that creates test cases from the AEG models. We demonstrate the approach with case studies of prototypes for the safety-critical computer-assisted resuscitation algorithm (CARA) software for a casualty intravenous fluid infusion pump and the Paderborn Shuttle System

Testing from a nondeterministic finite state machine using adaptive state counting

The problem of generating a checking experiment from a nondeterministic finite state machine has been represented in terms of state counting. However, test techniques that use state counting traditionally produce preset test suites. This paper extends the notion of state counting in order to allow the input/output sequences observed in testing to be utilized: Adaptive state counting is introduced. The main benefit of the proposed approach is that it may result in a reduction in the size of the test suite used. An additional benefit is that, where a failure is observed, it is possible to terminate test generation at this point

New Directions in Software Quality Assurance Automation

Abstract. A formalism is suggested for specifying environment behavior models for software test scenario generation based on attributed event grammars. The environment model may contain descriptions of the events triggered by the software outputs and of the hazardous states in which the system could arrive, thus providing a framework for specifying properties of software behavior within the given environment. The behavior of the system can be rendered as an event set with two partial ordering relations: precedence and inclusion (event trace). This formalism may be used as a basis for automation tools for test generation, test result monitoring and verification, for experiments to gather statistics about software safety, and for evaluating of dependencies of system's behavior on environment parameters. The monitoring activities can be implemented within a uniform framework as computations over event traces

Traffic generator for firewall testing

Thesis (Master)--Izmir Institute of Technology, Computer Engineering, Izmir, 2009Includes bibliographical references (leaves: 52-56)Text in English; Abstract: Turkish and Englishix, 92 leavesFirewalls lead at the front line of a computer network to restrict unauthorized access. The desired security level is determined by a policy and implemented by a firewall which not only has to be effective but also stable and reliable service is expected. In order to verify the level of security of the system, testing is required. The objective of this thesis is to test a firewall with software testing techniques taking into consideration the nominated policy and the firewall. Iptables software was examined and tested by two different algorithms that were modified according to software testing techniques, and the results were observed. Packets sent through the Firewall Under Test (FUT) are compared to packets passed through the FUT and test results were observed. The security performance of the modified algorithms proved to be successful

Environment behavior models for real-time reactive system testing automation

We explored the effectiveness of using attributed event grammars (AEG) based environment behavior models as a method for testing and analyzing real-time, reactive software systems. The AEG specifies possible event traces and provides a uniform approach for automatically generating and executing test cases. We have demonstrated the approach through a case study (Paderborn Shuttle System Control Software) and performed three kinds of experiments: software correctness testing, system performance analysis and study of design alternatives.http://archive.org/details/environmentbehav109452676Approved for public release; distribution is unlimited

La innovación pendiente. Reflexiones (y Provocaciones) sobre Educación, Tecnología y Conocimiento

Hasta ahora los dispositivos han acaparado toda la atención, pero los verdaderos cambios en la enseñanza y el aprendizaje han sido invisibles. Con la intención de darles la visibilidad que merecen, el profesor e investigador Cristóbal Cobo nos invita a pensar en la educación desde una nueva óptica, una que plantea que la verdadera innovación está en explorar nuevas formas de valorar y reconocer el conocimiento, tanto dentro como fuera del aula. Dado que la tecnología forma parte integral de nuestras vidas, esta reflexión trasciende el contexto educativo y parte de la base de que todo espacio se convierte en un entorno de aprendizaje. Pero olvidémonos de encontrar fórmulas en este trabajo. Al contrario, el gran aporte del autor consiste en analizar los desafíos que presentan las tecnologías en la enseñanza y las redefiniciones de los conceptos saber y conocimiento en una sociedad que está en plena transformación.Fundación Ceibal y Penguin Random House Grupo Editorial Urugua