Delayed failure of software components using stochastic testing

The present research investigates the delayed failure of software components and addresses the problem that the conventional approach to software testing is unlikely to reveal this type of failure. Delayed failure is defined as a failure that occurs some time after the condition that causes the failure, and is a consequence of long-latency error propagation. This research seeks to close a perceived gap between academic research into software testing and industrial software testing practice by showing that stochastic testing can reveal delayed failure, and supporting this conclusion by a model of error propagation and failure that has been validated by experiment. The focus of the present research is on software components described by a request-response model. Within this conceptual framework, a Markov chain model of error propagation and failure is used to derive the expected delayed failure behaviour of software components. Results from an experimental study of delayed failure of DBMS software components MySQL and Oracle XE using stochastic testing with random generation of SQL are consistent with expected behaviour based on the Markov chain model. Metrics for failure delay and reliability are shown to depend on the characteristics of the chosen experimental profile. SQL mutation is used to generate negative as well as positive test profiles. There appear to be few systematic studies of delayed failure in the software engineering literature, and no studies of stochastic testing related to delayed failure of software components, or specifically to delayed failure of DBMS. Stochastic testing is shown to be an effective technique for revealing delayed failure of software components, as well as a suitable technique for reliability and robustness testing of software components. These results provide a deeper insight into the testing technique and should lead to further research. Stochastic testing could provide a dependability benchmark for component-based software engineering

Robustness-Driven Resilience Evaluation of Self-Adaptive Software Systems

An increasingly important requirement for certain classes of software-intensive systems is the ability to self-adapt their structure and behavior at run-time when reacting to changes that may occur to the system, its environment, or its goals. A major challenge related to self-adaptive software systems is the ability to provide assurances of their resilience when facing changes. Since in these systems, the components that act as controllers of a target system incorporate highly complex software, there is the need to analyze the impact that controller failures might have on the services delivered by the system. In this paper, we present a novel approach for evaluating the resilience of self-adaptive software systems by applying robustness testing techniques to the controller to uncover failures that can affect system resilience. The approach for evaluating resilience, which is based on probabilistic model checking, quantifies the probability of satisfaction of system properties when the target system is subject to controller failures. The feasibility of the proposed approach is evaluated in the context of an industrial middleware system used to monitor and manage highly populated networks of devices, which was implemented using the Rainbow framework for architecture-based self-adaptation

GUITest: a Java library for fully automated GUI robustness testing

© Sebastian Bauersfeld, Tanja E. J. Vos | ACM 2012. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ASE 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, http://dx.doi.org/10.1145/2351676.2351739Graphical User Interfaces (GUIs) are substantial parts of today’s applications, no matter whether these run on tablets, smartphones or desktop platforms. Since the GUI is often the only component that humans interact with, it demands for thorough testing to ensure an efficient and satisfactory user experience. Being the glue between almost all of an application’s components, GUIs also lend themselves for system level testing. However, GUI testing is inherently diffi- cult and often involves great manual labor, even with modern tools which promise automation. This paper introduces a Java library called GUITest, which allows to generate fully automated GUI robustness tests for complex applications, without the need to manually generate models or input sequences. We will explain how it operates and present first results on its applicability and effectivity during a test involving Microsoft Word.This work is supported by EU grant ICT-257574 (FITTEST).Bauersfeld, S.; Vos, TE. (2012). GUITest: a Java library for fully automated GUI robustness testing. En ASE 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. ACM. 330-333. https://doi.org/10.1145/2351676.2351739S33033

General Approach for Combining Diverse Rare Variant Association Tests Provides Improved Robustness Across a Wider Range of Genetic Architectures

The widespread availability of genome sequencing data made possible by way of next-generation technologies has yielded a flood of different gene-based rare variant association tests. Most of these tests have been published because they have superior power for particular genetic architectures. However, for applied researchers it is challenging to know which test to choose in practice when little is known a priori about genetic architecture. Recently, tests have been proposed which combine two particular individual tests (one burden and one variance components) to minimize power loss while improving robustness to a wider range of genetic architectures. In our analysis we propose an expansion of these approaches, yielding a general method that works for combining any number of individual tests. We demonstrate that running multiple different tests on the same dataset and using a Bonferroni correction for multiple testing is never better than combining tests using our general method. We also find that using a test statistic that is highly robust to the inclusion of non-causal variants (Joint-infinity) together with a previously published combined test (SKAT-O) provides improved robustness to a wide range of genetic architectures and should be considered for use in practice. Software for this approach is supplied. We support the increased use of combined tests in practice-- as well as further exploration of novel combined testing approaches using the general framework provided here--to maximize robustness of rare-variant testing strategies against a wide range of genetic architectures

A Testability Analysis Framework for Non-Functional Properties

This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties

A Case Study on Formal Verification of Self-Adaptive Behaviors in a Decentralized System

Self-adaptation is a promising approach to manage the complexity of modern software systems. A self-adaptive system is able to adapt autonomously to internal dynamics and changing conditions in the environment to achieve particular quality goals. Our particular interest is in decentralized self-adaptive systems, in which central control of adaptation is not an option. One important challenge in self-adaptive systems, in particular those with decentralized control of adaptation, is to provide guarantees about the intended runtime qualities. In this paper, we present a case study in which we use model checking to verify behavioral properties of a decentralized self-adaptive system. Concretely, we contribute with a formalized architecture model of a decentralized traffic monitoring system and prove a number of self-adaptation properties for flexibility and robustness. To model the main processes in the system we use timed automata, and for the specification of the required properties we use timed computation tree logic. We use the Uppaal tool to specify the system and verify the flexibility and robustness properties.Comment: In Proceedings FOCLASA 2012, arXiv:1208.432

Development and manufacturing of a testing platform for unmanned aircraft vehicles

The field of unmanned aircraft vehicles (UAVs) has experienced significant technological advancements and diverse applications in recent years. Consequently, the need for improved testing methods to keep pace with these advancements has become crucial. This masters thesis presents the development of a novel test platform for UAVs, encompassing the development process, encountered challenges, and future prospects. The primary objective of this project is to design and create a UAV test bench that enhances the workflow of a research and development team. The final design fulfils all mandatory requirements established at the project's inception and includes additional features to address secondary needs. As a result, the developed prototype provides the team with a valuable tool applicable to various scenarios, including the development of new aircraft, and testing and troubleshooting of existing fleet members. The prototypes development follows an iterative design approach, adapting to changing requirements and overcoming challenges related to materials, budget constraints, and manufacturing complexity. To ensure robustness and reliability, software and electronics choices are based on established components and protocols, resulting in a test bench that outperforms the tested UAV in terms of reliability and robustness. The document starts by introducing the reader to the purpose and reasoning behind why the project was done. Then a market analysis is presented together with an explanation for the decision of starting a design from scratch. The next section describes in detail the design process of the prototype. This chapter is followed by the software architecture description. The last body chapter explains the systems integration in preparation for the manufacturing and assembly process. Finally, a conclusion chapter showing the project conclusions, cost approximation, and further work is presented. Through this research, a comprehensive understanding of UAV testing methodologies and the development of an advanced test platform is achieved, laying the foundation for continued advancements in the field of unmanned aircraft vehicles

Towards Identifying and closing Gaps in Assurance of autonomous Road vehicleS - a collection of Technical Notes Part 1

This report provides an introduction and overview of the Technical Topic Notes (TTNs) produced in the Towards Identifying and closing Gaps in Assurance of autonomous Road vehicleS (Tigars) project. These notes aim to support the development and evaluation of autonomous vehicles. Part 1 addresses: Assurance-overview and issues, Resilience and Safety Requirements, Open Systems Perspective and Formal Verification and Static Analysis of ML Systems. Part 2: Simulation and Dynamic Testing, Defence in Depth and Diversity, Security-Informed Safety Analysis, Standards and Guidelines