277,253 research outputs found
Evaluating Mission-Critical Self-Adaptive Software Systems: A Testing-Based Approach
Self-adaptive software is a closed-loop system that tries to manage, direct, or regulate its own behavior dynamically. Such a system aims at providing an automated and systematic approach to handling the increasing complexity of operation management. Mission-critical systems (e.g., e-business and telecommunication systems) are usually large, complex, and distributed. These systems must preserve their Quality of Service (QoS) at runtime under highly dynamic and non-deterministic conditions; therefore, they are suitable candidates for being equipped with self-adaptive capabilities. Although significant efforts have been devoted to modeling, designing, developing and deploying self-adaptive software since a decade ago, there is still a lack of well-established concrete processes for evaluating such systems.
This dissertation proposes a systematic evaluation process for mission-critical self-adaptive software systems. The process is a well-defined testing approach that needs a post-mortem analysis, takes the quantified QoS requirements as inputs, and comprises two main phases: i) conducting system-level testing, and ii) evaluating QoS requirements satisfaction. The process uses Service Level Agreements (SLAs) as quantified QoS requirements, and consequently as the adaptation requirements of mission-critical systems. Adaptation requirements are specific types of requirements used to engineer self-adaptive software. Moreover, for the first phase, the dissertation discusses the uniqueness and necessity of conducting system-level load and stress testing on a self-adaptive software system, for collecting runtime QoS data. In the second phase, the process makes use of utility functions to generate a single value indicating the QoS satisfaction of the evaluated system. The dissertation mainly focuses on evaluating the performance, availability and reliability characteristics of QoS.
An open source service-oriented Voice over IP (VoIP) application was selected as a case study. The VoIP application was transformed into a self-adaptive software system with various types of adaptation mechanisms. A set of empirical experiments was performed on the developed self-adaptive VoIP application, and the proposed process was adopted for evaluating the effectiveness of different adaptation mechanisms. To this end, the dissertation defines a sample SLA for the VoIP application, presents a report on the load and stress testing performed on the self-adaptive VoIP application, and presents a set of utility functions for evaluation. The experiments illustrate the validity, reliability, flexibility, and cost of the proposed evaluation process.
In sum, this dissertation introduces a novel evaluation process for mission-critical self-adaptive software systems, and shows that the proposed process can help researchers to systematically evaluate their self-adaptive systems
A Framework for Evaluating Model-Driven Self-adaptive Software Systems
In the last few years, Model Driven Development (MDD), Component-based
Software Development (CBSD), and context-oriented software have become
interesting alternatives for the design and construction of self-adaptive
software systems. In general, the ultimate goal of these technologies is to be
able to reduce development costs and effort, while improving the modularity,
flexibility, adaptability, and reliability of software systems. An analysis of
these technologies shows them all to include the principle of the separation of
concerns, and their further integration is a key factor to obtaining
high-quality and self-adaptable software systems. Each technology identifies
different concerns and deals with them separately in order to specify the
design of the self-adaptive applications, and, at the same time, support
software with adaptability and context-awareness. This research studies the
development methodologies that employ the principles of model-driven
development in building self-adaptive software systems. To this aim, this
article proposes an evaluation framework for analysing and evaluating the
features of model-driven approaches and their ability to support software with
self-adaptability and dependability in highly dynamic contextual environment.
Such evaluation framework can facilitate the software developers on selecting a
development methodology that suits their software requirements and reduces the
development effort of building self-adaptive software systems. This study
highlights the major drawbacks of the propped model-driven approaches in the
related works, and emphasise on considering the volatile aspects of
self-adaptive software in the analysis, design and implementation phases of the
development methodologies. In addition, we argue that the development
methodologies should leave the selection of modelling languages and modelling
tools to the software developers.Comment: model-driven architecture, COP, AOP, component composition,
self-adaptive application, context oriented software developmen
mRUBiS: An Exemplar for Model-Based Architectural Self-Healing and Self-Optimization
Self-adaptive software systems are often structured into an adaptation engine
that manages an adaptable software by operating on a runtime model that
represents the architecture of the software (model-based architectural
self-adaptation). Despite the popularity of such approaches, existing exemplars
provide application programming interfaces but no runtime model to develop
adaptation engines. Consequently, there does not exist any exemplar that
supports developing, evaluating, and comparing model-based self-adaptation off
the shelf. Therefore, we present mRUBiS, an extensible exemplar for model-based
architectural self-healing and self-optimization. mRUBiS simulates the
adaptable software and therefore provides and maintains an architectural
runtime model of the software, which can be directly used by adaptation engines
to realize and perform self-adaptation. Particularly, mRUBiS supports injecting
issues into the model, which should be handled by self-adaptation, and
validating the model to assess the self-adaptation. Finally, mRUBiS allows
developers to explore variants of adaptation engines (e.g., event-driven
self-adaptation) and to evaluate the effectiveness, efficiency, and scalability
of the engines
Robustness-Driven Resilience Evaluation of Self-Adaptive Software Systems
An increasingly important requirement for certain classes of software-intensive systems is the ability to self-adapt their structure and behavior at run-time when reacting to changes that may occur to the system, its environment, or its goals. A major challenge related to self-adaptive software systems is the ability to provide assurances of their resilience when facing changes. Since in these systems, the components that act as controllers of a target system incorporate highly complex software, there is the need to analyze the impact that controller failures might have on the services delivered by the system. In this paper, we present a novel approach for evaluating the resilience of self-adaptive software systems by applying robustness testing techniques to the controller to uncover failures that can affect system resilience. The approach for evaluating resilience, which is based on probabilistic model checking, quantifies the probability of satisfaction of system properties when the target system is subject to controller failures. The feasibility of the proposed approach is evaluated in the context of an industrial middleware system used to monitor and manage highly populated networks of devices, which was implemented using the Rainbow framework for architecture-based self-adaptation
Planning as Optimization: Dynamically Discovering Optimal Configurations for Runtime Situations
The large number of possible configurations of modern software-based systems,
combined with the large number of possible environmental situations of such
systems, prohibits enumerating all adaptation options at design time and
necessitates planning at run time to dynamically identify an appropriate
configuration for a situation. While numerous planning techniques exist, they
typically assume a detailed state-based model of the system and that the
situations that warrant adaptations are known. Both of these assumptions can be
violated in complex, real-world systems. As a result, adaptation planning must
rely on simple models that capture what can be changed (input parameters) and
observed in the system and environment (output and context parameters). We
therefore propose planning as optimization: the use of optimization strategies
to discover optimal system configurations at runtime for each distinct
situation that is also dynamically identified at runtime. We apply our approach
to CrowdNav, an open-source traffic routing system with the characteristics of
a real-world system. We identify situations via clustering and conduct an
empirical study that compares Bayesian optimization and two types of
evolutionary optimization (NSGA-II and novelty search) in CrowdNav
Layered evaluation of interactive adaptive systems : framework and formative methods
Peer reviewedPostprin
- …