Management and Service-aware Networking Architectures (MANA) for Future Internet Position Paper: System Functions, Capabilities and Requirements

Future Internet (FI) research and development threads have recently been gaining momentum all over the world and as such the international race to create a new generation Internet is in full swing: GENI, Asia Future Internet, Future Internet Forum Korea, European Union Future Internet Assembly (FIA). This is a position paper identifying the research orientation with a time horizon of 10 years, together with the key challenges for the capabilities in the Management and Service-aware Networking Architectures (MANA) part of the Future Internet (FI) allowing for parallel and federated Internet(s)

Large-scale Complex IT Systems

This paper explores the issues around the construction of large-scale complex systems which are built as 'systems of systems' and suggests that there are fundamental reasons, derived from the inherent complexity in these systems, why our current software engineering methods and techniques cannot be scaled up to cope with the engineering challenges of constructing such systems. It then goes on to propose a research and education agenda for software engineering that identifies the major challenges and issues in the development of large-scale complex, software-intensive systems. Central to this is the notion that we cannot separate software from the socio-technical environment in which it is used.Comment: 12 pages, 2 figure

A Deep Recurrent Q Network Towards Self-adapting Distributed Microservices Architecture (in press)

One desired aspect of microservices architecture is the ability to self-adapt its own architecture and behaviour in response to changes in the operational environment. To achieve the desired high levels of self-adaptability, this research implements the distributed microservices architectures model, as informed by the MAPE-K model. The proposed architecture employs a multi adaptation agents supported by a centralised controller, that can observe the environment and execute a suitable adaptation action. The adaptation planning is managed by a deep recurrent Q-network (DRQN). It is argued that such integration between DRQN and MDP agents in a MAPE-K model offers distributed microservice architecture with self-adaptability and high levels of availability and scalability. Integrating DRQN into the adaptation process improves the effectiveness of the adaptation and reduces any adaptation risks, including resources over-provisioning and thrashing. The performance of DRQN is evaluated against deep Q-learning and policy gradient algorithms including: i) deep q-network (DQN), ii) dulling deep Q-network (DDQN), iii) a policy gradient neural network (PGNN), and iv) deep deterministic policy gradient (DDPG). The DRQN implementation in this paper manages to outperform the above mentioned algorithms in terms of total reward, less adaptation time, lower error rates, plus faster convergence and training times. We strongly believe that DRQN is more suitable for driving the adaptation in distributed services-oriented architecture and offers better performance than other dynamic decision-making algorithms

A deep recurrent Q network towards self-adapting distributed microservice architecture

One desired aspect of microservice architecture is the ability to self-adapt its own architecture and behavior in response to changes in the operational environment. To achieve the desired high levels of self-adaptability, this research implements distributed microservice architecture model running a swarm cluster, as informed by the Monitor, Analyze, Plan, and Execute over a shared Knowledge (MAPE-K) model. The proposed architecture employs multiadaptation agents supported by a centralized controller, which can observe the environment and execute a suitable adaptation action. The adaptation planning is managed by a deep recurrent Q-learning network (DRQN). It is argued that such integration between DRQN and Markov decision process (MDP) agents in a MAPE-K model offers distributed microservice architecture with self-adaptability and high levels of availability and scalability. Integrating DRQN into the adaptation process improves the effectiveness of the adaptation and reduces any adaptation risks, including resource overprovisioning and thrashing. The performance of DRQN is evaluated against deep Q-learning and policy gradient algorithms, including (1) a deep Q-learning network (DQN), (2) a dueling DQN (DDQN), (3) a policy gradient neural network, and (4) deep deterministic policy gradient. The DRQN implementation in this paper manages to outperform the aforementioned algorithms in terms of total reward, less adaptation time, lower error rates, plus faster convergence and training time. We strongly believe that DRQN is more suitable for driving the adaptation in distributed services-oriented architecture and offers better performance than other dynamic decision-making algorithms

Self-management for large-scale distributed systems

Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management. In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers. In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck. In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control

Enabling Context-Aware Web Services: A Middleware Approach for Ubiquitous Environments

In ubiquitous environments, mobile applications should sense and react to environmental changes to provide a better user experience. In order to deal with these concerns, Service-Oriented Architectures (SOA) provide a solution allowing applications to interact with the services available in their surroundings. In particular, context-aware Web Services can adapt their behavior considering the user context. However, the limited resources of mobile devices restrict the adaptation degree. Furthermore, the diverse nature of context information makes difficult its retrieval, processing and distribution. To tackle these challenges, we present the CAPPUCINO platform for executing context-aware Web Services in ubiquitous environments. In particular, in this chapter we focus on the middleware part that is built as an autonomic control loop that deals with dynamic adaptation. In this autonomic loop we use FraSCAti, an implementation of the Service Component Architecture (SCA) specification, as the execution kernel for Web Services. The context distribution is achieved with SPACES, a flexible solution based on REST (REpresentational State Transfer ) principles and benefiting from the COSMOS (COntext entitieS coMpositiOn and Sharing ) context manage- ment framework. The application of our platform is illustrated with a mobile commerce application scenario that combines context-aware Web Services and social networks

Retrofitting Autonomic Capabilities onto Legacy Systems

Autonomic computing - self-configuring, self-healing, self-optimizing applications, systems and networks - is a promising solution to ever-increasing system complexity and the spiraling costs of human management as systems scale to global proportions. Most results to date, however, suggest ways to architect new software constructed from the ground up as autonomic systems, whereas in the real world organizations continue to use stovepipe legacy systems and/or build 'systems of systems' that draw from a gamut of disparate technologies from numerous vendors. Our goal is to retrofit autonomic computing onto such systems, externally, without any need to understand, modify or even recompile the target system's code. We present an autonomic infrastructure that operates similarly to active middleware, to explicitly add autonomic services to pre-existing systems via continual monitoring and a feedback loop that performs, as needed, reconfiguration and/or repair. Our lightweight design and separation of concerns enables easy adoption of individual components, independent of the rest of the full infrastructure, for use with a large variety of target systems. This work has been validated by several case studies spanning multiple application domains

A generic architecture style for self-adaptive cyber-physical systems

Die aktuellen Konzepte zur Gestaltung von Regelungssystemen basieren auf dynamischen Verhaltensmodellen, die mathematische Ansätze wie Differentialgleichungen zur Ableitung der entsprechenden Funktionen verwenden. Diese Konzepte stoßen jedoch aufgrund der zunehmenden Systemkomplexität allmählich an ihre Grenzen. Zusammen mit der Entwicklung dieser Konzepte entsteht eine Architekturevolution der Regelungssysteme. In dieser Dissertation wird eine Taxonomie definiert, um die genannte Architekturevolution anhand eines typischen Beispiels, der adaptiven Geschwindigkeitsregelung (ACC), zu veranschaulichen. Aktuelle ACC-Varianten, die auf der Regelungstheorie basieren, werden in Bezug auf ihre Architekturen analysiert. Die Analyseergebnisse zeigen, dass das zukünftige Regelungssystem im ACC eine umfangreichere Selbstadaptationsfähigkeit und Skalierbarkeit erfordert. Dafür sind kompliziertere Algorithmen mit unterschiedlichen Berechnungsmechanismen erforderlich. Somit wird die Systemkomplexität erhöht und führt dazu, dass das zukünftige Regelungssystem zu einem selbstadaptiven cyber-physischen System wird und signifikante Herausforderungen für die Architekturgestaltung des Systems darstellt. Inspiriert durch Ansätze des Software-Engineering zur Gestaltung von Architekturen von softwareintensiven Systemen wird in dieser Dissertation ein generischer Architekturstil entwickelt. Der entwickelte Architekturstil dient als Vorlage, um vernetzte Architekturen mit Verfolgung der entwickelten Designprinzipien nicht nur für die aktuellen Regelungssysteme, sondern auch für selbstadaptiven cyber-physischen Systeme in der Zukunft zu konstruieren. Unterschiedliche Auslösemechanismen und Kommunikationsparadigmen zur Gestaltung der dynamischen Verhalten von Komponenten sind in der vernetzten Architektur anwendbar. Zur Bewertung der Realisierbarkeit des Architekturstils werden aktuelle ACCs erneut aufgenommen, um entsprechende logische Architekturen abzuleiten und die Architekturkonsistenz im Vergleich zu den originalen Architekturen basierend auf der Regelungstheorie (z. B. in Form von Blockdiagrammen) zu untersuchen. Durch die Anwendung des entwickelten generischen Architekturstils wird in dieser Dissertation eine künstliche kognitive Geschwindigkeitsregelung (ACCC) als zukünftige ACC-Variante entworfen, implementiert und evaluiert. Die Evaluationsergebnisse zeigen signifikante Leistungsverbesserungen des ACCC im Vergleich zum menschlichen Fahrer und aktuellen ACC-Varianten.Current concepts of designing automatic control systems rely on dynamic behavioral modeling by using mathematical approaches like differential equations to derive corresponding functions, and slowly reach limitations due to increasing system complexity. Along with the development of these concepts, an architectural evolution of automatic control systems is raised. This dissertation defines a taxonomy to illustrate the aforementioned architectural evolution relying on a typical example of control application: adaptive cruise control (ACC). Current ACC variants, with their architectures considering control theory, are analyzed. The analysis results indicate that the future automatic control system in ACC requires more substantial self-adaptation capability and scalability. For this purpose, more complicated algorithms requiring different computation mechanisms must be integrated into the system and further increase system complexity. This makes the future automatic control system evolve into a self-adaptive cyber-physical system and consistitutes significant challenges for the system’s architecture design. Inspired by software engineering approaches for designing architectures of software-intensive systems, a generic architecture style is proposed. The proposed architecture style serves as a template by following the developed design principle to construct networked architectures not only for the current automatic control systems but also for self-adaptive cyber-physical systems in the future. Different triggering mechanisms and communication paradigms for designing dynamic behaviors are applicable in the networked architecture. To evaluate feasibility of the architecture style, current ACCs are retaken to derive corresponding logical architectures and examine architectural consistency compared to the previous architectures considering the control theory (e.g., in the form of block diagrams). By applying the proposed generic architecture style, an artificial cognitive cruise control (ACCC) is designed, implemented, and evaluated as a future ACC in this dissertation. The evaluation results show significant performance improvements in the ACCC compared to the human driver and current ACC variants