Engineering Self-Adaptive Collective Processes for Cyber-Physical Ecosystems

The pervasiveness of computing and networking is creating significant opportunities for building valuable socio-technical systems. However, the scale, density, heterogeneity, interdependence, and QoS constraints of many target systems pose severe operational and engineering challenges. Beyond individual smart devices, cyber-physical collectives can provide services or solve complex problems by leveraging a “system effect” while coordinating and adapting to context or environment change. Understanding and building systems exhibiting collective intelligence and autonomic capabilities represent a prominent research goal, partly covered, e.g., by the field of collective adaptive systems. Therefore, drawing inspiration from and building on the long-time research activity on coordination, multi-agent systems, autonomic/self-* systems, spatial computing, and especially on the recent aggregate computing paradigm, this thesis investigates concepts, methods, and tools for the engineering of possibly large-scale, heterogeneous ensembles of situated components that should be able to operate, adapt and self-organise in a decentralised fashion. The primary contribution of this thesis consists of four main parts. First, we define and implement an aggregate programming language (ScaFi), internal to the mainstream Scala programming language, for describing collective adaptive behaviour, based on field calculi. Second, we conceive of a “dynamic collective computation” abstraction, also called aggregate process, formalised by an extension to the field calculus, and implemented in ScaFi. Third, we characterise and provide a proof-of-concept implementation of a middleware for aggregate computing that enables the development of aggregate systems according to multiple architectural styles. Fourth, we apply and evaluate aggregate computing techniques to edge computing scenarios, and characterise a design pattern, called Self-organising Coordination Regions (SCR), that supports adjustable, decentralised decision-making and activity in dynamic environments.Con lo sviluppo di informatica e intelligenza artificiale, la diffusione pervasiva di device computazionali e la crescente interconnessione tra elementi fisici e digitali, emergono innumerevoli opportunità per la costruzione di sistemi socio-tecnici di nuova generazione. Tuttavia, l'ingegneria di tali sistemi presenta notevoli sfide, data la loro complessità—si pensi ai livelli, scale, eterogeneità, e interdipendenze coinvolti. Oltre a dispositivi smart individuali, collettivi cyber-fisici possono fornire servizi o risolvere problemi complessi con un “effetto sistema” che emerge dalla coordinazione e l'adattamento di componenti fra loro, l'ambiente e il contesto. Comprendere e costruire sistemi in grado di esibire intelligenza collettiva e capacità autonomiche è un importante problema di ricerca studiato, ad esempio, nel campo dei sistemi collettivi adattativi. Perciò, traendo ispirazione e partendo dall'attività di ricerca su coordinazione, sistemi multiagente e self-*, modelli di computazione spazio-temporali e, specialmente, sul recente paradigma di programmazione aggregata, questa tesi tratta concetti, metodi, e strumenti per l'ingegneria di ensemble di elementi situati eterogenei che devono essere in grado di lavorare, adattarsi, e auto-organizzarsi in modo decentralizzato. Il contributo di questa tesi consiste in quattro parti principali. In primo luogo, viene definito e implementato un linguaggio di programmazione aggregata (ScaFi), interno al linguaggio Scala, per descrivere comportamenti collettivi e adattativi secondo l'approccio dei campi computazionali. In secondo luogo, si propone e caratterizza l'astrazione di processo aggregato per rappresentare computazioni collettive dinamiche concorrenti, formalizzata come estensione al field calculus e implementata in ScaFi. Inoltre, si analizza e implementa un prototipo di middleware per sistemi aggregati, in grado di supportare più stili architetturali. Infine, si applicano e valutano tecniche di programmazione aggregata in scenari di edge computing, e si propone un pattern, Self-Organising Coordination Regions, per supportare, in modo decentralizzato, attività decisionali e di regolazione in ambienti dinamici

Using Process Technology to Control and Coordinate Software Adaptation

We have developed an infrastructure for end-to-end run-time monitoring, behavior/performance analysis, and dynamic adaptation of distributed software. This infrastructure is primarily targeted to pre-existing systems and thus operates outside the target application, without making assumptions about the target's implementation, internal communication/computation mechanisms, source code availability, etc. This paper assumes the existence of the monitoring and analysis components, presented elsewhere, and focuses on the mechanisms used to control and coordinate possibly complex repairs/reconfigurations to the target system. These mechanisms require lower level effectors somehow attached to the target system, so we briefly sketch one such facility (elaborated elsewhere). Our main contribution is the model, architecture, and implementation of Workflakes, the decentralized process engine we use to tailor, control, coordinate, etc. a cohort of such effectors. We have validated the Workflakes approach with case studies in several application domains. Due to space restrictions we concentrate primarily on one case study, briefly discuss a second, and only sketch others

Self-healing and SDN: bridging the gap

Achieving high programmability has become an essential aim of network research due to the ever-increasing internet traffic. Software-Defined Network (SDN) is an emerging architecture aimed to address this need. However, maintaining accurate knowledge of the network after a failure is one of the largest challenges in the SDN. Motivated by this reality, this paper focuses on the use of self-healing properties to boost the SDN robustness. This approach, unlike traditional schemes, is not based on proactively configuring multiple (and memory-intensive) backup paths in each switch or performing a reactive and time-consuming routing computation at the controller level. Instead, the control paths are quickly recovered by local switch actions and subsequently optimized by global controller knowledge. Obtained results show that the proposed approach recovers the control topology effectively in terms of time and message load over a wide range of generated networks. Consequently, scalability issues of traditional fault recovery strategies are avoided.Postprint (published version

Contributions to topology discovery, self-healing and VNF placement in software-defined and virtualized networks

The evolution of information and communication technologies (e.g. cloud computing, the Internet of Things (IoT) and 5G, among others) has enabled a large market of applications and network services for a massive number of users connected to the Internet. Achieving high programmability while decreasing complexity and costs has become an essential aim of networking research due to the ever-increasing pressure generated by these applications and services. However, meeting these goals is an almost impossible task using traditional IP networks. Software-Defined Networking (SDN) is an emerging network architecture that could address the needs of service providers and network operators. This new technology consists in decoupling the control plane from the data plane, enabling the centralization of control functions on a concentrated or distributed platform. It also creates an abstraction between the network infrastructure and network applications, which allows for designing more flexible and programmable networks. Recent trends of increased user demands, the explosion of Internet traffic and diverse service requirements have further driven the interest in the potential capabilities of SDN to enable the introduction of new protocols and traffic management models. This doctoral research is focused on improving high-level policies and control strategies, which are becoming increasingly important given the limitations of current solutions for large-scale SDN environments. Specifically, the three largest challenges addressed in the development of this thesis are related to the processes of topology discovery, fault recovery and Virtual Network Function (VNF) placement in software-defined and virtualized networks. These challenges led to the design of a set of effective techniques, ranging from network protocols to optimal and heuristic algorithms, intended to solve existing problems and contribute to the deployment and adoption of such programmable networks.For the first challenge, this work presents a novel protocol that, unlike existing approaches, enables a distributed layer 2 discovery without the need for previous IP configurations or controller knowledge of the network. By using this mechanism, the SDN controller can discover the network view without incurring scalability issues, while taking advantage of the shortest control paths toward each switch. Moreover, this novel approach achieves noticeable improvement with respect to state-of-the-art techniques. To address the resilience concern of SDN, we propose a self-healing mechanism that recovers the control plane connectivity in SDN-managed environments without overburdening the controller performance. The main idea underlying this proposal is to enable real-time recovery of control paths in the face of failures without the intervention of a controller. Obtained results show that the proposed approach recovers the control topology efficiently in terms of time and message load over a wide range of generated networks. The third contribution made in this thesis combines topology knowledge with bin packing techniques in order to efficiently place the required VNF. An online heuristic algorithm with low-complexity was developed as a suitable solution for dynamic infrastructures. Extensive simulations, using network topologies representative of different scales, validate the good performance of the proposed approaches regarding the number of required instances and the delay among deployed functions. Additionally, the proposed heuristic algorithm improves the execution times by a fifth order of magnitude compared to the optimal formulation of this problem.Postprint (published version