Diseño e implementación de estrategias de auto-optimización y autoadaptación para sistemas distribuidos a gran escala

Large scale distributed software systems are complex systems that need to be able to adapt to a highly dynamic environment and changing user needs. In this context, the main objective of this project is the development of new selfadaptive strategies, along with the methodologies and tools required for their analysis and design. In this work, we design and implement a self-adaptive architecture inspired by IBM's Monitor-Analyze-Plan-Act over a Knowledge base architecture, and we develop new self-adaptive strategies specific for wireless sensor networks following a methodology borrowed from control engineering. More in detail, we start this work by designing in UML a software library for the development of self-adaptive capabilities, which will be implemented as a Java package. After that, we model two distributed software systems using an actor oriented approach in Ptolemy II. Next, we develop the self-adaptive strategies based on fuzzy inference systems and introduce them in the models as new actors. Finally, we are able to execute a simulation of the system, which allows us to perform an automatic optimization of the parameters of the sytem with the cross-entropy method and to test the suitability of the designed strategies. Based on the simulation results, we have assessed the good results yielded by the strategies and the potential of the modeling tool for the design and simulation of distributed software systems. But more importantly, this work demonstrates the usefulness of a control engineering approach to solve problems related to the dynamic behavior of software systems.Los sistemas distribuidos a gran escala son sistemas complejos que necesitan adaptarse a un entorno altamente dinámico y a las distintas necesidades del usuario. En este contexto, el objetivo principal de este proyecto es el desarrollo de nuevas estrategias de auto-adaptación, a la vez que las metodologías y herramientas necesarias para su análisis y diseño. En este trabajo, diseñamos e implementamos una arquitectura para capacidades autoadaptativas en sistemas software insipirada en la arquitectura Monitor- Analyze-Plan-Act over a Knowledge base de IBM, y desarrollamos nuevas estrategias de auto-adaptación específicas para redes de sensores inhalámbricas siguiendo una metodología tomada de la ingeniería de control. Más concretamente, comenzamos este trabajo diseñando en UML una librería software para el desarrollo de capacidades auto-adaptativas, que luego implementamos como un paquete Java. A continuación, modelamos dos sistemas distribuidos usando un enfoque orientado a actores en Ptolemy II. Posteriormente, desarrollamos estrategias auto-adaptativas basadas en sistemas de inferencia difusa y las insertamos en los modelos como nuevos actores. Finalmente, ejecutamos varias simulaciones del sistema, lo cual nos permite realizar una optimización automática de los parámetros del sistema mediante el uso del método de entropía cruzada y, además, probar el desempeño de las estrategias diseñadas. Basándonos en los resultados de estas simulaciones, hemos podido comprobar los buenos resultados que ofrecen las estrategias de auto-adaptación implementadas y el potencial de la herramienta de modelado para el diseño y la simulación de sistemas distribuidos. Pero lo más importante es que este trabajo demuestra la utilidad de enfocar desde la ingeniería de control la resolución de problemas relacionados con el comportamiento dinámico de sistemas software

A cell outage management framework for dense heterogeneous networks

In this paper, we present a novel cell outage management (COM) framework for heterogeneous networks with split control and data planes-a candidate architecture for meeting future capacity, quality-of-service, and energy efficiency demands. In such an architecture, the control and data functionalities are not necessarily handled by the same node. The control base stations (BSs) manage the transmission of control information and user equipment (UE) mobility, whereas the data BSs handle UE data. An implication of this split architecture is that an outage to a BS in one plane has to be compensated by other BSs in the same plane. Our COM framework addresses this challenge by incorporating two distinct cell outage detection (COD) algorithms to cope with the idiosyncrasies of both data and control planes. The COD algorithm for control cells leverages the relatively larger number of UEs in the control cell to gather large-scale minimization-of-drive-test report data and detects an outage by applying machine learning and anomaly detection techniques. To improve outage detection accuracy, we also investigate and compare the performance of two anomaly-detecting algorithms, i.e., k-nearest-neighbor- and local-outlier-factor-based anomaly detectors, within the control COD. On the other hand, for data cell COD, we propose a heuristic Grey-prediction-based approach, which can work with the small number of UE in the data cell, by exploiting the fact that the control BS manages UE-data BS connectivity and by receiving a periodic update of the received signal reference power statistic between the UEs and data BSs in its coverage. The detection accuracy of the heuristic data COD algorithm is further improved by exploiting the Fourier series of the residual error that is inherent to a Grey prediction model. Our COM framework integrates these two COD algorithms with a cell outage compensation (COC) algorithm that can be applied to both planes. Our COC solution utilizes an actor-critic-based reinforcement learning algorithm, which optimizes the capacity and coverage of the identified outage zone in a plane, by adjusting the antenna gain and transmission power of the surrounding BSs in that plane. The simulation results show that the proposed framework can detect both data and control cell outage and compensate for the detected outage in a reliable manner

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Routing, Localization And Positioning Protocols For Wireless Sensor And Actor Networks

Wireless sensor and actor networks (WSANs) are distributed systems of sensor nodes and actors that are interconnected over the wireless medium. Sensor nodes collect information about the physical world and transmit the data to actors by using one-hop or multi-hop communications. Actors collect information from the sensor nodes, process the information, take decisions and react to the events. This dissertation presents contributions to the methods of routing, localization and positioning in WSANs for practical applications. We first propose a routing protocol with service differentiation for WSANs with stationary nodes. In this setting, we also adapt a sports ranking algorithm to dynamically prioritize the events in the environment depending on the collected data. We extend this routing protocol for an application, in which sensor nodes float in a river to gather observations and actors are deployed at accessible points on the coastline. We develop a method with locally acting adaptive overlay network formation to organize the network with actor areas and to collect data by using locality-preserving communication. We also present a multi-hop localization approach for enriching the information collected from the river with the estimated locations of mobile sensor nodes without using positioning adapters. As an extension to this application, we model the movements of sensor nodes by a subsurface meandering current mobility model with random surface motion. Then we adapt the introduced routing and network organization methods to model a complete primate monitoring system. A novel spatial cut-off preferential attachment model and iii center of mass concept are developed according to the characteristics of the primate groups. We also present a role determination algorithm for primates, which uses the collection of spatial-temporal relationships. We apply a similar approach to human social networks to tackle the problem of automatic generation and organization of social networks by analyzing and assessing interaction data. The introduced routing and localization protocols in this dissertation are also extended with a novel three dimensional actor positioning strategy inspired by the molecular geometry. Extensive simulations are conducted in OPNET simulation tool for the performance evaluation of the proposed protocol

Dynamic distributed clustering in wireless sensor networks via Voronoi tessellation control

This paper presents two dynamic and distributed clustering algorithms for Wireless Sensor Networks (WSNs). Clustering approaches are used in WSNs to improve the network lifetime and scalability by balancing the workload among the clusters. Each cluster is managed by a cluster head (CH) node. The first algorithm requires the CH nodes to be mobile: by dynamically varying the CH node positions, the algorithm is proved to converge to a specific partition of the mission area, the generalised Voronoi tessellation, in which the loads of the CH nodes are balanced. Conversely, if the CH nodes are fixed, a weighted Voronoi clustering approach is proposed with the same load-balancing objective: a reinforcement learning approach is used to dynamically vary the mission space partition by controlling the weights of the Voronoi regions. Numerical simulations are provided to validate the approaches