On Learning in Collective Self-adaptive Systems: State of Practice and a 3D Framework

Collective self-adaptive systems (CSAS) are distributed and interconnected systems composed of multiple agents that can perform complex tasks such as environmental data collection, search and rescue operations, and discovery of natural resources. By providing individual agents with learning capabilities, CSAS can cope with challenges related to distributed sensing and decision-making and operate in uncertain environments. This unique characteristic of CSAS enables the collective to exhibit robust behaviour while achieving system-wide and agent-specific goals. Although learning has been explored in many CSAS applications, selecting suitable learning models and techniques remains a significant challenge that is heavily influenced by expert knowledge. We address this gap by performing a multifaceted analysis of existing CSAS with learning capabilities reported in the literature. Based on this analysis, we introduce a 3D framework that illustrates the learning aspects of CSAS considering the dimensions of autonomy, knowledge access, and behaviour, and facilitates the selection of learning techniques and models. Finally, using example applications from this analysis, we derive open challenges and highlight the need for research on collaborative, resilient and privacy-aware mechanisms for CSAS

A comprehensive survey on cooperative intersection management for heterogeneous connected vehicles

Nowadays, with the advancement of technology, world is trending toward high mobility and dynamics. In this context, intersection management (IM) as one of the most crucial elements of the transportation sector demands high attention. Today, road entities including infrastructures, vulnerable road users (VRUs) such as motorcycles, moped, scooters, pedestrians, bicycles, and other types of vehicles such as trucks, buses, cars, emergency vehicles, and railway vehicles like trains or trams are able to communicate cooperatively using vehicle-to-everything (V2X) communications and provide traffic safety, efficiency, infotainment and ecological improvements. In this paper, we take into account different types of intersections in terms of signalized, semi-autonomous (hybrid) and autonomous intersections and conduct a comprehensive survey on various intersection management methods for heterogeneous connected vehicles (CVs). We consider heterogeneous classes of vehicles such as road and rail vehicles as well as VRUs including bicycles, scooters and motorcycles. All kinds of intersection goals, modeling, coordination architectures, scheduling policies are thoroughly discussed. Signalized and semi-autonomous intersections are assessed with respect to these parameters. We especially focus on autonomous intersection management (AIM) and categorize this section based on four major goals involving safety, efficiency, infotainment and environment. Each intersection goal provides an in-depth investigation on the corresponding literature from the aforementioned perspectives. Moreover, robustness and resiliency of IM are explored from diverse points of view encompassing sensors, information management and sharing, planning universal scheme, heterogeneous collaboration, vehicle classification, quality measurement, external factors, intersection types, localization faults, communication anomalies and channel optimization, synchronization, vehicle dynamics and model mismatch, model uncertainties, recovery, security and privacy

Adaptive Planning and Prediction in Agent-Supported Distributed Collaboration.

Agents that act as user assistants will become invaluable as the number of information sources continue to proliferate. Such agents can support the work of users by learning to automate time-consuming tasks and filter information to manageable levels. Although considerable advances have been made in this area, it remains a fertile area for further development. One application of agents under careful scrutiny is the automated negotiation of conflicts between different user's needs and desires. Many techniques require explicit user models in order to function. This dissertation explores a technique for dynamically constructing user models and the impact of using them to anticipate the need for negotiation. Negotiation is reduced by including an advising aspect to the agent that can use this anticipation of conflict to adjust user behavior

Multi-platform coordination and resource management in command and control

Depuis plusieurs années, nous constatons l'augmentation de l'utilisation des techniques d'agents et multiagent pour assister l'humain dans ses tâches. Ce travail de maîtrise se situe dans la même voie. Précisément, nous proposons d'utiliser les techniques multiagent de planification et de coordination pour la gestion de ressources dans les systèmes de commande et contrôle (C2) temps réel. Le problème particulier que nous avons étudié est la conception d'un système d'aide à la décision pour les opérations anti-aérienne sur les frégates canadiennes. Dans le cas où plusieurs frégates doivent se défendre contre des menaces, la coordination est un problème d'importance capitale. L'utilisation de mécanismes de coordination efficaces permet d'éviter les actions conflictuelles et la redondance dans les engagements. Dans ce mémoire, nous présentons quatre mécanismes de coordination basés sur le partage de tâche. Trois sont basés sur les communications : la coordination centrale, le Contract Net, la coordination similaire à celle proposée par Brown; tandis que la défense de zone est basée sur les lois sociales. Nous exposons enfin les résultats auxquels nous sommes arrivés en simulant ces différents mécanismes.The use of agent and multiagent techniques to assist humans in their daily routines has been increasing for many years, notably in Command and Control (C2) systems. This thesis is is situated in this domain. Precisely, we propose to use multiagent planning and coordination techniques for resource management in real-time \acs{C2} systems. The particular problem we studied is the design of a decision-support for anti-air warfare on Canadian frigates. In the case of several frigates defending against incoming threats, multiagent coordination is a complex problem of capital importance. Better coordination mechanisms are important to avoid redundancy in engagements and inefficient defence caused by conflicting actions. In this thesis, we present four different coordination mechanisms based on task sharing. Three of these mechanisms are based on communications: central coordination, Contract Net coordination and Brown coordination, while the zone defence coordination is based on social laws. Finally, we expose the results obtained while simulating these various mechanisms

Improving Pan-African research and education networks through traffic engineering: A LISP/SDN approach

The UbuntuNet Alliance, a consortium of National Research and Education Networks (NRENs) runs an exclusive data network for education and research in east and southern Africa. Despite a high degree of route redundancy in the Alliance's topology, a large portion of Internet traffic between the NRENs is circuitously routed through Europe. This thesis proposes a performance-based strategy for dynamic ranking of inter-NREN paths to reduce latencies. The thesis makes two contributions: firstly, mapping Africa's inter-NREN topology and quantifying the extent and impact of circuitous routing; and, secondly, a dynamic traffic engineering scheme based on Software Defined Networking (SDN), Locator/Identifier Separation Protocol (LISP) and Reinforcement Learning. To quantify the extent and impact of circuitous routing among Africa's NRENs, active topology discovery was conducted. Traceroute results showed that up to 75% of traffic from African sources to African NRENs went through inter-continental routes and experienced much higher latencies than that of traffic routed within Africa. An efficient mechanism for topology discovery was implemented by incorporating prior knowledge of overlapping paths to minimize redundancy during measurements. Evaluation of the network probing mechanism showed a 47% reduction in packets required to complete measurements. An interactive geospatial topology visualization tool was designed to evaluate how NREN stakeholders could identify routes between NRENs. Usability evaluation showed that users were able to identify routes with an accuracy level of 68%. NRENs are faced with at least three problems to optimize traffic engineering, namely: how to discover alternate end-to-end paths; how to measure and monitor performance of different paths; and how to reconfigure alternate end-to-end paths. This work designed and evaluated a traffic engineering mechanism for dynamic discovery and configuration of alternate inter-NREN paths using SDN, LISP and Reinforcement Learning. A LISP/SDN based traffic engineering mechanism was designed to enable NRENs to dynamically rank alternate gateways. Emulation-based evaluation of the mechanism showed that dynamic path ranking was able to achieve 20% lower latencies compared to the default static path selection. SDN and Reinforcement Learning were used to enable dynamic packet forwarding in a multipath environment, through hop-by-hop ranking of alternate links based on latency and available bandwidth. The solution achieved minimum latencies with significant increases in aggregate throughput compared to static single path packet forwarding. Overall, this thesis provides evidence that integration of LISP, SDN and Reinforcement Learning, as well as ranking and dynamic configuration of paths could help Africa's NRENs to minimise latencies and to achieve better throughputs

Connected and Automated Vehicle Enabled Traffic Intersection Control with Reinforcement Learning

Recent advancements in vehicle automation have led to a proliferation of studies in traffic control strategies for the next generation of land vehicles. Current traffic signal based intersection control methods have significant limitations on dealing with rapidly evolving mobility, connectivity and social challenges. Figures for Europe over the period 2007-16 show that 20% of road accidents that have fatalities occur at intersections. Connected and Automated Mobility (CAM) presents a new paradigm for the integration of radically different traffic control methods into cities and towns for increased travel time efficiency and safety. Vehicle-to-Everything (V2X) connectivity between Intelligent Transportation System (ITS) users will make a significant contribution to transforming the current signalised traffic control systems into a more cooperative and reactive control system. This research work proposes a disruptive unsignalised traffic control method using a Reinforcement Learning (RL) algorithm to determine vehicle priorities at intersections and to schedule their crossing with the objectives of reducing congestion and increasing safety. Unlike heuristic rule-based methods, RL agents can learn the complex non-linear relationship between the elements that play a key role in traffic flow, from which an optimal control policy can be obtained. This work also focuses on the data requirements that inform Vehicle-to-Infrastructure (V2I) communication needs of such a system. The proposed traffic control method has been validated on a state-of-the-art simulation tool and a comparison of results with a traditional signalised control method indicated an up to 84% and 41% improvement in terms of reducing vehicle delay times and reducing fuel consumption respectively. In addition to computer simulations, practical experiments have also been conducted on a scaled road network with a single intersection and multiple scaled Connected and Automated Vehicles (CAV) to further validate the proposed control system in a representative but cost-effective setup. A strong correlation has been found between the computer simulation and practical experiment results. The outcome of this research work provides important insights into enabling cooperation between vehicles and traffic infrastructure via V2I communications, and integration of RL algorithms into a safety-critical control system

Machine learning based congestion control in wireless sensor networks

Master'sMASTER OF ENGINEERIN

Multi-Agent Systems

A multi-agent system (MAS) is a system composed of multiple interacting intelligent agents. Multi-agent systems can be used to solve problems which are difficult or impossible for an individual agent or monolithic system to solve. Agent systems are open and extensible systems that allow for the deployment of autonomous and proactive software components. Multi-agent systems have been brought up and used in several application domains