Multi-agent reinforcement learning for route guidance system

Nowadays, multi-agent systems are used to create applications in a variety of areas, including economics, management, transportation, telecommunications, etc. Importantly, in many domains, the reinforcement learning agents try to learn a task by directly interacting with its environment. The main challenge in route guidance system is to direct vehicles to their destination in a dynamic traffic situation, with the aim of reducing travel times and ensuring efficient use of available road network capacity. This paper proposes a multi-agent reinforcement learning algorithm to find the best and shortest path between the origin and destination nodes. The shortest path such as the lowest cost is calculated using multi-agent reinforcement learning model and it will be suggested to the vehicle drivers in a route guidance system. The proposed algorithm has been evaluated based on Dijkstra's algorithm to find the optimal solution using Kuala Lumpur (KL) road network map. A number of route cases have been used to evaluate the proposed approach based on the road network problems. Finally, the experiment results demonstrate that the proposed approach is feasible and efficient

An OSA-CBM Multi-Agent Vehicle Health Management Architecture for Self-Health Awareness

Integrated Vehicle Health Management (IVHM) systems on modern aircraft or autonomous unmanned vehicles should provide diagnostic and prognostic capabilities with lower support costs and amount of data traffic. When mission objectives cannot be reached for the control system since unanticipated operating conditions exists, namely a failure, the mission plan must be revised or altered according to the health monitoring system assessment. Representation of the system health knowledge must facilitate interaction with the control system to compensate for subsystem degradation. Several generic architectures have been described for the implementation of health monitoring systems and their integration with the control system. In particular, the Open System Architecture - Condition-Based Maintenance (OSA-CBM) approach is considered in this work as initial point, and it is evolved in the sense of self-health awareness, by defining an appropriated multi-agent smart health management architecture based on smart device models, communication agents and a distributed control system. A case study about its application on fuel-cells as auxiliary power generator will demonstrate the integration.Postprint (published version

Intelligent Real-Time Decision Support Systems for Road Traffic Management. Multi-agent based Fuzzy Neural Networks with a GA learning approach in managing control actions of road traffic centres.

The selection of the most appropriate traffic control actions to solve non-recurrent traffic congestion is a complex task which requires significant expert knowledge and experience. In this thesis we develop and investigate the application of an intelligent traffic control decision support system for road traffic management to assist the human operator to identify the most suitable control actions in order to deal with non-recurrent and non-predictable traffic congestion in a real-time situation. Our intelligent system employs a Fuzzy Neural Networks (FNN) Tool that combines the capabilities of fuzzy reasoning in measuring imprecise and dynamic factors and the capabilities of neural networks in terms of learning processes. In this work we present an effective learning approach with regard to the FNN-Tool, which consists of three stages: initializing the membership functions of both input and output variables by determining their centres and widths using self-organizing algorithms; employing an evolutionary Genetic Algorithm (GA) based learning method to identify the fuzzy rules; tune the derived structure and parameters using the back-propagation learning algorithm. We evaluate experimentally the performance and the prediction capability of this three-stage learning approach using well-known benchmark examples. Experimental results demonstrate the ability of the learning approach to identify all relevant fuzzy rules from the training data. A comparative analysis shows that the proposed learning approach has a higher degree of predictive capability than existing models. We also address the scalability issue of our intelligent traffic control decision support system by using a multi-agent based approach. The large network is divided into sub-networks, each of which has its own associated agent. Finally, our intelligent traffic control decision support system is applied to a number of road traffic case studies using the traffic network in Riyadh, in Saudi Arabia. The results obtained are promising and show that our intelligent traffic control decision support system can provide an effective support for real-time traffic control

Decentralized multi-agent path finding for UAV traffic management

The development of a real-world Unmanned Aircraft System (UAS) Traffic Management (UTM) system to ensure the safe integration of Unmanned Aerial Vehicles (UAVs) in low altitude airspace, has recently generated novel research challenges. A key problem is the development of Pre-Flight Conflict Detection and Resolution (CDR) methods that provide collision-free flight paths to all UAVs before their takeoff. Such problem can be represented as a Multi-Agent Path Finding (MAPF) problem. Currently, most MAPF methods assume that the UTM system is a centralized entity in charge of CDR. However, recent discussions on UTM suggest that such centralized control might not be practical or desirable. Therefore, we explore Pre-Flight CDR methods where independent UAS Service Providers (UASSPs) with their own interests, communicate with each other to resolve conflicts among their UAV operations--without centralized UTM directives. We propose a novel MAPF model that supports the decentralized resolution of conflicts, whereby different `agents', here UASSPs, manage their UAV operations. We present two approaches: (1) a prioritization approach and (2) a simple yet practical pairwise negotiation approach where UASSPs agents determine an agreement to solve conflicts between their UAV operations. We evaluate the performance of our proposed approaches with simulation scenarios based on a consultancy study of predicted UAV traffic for delivery services in Sendai, Japan, 2030. We demonstrate that our negotiation approach improves the ``fairness'' between UASSPs, i.e. the distribution of costs between UASSPs in terms of total delays and rejected operations due to replanning is more balanced when compared to the prioritization approach

Engineering Agent Systems for Decision Support

This paper discusses how agent technology can be applied to the design of advanced Information Systems for Decision Support. In particular, it describes the different steps and models that are necessary to engineer Decision Support Systems based on a multiagent architecture. The approach is illustrated by a case study in the traffic management domain