Artificial Intelligence based multi-agent control system

Le metodologie di Intelligenza Artificiale (AI) si occupano della possibilità di rendere le macchine in grado di compiere azioni intelligenti con lo scopo di aiutare l’essere umano; quindi è possibile affermare che l’Intelligenza Artificiale consente di portare all’interno delle macchine, caratteristiche tipiche considerate come caratteristiche umane. Nello spazio dell’Intelligenza Artificiale ci sono molti compiti che potrebbero essere richiesti alla macchina come la percezione dell’ambiente, la percezione visiva, decisioni complesse. La recente evoluzione in questo campo ha prodotto notevoli scoperte, princi- palmente in sistemi ingegneristici come sistemi multi-agente, sistemi in rete, impianti, sistemi veicolari, sistemi sanitari; infatti una parte dei suddetti sistemi di ingegneria è presente in questa tesi di dottorato. Lo scopo principale di questo lavoro è presentare le mie recenti attività di ricerca nel campo di sistemi complessi che portano le metodologie di intelligenza artifi- ciale ad essere applicati in diversi ambienti, come nelle reti di telecomunicazione, nei sistemi di trasporto e nei sistemi sanitari per la Medicina Personalizzata. Gli approcci progettati e sviluppati nel campo delle reti di telecomunicazione sono presentati nel Capitolo 2, dove un algoritmo di Multi Agent Reinforcement Learning è stato progettato per implementare un approccio model-free al fine di controllare e aumentare il livello di soddisfazione degli utenti; le attività di ricerca nel campo dei sistemi di trasporto sono presentate alla fine del capitolo 2 e nel capitolo 3, in cui i due approcci riguardanti un algoritmo di Reinforcement Learning e un algoritmo di Deep Learning sono stati progettati e sviluppati per far fronte a soluzioni di viaggio personalizzate e all’identificazione automatica dei mezzi trasporto; le ricerche svolte nel campo della Medicina Personalizzata sono state presentate nel Capitolo 4 dove è stato presentato un approccio basato sul controllo Deep Learning e Model Predictive Control per affrontare il problema del controllo dei fattori biologici nei pazienti diabetici.Artificial Intelligence (AI) is a science that deals with the problem of having machines perform intelligent, complex, actions with the aim of helping the human being. It is then possible to assert that Artificial Intelligence permits to bring into machines, typical characteristics and abilities that were once limited to human intervention. In the field of AI there are several tasks that ideally could be delegated to machines, such as environment aware perception, visual perception and complex decisions in the various field. The recent research trends in this field have produced remarkable upgrades mainly on complex engineering systems such as multi-agent systems, networked systems, manufacturing, vehicular and transportation systems, health care; in fact, a portion of the mentioned engineering system is discussed in this PhD thesis, as most of them are typical field of application for traditional control systems. The main purpose if this work is to present my recent research activities in the field of complex systems, bringing artificial intelligent methodologies in different environments such as in telecommunication networks, transportation systems and health care for Personalized Medicine. The designed and developed approaches in the field of telecommunication net- works is presented in Chapter 2, where a multi-agent reinforcement learning algorithm was designed to implement a model-free control approach in order to regulate and improve the level of satisfaction of the users, while the research activities in the field of transportation systems are presented at the end of Chapter 2 and in Chapter 3, where two approaches regarding a Reinforcement Learning algorithm and a Deep Learning algorithm were designed and developed to cope with tailored travels and automatic identification of transportation moralities. Finally, the research activities performed in the field of Personalized Medicine have been presented in Chapter 4 where a Deep Learning and Model Predictive control based approach are presented to address the problem of controlling biological factors in diabetic patients

Adaptive Railway Traffic Control using Approximate Dynamic Programming

Railway networks around the world have become challenging to operate in recent decades, with a mixture of track layouts running several different classes of trains with varying operational speeds. This complexity has come about as a result of the sustained increase in passenger numbers where in many countries railways are now more popular than ever before as means of commuting to cities. To address operational challenges, governments and railway undertakings are encouraging development of intelligent and digital transport systems to regulate and optimise train operations in real-time to increase capacity and customer satisfaction by improved usage of existing railway infrastructure. Accordingly, this thesis presents an adaptive railway traffic control system for realtime operations based on a data-based approximate dynamic programming (ADP) approach with integrated reinforcement learning (RL). By assessing requirements and opportunities, the controller aims to reduce delays resulting from trains that entered a control area behind schedule by re-scheduling control plans in real-time at critical locations in a timely manner. The present data-based approach depends on an approximation to the value function of dynamic programming after optimisation from a specified state, which is estimated dynamically from operational experience using RL techniques. By using this approximation, ADP avoids extensive explicit evaluation of performance and so reduces the computational burden substantially. In this thesis, formulations of the approximation function and variants of the RL learning techniques used to estimate it are explored. Evaluation of this controller shows considerable improvements in delays by comparison with current industry practices

A literature review of Artificial Intelligence applications in railway systems

Nowadays it is widely accepted that Artificial Intelligence (AI) is significantly influencing a large number of domains, including railways. In this paper, we present a systematic literature review of the current state-of-the-art of AI in railway transport. In particular, we analysed and discussed papers from a holistic railway perspective, covering sub-domains such as maintenance and inspection, planning and management, safety and security, autonomous driving and control, revenue management, transport policy, and passenger mobility. This review makes an initial step towards shaping the role of AI in future railways and provides a summary of the current focuses of AI research connected to rail transport. We reviewed about 139 scientific papers covering the period from 2010 to December 2020. We found that the major research efforts have been put in AI for rail maintenance and inspection, while very limited or no research has been found on AI for rail transport policy and revenue management. The remaining sub-domains received mild to moderate attention. AI applications are promising and tend to act as a game-changer in tackling multiple railway challenges. However, at the moment, AI research in railways is still mostly at its early stages. Future research can be expected towards developing advanced combined AI applications (e.g. with optimization), using AI in decision making, dealing with uncertainty and tackling newly rising cybersecurity challenges

Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges

Machine learning (ML) is widely used for key tasks in Connected and Automated Vehicles (CAV), including perception, planning, and control. However, its reliance on vehicular data for model training presents significant challenges related to in-vehicle user privacy and communication overhead generated by massive data volumes. Federated learning (FL) is a decentralized ML approach that enables multiple vehicles to collaboratively develop models, broadening learning from various driving environments, enhancing overall performance, and simultaneously securing local vehicle data privacy and security. This survey paper presents a review of the advancements made in the application of FL for CAV (FL4CAV). First, centralized and decentralized frameworks of FL are analyzed, highlighting their key characteristics and methodologies. Second, diverse data sources, models, and data security techniques relevant to FL in CAVs are reviewed, emphasizing their significance in ensuring privacy and confidentiality. Third, specific and important applications of FL are explored, providing insight into the base models and datasets employed for each application. Finally, existing challenges for FL4CAV are listed and potential directions for future work are discussed to further enhance the effectiveness and efficiency of FL in the context of CAV

Developing Train Station Parking Algorithms: New Frameworks Based on Fuzzy Reinforcement Learning

Train station parking (TSP) accuracy is important to enhance the efficiency of train operation and the safety of passengers for urban rail transit. However, TSP is always subject to a series of uncertain factors such as extreme weather and uncertain conditions of rail track resistances. To increase the parking accuracy, robustness, and self-learning ability, we propose new train station parking frameworks by using the reinforcement learning (RL) theory combined with the information of balises. Three algorithms were developed, involving a stochastic optimal selection algorithm (SOSA), a Q-learning algorithm (QLA), and a fuzzy function based Q-learning algorithm (FQLA) in order to reduce the parking error in urban rail transit. Meanwhile, five braking rates are adopted as the action vector of the three algorithms and some statistical indices are developed to evaluate parking errors. Simulation results based on real-world data show that the parking errors of the three algorithms are all within the "mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1"""mml:mrow""mml:mo"±"/mml:mo""/mml:mrow""/mml:math"30cm, which meet the requirement of urban rail transit. Document type: Articl

Applications of Genetic Algorithm and Its Variants in Rail Vehicle Systems: A Bibliometric Analysis and Comprehensive Review

Railway systems are time-varying and complex systems with nonlinear behaviors that require effective optimization techniques to achieve optimal performance. Evolutionary algorithms methods have emerged as a popular optimization technique in recent years due to their ability to handle complex, multi-objective issues of such systems. In this context, genetic algorithm (GA) as one of the powerful optimization techniques has been extensively used in the railway sector, and applied to various problems such as scheduling, routing, forecasting, design, maintenance, and allocation. This paper presents a review of the applications of GAs and their variants in the railway domain together with bibliometric analysis. The paper covers highly cited and recent studies that have employed GAs in the railway sector and discuss the challenges and opportunities of using GAs in railway optimization problems. Meanwhile, the most popular hybrid GAs as the combination of GA and other evolutionary algorithms methods such as particle swarm optimization (PSO), ant colony optimization (ACO), neural network (NN), fuzzy-logic control, etc with their dedicated application in the railway domain are discussed too. More than 250 publications are listed and classified to provide a comprehensive analysis and road map for experts and researchers in the field helping them to identify research gaps and opportunities

A Survey on Control Methods for Virtual Coupling in Railway Operation

In order to meet the rapid growth of railway transportation demand, the technology that can improve its capacity has been widely concerned. Virtual Coupling (VC) is a new technology to improve capacity by decreasing headways between successive trains. Its basic principle is to control the train formation operation in coordination with the goal of keeping the same speed under the support of data transmission technology such as train-to-train communication. Our paper first reviews the existing theoretical research on train formation operation control, which is divided into four categories: train following, feedback control, optimal control and computational intelligent method. Secondly, by reviewing the related projects in Europe and China, based on the scenario analysis method, five general scenarios and two emergency scenarios in the whole process of VC operation are sorted out. Then, an original, complete list of performance indicators (PIs) is proposed for evaluating the performance of VC in different scenarios. Additionally, the paper gives a brief insight into VC operation by providing an adequate context to understand the proposed PIs