Optimising arrival management in air traffic control

Efficient landing is a key component of improving air transport, both for passengers' experience and fuel consumption. With just two runways, Heathrow airport was running at 98% capacity before the pandemic. The existing queuing system allows buffer time for the aircraft to land on one runway, but this can add delays to journeys and be fuel inefficient. With the recovery of the travel industry after the 2020 pandemic, improving landing procedures remains a pertinent problem to NATS (who manages all the air traffic in UK airspace). In this thesis, we develop alternative methods to sequence aircraft as they approach for landing at Heathrow. In the first part of this thesis, we cast the arrival management problem in a reinforcement learning framework. We design a basic air traffic model and apply both table representation methods and nonlinear approximation with a neural network. Specifically, we compare the performance of Q-learning, SARSA, and DDPG on this environment. Further we explore dimension reduction/feature representation through path signatures. Finally we design multi-grid inspired neural network structures and see that these lead to faster training but ultimately comparable performance. For the second part, we look at the problem from a different perspective. We take inspiration from the theory of optimal transport and formulate an entropy-regularised optimisation problem. We design an algorithm with block gradient descent-like steps and note that the conflicts in the set-up of our problem introduces non-convexity even when working in the (convex) space of distributions over arrival times. By adding an additional 'considerate' cost, akin to a Pigouvian tax, the performance of the algorithm is enhanced. Finally, the last part of this thesis shows the flexibility of our approach. We adapt our work to apply to a new air traffic design concept being researched at NATS

Connected and Automated Vehicle Enabled Traffic Intersection Control with Reinforcement Learning

Recent advancements in vehicle automation have led to a proliferation of studies in traffic control strategies for the next generation of land vehicles. Current traffic signal based intersection control methods have significant limitations on dealing with rapidly evolving mobility, connectivity and social challenges. Figures for Europe over the period 2007-16 show that 20% of road accidents that have fatalities occur at intersections. Connected and Automated Mobility (CAM) presents a new paradigm for the integration of radically different traffic control methods into cities and towns for increased travel time efficiency and safety. Vehicle-to-Everything (V2X) connectivity between Intelligent Transportation System (ITS) users will make a significant contribution to transforming the current signalised traffic control systems into a more cooperative and reactive control system. This research work proposes a disruptive unsignalised traffic control method using a Reinforcement Learning (RL) algorithm to determine vehicle priorities at intersections and to schedule their crossing with the objectives of reducing congestion and increasing safety. Unlike heuristic rule-based methods, RL agents can learn the complex non-linear relationship between the elements that play a key role in traffic flow, from which an optimal control policy can be obtained. This work also focuses on the data requirements that inform Vehicle-to-Infrastructure (V2I) communication needs of such a system. The proposed traffic control method has been validated on a state-of-the-art simulation tool and a comparison of results with a traditional signalised control method indicated an up to 84% and 41% improvement in terms of reducing vehicle delay times and reducing fuel consumption respectively. In addition to computer simulations, practical experiments have also been conducted on a scaled road network with a single intersection and multiple scaled Connected and Automated Vehicles (CAV) to further validate the proposed control system in a representative but cost-effective setup. A strong correlation has been found between the computer simulation and practical experiment results. The outcome of this research work provides important insights into enabling cooperation between vehicles and traffic infrastructure via V2I communications, and integration of RL algorithms into a safety-critical control system

Task-Oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications

Communication system design has been traditionally guided by task-agnostic principles, which aim at efficiently transmitting as many correct bits as possible through a given channel. However, in the era of cyber-physical systems, the effectiveness of communications is not dictated simply by the bit rate, but most importantly by the efficient completion of the task in hand, e.g., controlling remotely a robot, automating a production line or collaboratively sensing through a drone swarm. In parallel, it is projected that by 2023, half of the worldwide network connections will be among machines rather than humans. In this context, it is crucial to establish a new paradigm for designing communication strategies for multi-agent cyber-physical systems. This is a daunting task, since it requires a combination of principles from information, communication, control theories and computer science in order to formalize a general framework for task-oriented communication designs. In this direction, this paper reviews and structures the relevant theoretical work across a wide range of scientific communities. Subsequently, it proposes a general conceptual framework for task-oriented communication design, along with its specializations according to targeted use cases. Furthermore, it provides a survey of relevant contributions in dominant applications, such as industrial internet of things, multi-unmanned aerial vehicle (UAV) systems, autonomous vehicles, distributed learning systems, smart manufacturing plants, 5G and beyond self-organizing networks, and tactile internet. Finally, this paper also highlights the most important open research topics from both the theoretical framework and application points of view

Cooperative Vehicle Tracking in Large Environments

Vehicle position tracking and prediction over large areas is of significant importance in many industrial applications, such as mining operations. In a small area, this can be easily achieved by providing vehicles with a constant communication link to a control centre and having the vehicles broadcast their position. The problem changes dramatically when vehicles operate within a large environment of potentially hundreds of square kilometres and in difficult terrain. This thesis presents algorithms for cooperative tracking of vehicles based on a vehicle motion model that incorporates the properties of the working area, and information collected by infrastructure collection points and other mobile agents. The probabilistic motion prediction approach provides long-term estimates of vehicle positions using motion profiles built for the particular environment and considering the vehicle stopping probability. A limited number of data collection points distributed around the field are used to update the position estimates, with negative information also used to improve the estimation. The thesis introduces the concept of observation harvesting, a process in which peer-to-peer communication between vehicles allows egocentric position updates and inter-vehicle measurements to be relayed among vehicles and finally conveyed to the collection points for an improved position estimate. It uses a store-and-synchronise concept to deal with intermittent communication and aims to disseminate data in an opportunistic manner. A nonparametric filtering algorithm for cooperative tracking is proposed to incorporate the information harvested, including the negative, relative, and time delayed observations. An important contribution of this thesis is to enable the optimisation of fleet scheduling when full coverage networks are not available or feasible. The proposed approaches were validated with comprehensive experimental results using data collected from a large-scale mining operation

Cooperative Vehicle Tracking in Large Environments

Vehicle position tracking and prediction over large areas is of significant importance in many industrial applications, such as mining operations. In a small area, this can be easily achieved by providing vehicles with a constant communication link to a control centre and having the vehicles broadcast their position. The problem changes dramatically when vehicles operate within a large environment of potentially hundreds of square kilometres and in difficult terrain. This thesis presents algorithms for cooperative tracking of vehicles based on a vehicle motion model that incorporates the properties of the working area, and information collected by infrastructure collection points and other mobile agents. The probabilistic motion prediction approach provides long-term estimates of vehicle positions using motion profiles built for the particular environment and considering the vehicle stopping probability. A limited number of data collection points distributed around the field are used to update the position estimates, with negative information also used to improve the estimation. The thesis introduces the concept of observation harvesting, a process in which peer-to-peer communication between vehicles allows egocentric position updates and inter-vehicle measurements to be relayed among vehicles and finally conveyed to the collection points for an improved position estimate. It uses a store-and-synchronise concept to deal with intermittent communication and aims to disseminate data in an opportunistic manner. A nonparametric filtering algorithm for cooperative tracking is proposed to incorporate the information harvested, including the negative, relative, and time delayed observations. An important contribution of this thesis is to enable the optimisation of fleet scheduling when full coverage networks are not available or feasible. The proposed approaches were validated with comprehensive experimental results using data collected from a large-scale mining operation