Image-based multi-agent reinforcement learning for demand–capacity balancing

Air traffic flow management (ATFM) is of crucial importance to the European Air Traffic Control System due to two factors: first, the impact of ATFM, including safety implications on ATC operations; second, the possible consequences of ATFM measures on both airports and airlines operations. Thus, the central flow management unit continually seeks to improve traffic flow management to reduce delays and congestion. In this work, we investigated the use of reinforcement learning (RL) methods to compute policies to solve demand–capacity imbalances (a.k.a. congestion) during the pre-tactical phase. To address cases where the expected demands exceed the airspace sector capacity, we considered agents representing flights who have to decide on ground delays jointly. To overcome scalability issues, we propose using raw pixel images as input, which can represent an arbitrary number of agents without changing the system’s architecture. This article compares deep Q-learning and deep deterministic policy gradient algorithms with different configurations. Experimental results, using real-world data for training and validation, confirm the effectiveness of our approach to resolving demand–capacity balancing problems, showing the robustness of the RL approach presented in this article.This work was funded by EUROCONTROL under Ph.D. Research contract no. 18-220569- C2 and by the Ministry of Economy, Industry, and Competitiveness of Spain under grant number PID2020-116377RB-C21.Peer ReviewedPostprint (published version

Multivariate study of vehicle exhaust particles using machine learning and statistical techniques

This research has examined the application of machine learning and statistical methods for developing roadside particle (number/mass concentrations) prediction models that can be used for air quality management. Data collected from continuous monitoring stations including pollutants, traffic and meteorological variables were used for training the models. A hybrid feature selection method involving Genetic Algorithms and Random Forests was successfully used in selecting the most relevant predictor variables for the models from the variables selected based on their correlation with the PM$_{10}$, PM$_{2.5}$ and PNC concentrations. The study found that the hybrid feature selection can be used with both statistical and machine learning methods to produce less expensive and more efficient air quality prediction models. Among the machine learning models studied the Boosted Regression Trees (BRT), Random Forests (RF), Extreme Learning Machines (ELM) and Deep Learning Algorithms were found to be the most suitable for the predictions of roadside PM$_{10}$, PM$_{2.5}$, and PNC concentrations. The machine learning models performed better than the ADMS-road model in spatiotemporal predictions involving monitoring sites locations. Moreover, they performed much better in predicting the concentrations in street Canyons. The ANN and BRT were found to be suitable for air quality management applications involving traffic management scenarios

Discussion on density-based clustering methods applied for automated identification of airspace flows

Air Traffic Management systems generate a huge amount of track data daily. Flight trajectories can be clustered to extract main air traffic flows by means of unsupervised machine learning techniques. A well-known methodology for unsupervised extraction of air traffic flows conducts a two-step process. The first step reduces the dimensionality of the track data, whereas the second step clusters the data based on a density-based algorithm, DBSCAN. This paper explores advancements in density-based clustering such as OPTICS or HDBSCAN*. This assessment is based on quantitative and qualitative evaluations of the clustering solutions offered by these algorithms. In addition, the paper proposes a hierarchical clustering algorithm for handling noise in this methodology. This algorithm is based on a recursive application of DBSCAN* (RDBSCAN*). The paper demonstrates the sensitivity of these algorithms to different hyper-parameters, recommending a specific setting for the main one, which is common for all methods. RDBSCAN* outperforms the other algorithms in terms of the density-based internal validity metric. Finally, the outcome of the clustering shows that the algorithm extracts main clusters of the dataset effectively, connecting outliers to these main clusters

On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features

Fatigue detection for air traffic controllers is an important yet challenging problem in aviation safety research. Most of the existing methods for this problem are based on facial features. In this paper, we propose an ensemble learning model that combines both facial features and voice features and design a fatigue detection method through multifeature fusion, referred to as Facial and Voice Stacking (FV-Stacking). Specifically, for facial features, we first use OpenCV and Dlib libraries to extract mouth and eye areas and then employ a combination of M-Convolutional Neural Network (M-CNN) and E-Convolutional Neural Network (E-CNN) to determine the state of mouth and eye closure based on five features, i.e., blinking times, average blinking time, average blinking interval, Percentage of Eyelid Closure over the Pupil over Time (PERCLOS), and Frequency of Open Mouth (FOM). For voice features, we extract the Mel-Frequency Cepstral Coefficients (MFCC) features of speech. Such facial features and voice features are fused through a carefully designed stacking model for fatigue detection. Real-life experiments are conducted on 14 air traffic controllers in Southwest Air Traffic Management Bureau of Civil Aviation of China. The results show that the proposed FV-Stacking method achieves a detection accuracy of 97%, while the best accuracy achieved by a single model is 92% and the best accuracy achieved by the state-of-the-art detection methods is 88%

Large-Scale Unmanned Aerial Systems Traffic Density Prediction and Management

In recent years, the applications of Unmanned Aerial Systems (UAS) has become more and more popular. We envision that in the near future, the complicated and high density UAS traffic will impose significant burden to air traffic management. Lot of works focus on the application development of individual Small Unmanned Aerial Systems (sUAS) or sUAS management Policy, however, the study of the UAS cluster behaviors such as forecasting and managing of the UAS traffic has generally not been addressed. In order to address the above issue, there is an urgent need to investigate three research directions. The first direction is to develop a high fidelity simulator for the UAS cluster behavior evaluation. The second direction to study real time trajectory planning algorithms to mitigate the high dense UAS traffic. The last direction is to investigate techniques that rapidly and accurately forecast the UAS traffic pattern in the future. In this thesis, we elaborate these three research topics and present a universal paradigm to predict and manage the traffic for the large-scale unmanned aerial systems. To enable the research in UAS traffic management and prediction, a Java based Multi-Agent Air Traffic and Resource Usage Simulation (MATRUS) framework is first developed. We use two types of UAS trajectories, Point-to-Point (P2P) and Man- hattan, as the case study to describe the capability of presented framework. Various communication and propagation models (i.e. log-distance-path loss) can be integrated with the framework to model the communication between UASs and base stations. The results show that MATRUS has the ability to evaluate different sUAS traffic management policies, and can provide insights on the relationships between air traf- fic and communication resource usage for further studies. Moreover, the framework can be extended to study the effect of sUAS Detect-and-Avoid (DAA) mechanisms, implement additional traffic management policies, and handle more complex traffic demands and geographical distributions. Based on the MATRUS framework, we propose a Sparse Represented Temporal- Spatial (SRTS) UAS trajectory planning algorithm. The SRTS algorithm allows the sUAS to avoid static no-fly areas (i.e. static obstacles) or other areas that have congested air traffic or communication traffic. The core functionality of the routing algorithm supports the instant refresh of the in-flight environment making it appropri- ate for highly dynamic air traffic scenarios. The characterization of the routing time and memory usage demonstrate that the SRTS algorithm outperforms a traditional Temporal-Spatial routing algorithm. The deep learning based approach has shown an outstanding success in many areas, we first investigated the possibility of applying the deep neural network in predicting the trajectory of a single vehicle in a given traffic scene. A new trajectory prediction model is developed, which allows information sharing among vehicles using a graph neural network. The prediction is based on the embedding feature, which is derived from multi-dimensional input sequences including the historical trajectory of target and neighboring vehicles, and their relative positions. Compared to other existing trajectory prediction methods, the proposed approach can reduce the pre- diction error by up to 50.00%. Then, we present a deep neural network model that extracts the features from both spatial and temporal domains to predict the UAS traffic density. In addition, a novel input representation of the future sUAS mission information is proposed. The pre-scheduled missions are categorized into 3 types according to their launching times. The results show that our presented model out- performs all of the baseline models. Meanwhile, the qualitative results demonstrate that our model can accurately predict the hot spot in the future traffic map

Prediction of Weather Impacts on Airport Arrival Meter Fix Capacity

This paper introduces a data driven model for predicting airport arrival capacity with a look-ahead time 2-8 hour forecast. The model is suitable for air traffic flow management by explicitly investigating the impact of convective weather on airport arrival meter fix throughput. Estimation of the arrival airport capacity under arrival meter fix flow constraints due to severe weather is an important part of Air Traffic Management (ATM). Airport arrival capacity can be reduced if one or more airport arrival meter fixes are partially or completely blocked by convective weather. When the predicted airport arrival demands exceed the predicted available airport's arrival capacity for a sustained period, Ground Delay Program (GDP) operations will be triggered by ATM system. Serious imbalances between demand and capacity occur most frequently when the airport capacity is severely degraded due to either bad airport terminal surface weather or inclement convective weather around airport arrival fixes. A model that predicts the weather-impacted airport arrival meter fix throughput may help ATM personnel to plan GDP operations more efficiently. This paper identifies the characteristics of air traffic flow across arrival meter fixes at Newark Liberty International Airport (EWR). The proposed approach, based on machine-learning methods, is developed to predict the weather impacted EWR arrival Meter Fix (MF) throughput. Sector forecast coverage is used to envision the weather impact on airport arrival MF flow, and the validation is accomplished by using Convective Weather Avoidance Model (CWAM) 0.5 to 2-hour and Collaborative Convective Forecast Product (CCFP) 4 to 8-hour look-ahead forecast data for the period of April-September in 2014. Furthermore, the regression tree ensemble learning of random forests approach for translating a sector forecast coverage model to an EWR arrival meter fix throughput model is examined. The results suggest that ATM decision makers in charge of MF flow control and GDP planning may benefit from adopting the airport arrival meter capacity prediction models to estimate the inclement weather impacts