SOARNET, Deep Learning Thermal Detection For Free Flight

Thermals are regions of rising hot air formed on the ground through the warming of the surface by the sun. Thermals are commonly used by birds and glider pilots to extend flight duration, increase cross-country distance, and conserve energy. This kind of powerless flight using natural sources of lift is called soaring. Once a thermal is encountered, the pilot flies in circles to keep within the thermal, so gaining altitude before flying off to the next thermal and towards the destination. A single thermal can net a pilot thousands of feet of elevation gain, however estimating thermal locations is not an easy task. Pilots look for different indicators: color variation on the ground because the difference in the amount of heat absorbed by the ground varies based on the color/composition, birds circling in an area gaining lift, and certain types of cloud formations (cumulus clouds). The above methods are not always reliable enough and pilots study the weather for thermals by estimating solar heating of the ground using cloud cover and time of year and the lapse rate and dew point of the troposphere. In this paper, we present a Machine Learning based solution for assisting in forecasting thermals. We created a custom dataset using flight data recorded and uploaded to public databases by soaring pilots. We determine where and when the pilot encountered thermals to pull weather and satellite images corresponding to the location and time of the flight. Using this dataset we train an algorithm to automatically predict the location of thermals given as input the current weather conditions and terrain information obtained from Google Earth Engine and thermal regions encountered as truth labels. We were able to converge very well on the training and validation set, proving our method with around a 0.98 F1 score. These results indicate success in creating a custom dataset and a powerful neural network with the necessity of bolstering our custom dataset

Uncertainty-based sensor fusion architecture using Bayesian-LSTM neural network

Uncertainty-based sensor management for positioning is an essential component in safe drone operations inside urban environments with large urban valleys. These canyons significantly restrict the Line-Of-Sight signal conditions required for accurate positioning using Global Navigation Satellite Systems (GNSS). Therefore, sensor fusion solutions need to be in place whichcan take advantage of alternative Positioning, Navigation and Timing (PNT) sensors such as accelerometers or gyroscopes to complement GNSS information. Recent stateof-art research has focused on Machine Learning (ML) techniques such as Support Vector Machines (SVM) that utilize statistical learning to provide an output for a given input. However, understanding the uncertainty of these predictions made by Deep Learning (DL) models can help improve integrity of fusion systems. Therefore, there is a need for a DL model that can also provide uncertainty-related information as the output. This paper proposes a Bayesian-LSTM Neural Network (BLSTMNN) that is used to fuse GNSS and Inertial Measurement Unit (IMU) data. Furthermore, Protection Level (PL) is estimated based on the uncertainty distribution given by the system. To test the algorithm, Hardware-In-the-Loop (HIL) simulationhas been performed,utilizingSpirent’s GSS7000 simulator and OKTAL-SE Sim3D to simulate GNSS propagation in urban canyons. SimSENSOR is used to simulate the accelerometer and gyroscope. Results show that Bayesian-LSTM provides the best fusion performance compared to GNSS alone, and GNSS/IMU fusion using EKF and SVM. Furthermore, regarding uncertainty estimates, the proposed algorithm can estimate the positioning boundaries correctly, with an error rate of 0.4% and with an accuracy of 99.6%

AMANDA : density-based adaptive model for nonstationary data under extreme verification latency scenarios

Gradual concept-drift refers to a smooth and gradual change in the relations between input and output data in the underlying distribution over time. The problem generates a model obsolescence and consequently a quality decrease in predictions. Besides, there is a challenging task during the stream: The extreme verification latency (EVL) to verify the labels. For batch scenarios, state-of-the-art methods propose an adaptation of a supervised model by using an unconstrained least squares importance fitting (uLSIF) algorithm or a semi-supervised approach along with a core support extraction (CSE) method. However, these methods do not properly tackle the mentioned problems due to their high computational time for large data volumes, lack in representing the right samples of the drift or even for having several parameters for tuning. Therefore, we propose a density-based adaptive model for nonstationary data (AMANDA), which uses a semi-supervised classifier along with a CSE method. AMANDA has two variations: AMANDA with a fixed cutting percentage (AMANDA-FCP); and AMANDA with a dynamic cutting percentage (AMANDADCP). Our results indicate that the two variations of AMANDA outperform the state-of-the-art methods for almost all synthetic datasets and real ones with an improvement up to 27.98% regarding the average error. We have found that the use of AMANDA-FCP improved the results for a gradual concept-drift even with a small size of initial labeled data. Moreover, our results indicate that SSL classifiers are improved when they work along with our static or dynamic CSE methods. Therefore, we emphasize the importance of research directions based on this approach.Concept-drift gradual refere-se à mudança suave e gradual na distribuição dos dados conforme o tempo passa. Este problema causa obsolescência no modelo de aprendizado e queda na qualidade das previsões. Além disso, existe um complicador durante o processamento dos dados: a latência de verificação extrema (LVE) para se verificar os rótulos. Métodos do estado da arte propõem uma adaptação do modelo supervisionado usando uma abordagem de estimação de importância baseado em mínimos quadrados ou usando uma abordagem semi-supervisionada em conjunto com a extração de instâncias centrais, na sigla em inglês (CSE). Entretanto, estes métodos não tratam adequadamente os problemas mencionados devido ao fato de requererem alto tempo computacional para processar grandes volumes de dados, falta de correta seleção das instâncias que representam a mudança da distribuição, ou ainda por demandarem o ajuste de grande quantidade de parâmetros. Portanto, propomos um modelo adaptativo baseado em densidades para dados não-estacionários (AMANDA), que tem como base um classificador semi-supervisionado e um método CSE baseado em densidade. AMANDA tem duas variações: percentual de corte fixo (AMANDAFCP); e percentual de corte dinâmico (AMANDA-DCP). Nossos resultados indicam que as duas variações da proposta superam o estado da arte em quase todas as bases de dados sintéticas e reais em até 27,98% em relação ao erro médio. Concluímos que a aplicação do método AMANDA-FCP faz com que a classificação melhore mesmo quando há uma pequena porção inicial de dados rotulados. Mais ainda, os classificadores semi-supervisionados são melhorados quando trabalham em conjunto com nossos métodos de CSE, estático ou dinâmico

A hybrid deep learning approach for robust multi-sensor GNSS/INS/VO fusion in urban canyons

This paper addresses the significant challenges of robust autonomous navigation in Unmanned Aerial Vehicles (UAVs) within densely populated environments. The focus is on enhancing the performance of Position, Navigation, and Timing (PNT), as specified by the International Civil Aviation Organization, in terms of accuracy, integrity, continuity, and availability. The novel contribution introduces a Robust Multi-Sensor Fusion Architecture (RMSFA) that utilizes a Bayesian-LSTM machine learning algorithm, fusing GNSS, INS, and monocular odometry. Unlike existing solutions that rely on sensor redundancies or methods such as Receiver Autonomous Integrity Monitoring (RAIM), which have limitations in performance, or adaptability to erroneous signals, the proposed system offers improvements in both positioning accuracy and integrity. Furthermore, GNSS data is preprocessed to remove NoneLine-of-Sight data (NLOS) to improve positioning accuracy. Additionally, INS data errors are corrected using a GRU-based error correction architecture to improve INS positioning and reduce drifting. The addition of these post-processing steps reduced the 95th percentile horizontal error by 97.4% and 71.5% respectively. A CNN-LSTM architecture is used to obtain a Visual Odometer (VO) from the camera sensor. The Bayesian-LSTM architecture fusion performance was then compared to a GNSS/IMU/VO EKF-GRU architecture. The comparison showed a 95th percentile error improvement in the horizontal direction of 30.1% for the BayesianLSTM. The architecture was tested in a realistic simulated environment utilizing Unreal Engine and AirSim for UAV simulation, Spirent GNSS7000 simulator for Hardware-in-the-Loop (HIL) simulation, and OKTAL-SE Sim3D to mimic the effects of multipath on GNSS signals. Overall, this work represents a step toward improving the safety and effectiveness of drone navigation by providing a more robust navigation system suitable for safety-critical situations, without the stated disadvantages in previously mentioned literatures

Design and Implementation of an Artificial Neural Network Controller for Quadrotor Flight in Confined Environment

Quadrotors offer practical solutions for many applications, such as emergency rescue, surveillance, military operations, videography and many more. For this reason, they have recently attracted the attention of research and industry. Even though they have been intensively studied, quadrotors still suffer from some challenges that limit their use, such as trajectory measurement, attitude estimation, obstacle avoidance, safety precautions, and land cybersecurity. One major problem is flying in a confined environment, such as closed buildings and tunnels, where the aerodynamics around the quadrotor are affected by close proximity objects, which result in tracking performance deterioration, and sometimes instability. To address this problem, researchers followed three different approaches; the Modeling approach, which focuses on the development of a precise dynamical model that accounts for the different aerodynamic effects, the Sensor Integration approach, which focuses on the addition of multiple sensors to the quadrotor and applying algorithms to stabilize the quadrotor based on their measurements, and the Controller Design approach, which focuses on the development of an adaptive and robust controller. In this research, a learning controller is proposed as a solution for the issue of quadrotor trajectory control in confined environments. This controller utilizes Artificial Neural Networks to adjust for the unknown aerodynamics on-line. A systematic approach for controller design is developed, so that, the approach could be followed for the development of controllers for other nonlinear systems of similar form. One goal for this research is to develop a global controller that could be applied to any quadrotor with minimal adjustment. A novel Artificial Neural Network structure is presented that increases learning efficiency and speed. In addition, a new learning algorithm is developed for the Artificial Neural Network, when utilized with the developed controller. Simulation results for the designed controller when applied to the Qball-X4 quadrotor are presented that show the effectiveness of the proposed Artificial Neural Network structure and the developed learning algorithm in the presence of variety of different unknown aerodynamics. These results are confirmed with real time experimentation, as the developed controller was successfully applied to Quanser’s Qball-X4 quadrotor for the flight control in confined environment. The practical challenges associated with the application of such a controller for quadrotor flight in confined environment are analyzed and adequately resolved to achieve an acceptable tracking performance

Advances in Automated Driving Systems

Electrification, automation of vehicle control, digitalization and new mobility are the mega-trends in automotive engineering, and they are strongly connected. While many demonstrations for highly automated vehicles have been made worldwide, many challenges remain in bringing automated vehicles to the market for private and commercial use. The main challenges are as follows: reliable machine perception; accepted standards for vehicle-type approval and homologation; verification and validation of the functional safety, especially at SAE level 3+ systems; legal and ethical implications; acceptance of vehicle automation by occupants and society; interaction between automated and human-controlled vehicles in mixed traffic; human–machine interaction and usability; manipulation, misuse and cyber-security; the system costs of hard- and software and development efforts. This Special Issue was prepared in the years 2021 and 2022 and includes 15 papers with original research related to recent advances in the aforementioned challenges. The topics of this Special Issue cover: Machine perception for SAE L3+ driving automation; Trajectory planning and decision-making in complex traffic situations; X-by-Wire system components; Verification and validation of SAE L3+ systems; Misuse, manipulation and cybersecurity; Human–machine interactions, driver monitoring and driver-intention recognition; Road infrastructure measures for the introduction of SAE L3+ systems; Solutions for interactions between human- and machine-controlled vehicles in mixed traffic