Flight Performance Analysis of an Image Processing Algorithm for Integrated Sense-and-Avoid Systems

This paper is focused on the development and the flight performance analysis of an image-processing technique aimed at detecting flying obstacles in airborne panchromatic images. It was developed within the framework of a research project which aims at realizing a prototypical obstacle detection and identification System, characterized by a hierarchical multisensor configuration. This configuration comprises a radar, that is, the main sensor, and four electro-optical cameras. Cameras are used as auxiliary sensors to the radar, in order to increase intruder aircraft position measurement, in terms of accuracy and data rate. The paper thoroughly describes the selection and customization of the developed image-processing techniques in order to guarantee the best results in terms of detection range, missed detection rate, and false-alarm rate. Performance is evaluated on the basis of a large amount of images gathered during flight tests with an intruder aircraft. The improvement in terms of accuracy and data rate, compared with radar-only tracking, is quantitatively demonstrated

PCA-based line detection from range data for mapping and localization-aiding of UAVs

This paper presents an original technique for robust detection of line features from range data, which is also the core element of an algorithm conceived for mapping 2D environments. A new approach is also discussed to improve the accuracy of position and attitude estimates of the localization by feeding back angular information extracted from the detected edges in the updating map. The innovative aspects of the line detection algorithm regard the proposed hierarchical clusterization method for segmentation. Instead, line fitting is carried out by exploiting the Principal Component Analysis, unlike traditional techniques relying on Least Squares linear regression. Numerical simulations are purposely conceived to compare these approaches for line fitting. Results demonstrate the applicability of the proposed technique as it provides comparable performance in terms of computational load and accuracy compared to the least squares method. Also, performance of the overall line detection architecture, as well as of the solutions proposed for line-based mapping and localization-aiding is evaluated exploiting real range data acquired in indoor environments using an UTM-30LX-EW 2D LIDAR. This paper lies in the framework of autonomous navigation of unmanned vehicles moving in complex 2D areas, e.g. unexplored, full of obstacles, GPS-challenging or denied

Radar/electro-optical data fusion for non-cooperative UAS sense and avoid

Abstract This paper focuses on hardware/software implementation and flight results relevant to a multi-sensor obstacle detection and tracking system based on radar/electro-optical (EO) data fusion. The sensing system was installed onboard an optionally piloted very light aircraft (VLA). Test flights with a single intruder plane of the same class were carried out to evaluate the level of achievable situational awareness and the capability to support autonomous collision avoidance. System architecture is presented and special emphasis is given to adopted solutions regarding real time integration of sensors and navigation measurements and high accuracy estimation of sensors alignment. On the basis of Global Positioning System (GPS) navigation data gathered simultaneously with multi-sensor tracking flight experiments, potential of radar/EO fusion is compared with standalone radar tracking. Flight results demonstrate a significant improvement of collision detection performance, mostly due to the change in angular rate estimation accuracy, and confirm data fusion effectiveness for facing EO detection issues. Relative sensors alignment, performance of the navigation unit, and cross-sensor cueing are found to be key factors to fully exploit the potential of multi-sensor architectures

An Innovative Procedure for Calibration of Strapdown Electro-Optical Sensors Onboard Unmanned Air Vehicles

This paper presents an innovative method for estimating the attitude of airborne electro-optical cameras with respect to the onboard autonomous navigation unit. The procedure is based on the use of attitude measurements under static conditions taken by an inertial unit and carrier-phase differential Global Positioning System to obtain accurate camera position estimates in the aircraft body reference frame, while image analysis allows line-of-sight unit vectors in the camera based reference frame to be computed. The method has been applied to the alignment of the visible and infrared cameras installed onboard the experimental aircraft of the Italian Aerospace Research Center and adopted for in-flight obstacle detection and collision avoidance. Results show an angular uncertainty on the order of 0.1° (rms)

Real-Time Hardware-in-the-Loop Laboratory Testing for Multisensor Sense and Avoid Systems

This paper focuses on a hardware-in-the-loop facility aimed at real-time testing of architectures and algorithms of multisensor sense and avoid systems. It was developed within a research project aimed at flight demonstration of autonomous non-cooperative collision avoidance for Unmanned Aircraft Systems. In this framework, an optionally piloted Very Light Aircraft was used as experimental platform. The flight system is based on multiple-sensor data integration and it includes a Ka-band radar, four electro-optical sensors, and two dedicated processing units. The laboratory test system was developed with the primary aim of prototype validation before multi-sensor tracking and collision avoidance flight tests. System concept, hardware/software components, and operating modes are described in the paper. The facility has been built with a modular approach including both flight hardware and simulated systems and can work on the basis of experimentally tested or synthetically generated scenarios. Indeed, hybrid operating modes are also foreseen which enable performance assessment also in the case of alternative sensing architectures and flight scenarios that are hardly reproducible during flight tests. Real-time multisensor tracking results based on flight data are reported, which demonstrate reliability of the laboratory simulation while also showing the effectiveness of radar/electro-optical fusion in a non-cooperative collision avoidance architecture

Particle Filtering for Obstacle Tracking in UAS Sense and Avoid Applications

Obstacle detection and tracking is a key function for UAS sense and avoid applications. In fact, obstacles in the flight path must be detected and tracked in an accurate and timely manner in order to execute a collision avoidance maneuver in case of collision threat. The most important parameter for the assessment of a collision risk is the Distance at Closest Point of Approach, that is, the predicted minimum distance between own aircraft and intruder for assigned current position and speed. Since assessed methodologies can cause some loss of accuracy due to nonlinearities, advanced filtering methodologies, such as particle filters, can provide more accurate estimates of the target state in case of nonlinear problems, thus improving system performance in terms of collision risk estimation. The paper focuses on algorithm development and performance evaluation for an obstacle tracking system based on a particle filter. The particle filter algorithm was tested in off-line simulations based on data gathered during flight tests. In particular, radar-based tracking was considered in order to evaluate the impact of particle filtering in a single sensor framework. The analysis shows some accuracy improvements in the estimation of Distance at Closest Point of Approach, thus reducing the delay in collision detection

From the Editors of the Special Issue on Urban Air Mobility and UAS Airspace Integration: Vision, Challenges, and Enabling Avionics Technologies

The integration of unmanned aircraft systems (UAS) in all classes of airspace represents, at the same time, an evolutionary and a revolutionary step in air transport operations. As a result, new concepts have emerged for UAS traffic management to support the anticipated traffic density growth and the need for safe beyond visual line-of-sight operations. Closely linked with these developments, urban/advanced air mobility (UAM/AAM) has appeared as a new and disruptive dimension for aviation, potentially enabling mobility of goods and people at a different scale compared with current operations, while also emphasizing the need of seamless integration with the existing air traffic management (ATM) framework. These UAS capabilities are reshaping the future of aviation, but also challenge traditional paradigms, requiring significant advances both in technologies and regulations, while keeping strong links with public communities and the perception of societal benefits. As an example, a key role is played by the progress of communications, navigation and surveillance technologies, such as sense-and-avoid and global navigation satellite systems-resilient, alternate position, navigation, and timing systems, and by the seamless integration of airborne and ground infrastructure within a cyber-aware context. Similarly, significant restructuring of the existing regulatory framework is needed to ensure that the integrity and safety of the AAM/ATM integrated airspace is maintained while enabling autonomous operations with higher technological flexibility and refresh rates

Real Time Corner Detection for Miniaturized Electro-Optical Sensors Onboard Small Unmanned Aerial Systems

This paper describes the target detection algorithm for the image processor of a vision-based system that is installed onboard an unmanned helicopter. It has been developed in the framework of a project of the French national aerospace research center Office National d’Etudes et de Recherches Aérospatiales (ONERA) which aims at developing an air-to-ground target tracking mission in an unknown urban environment. In particular, the image processor must detect targets and estimate ground motion in proximity of the detected target position. Concerning the target detection function, the analysis has dealt with realizing a corner detection algorithm and selecting the best choices in terms of edge detection methods, filtering size and type and the more suitable criterion of detection of the points of interest in order to obtain a very fast algorithm which fulfills the computation load requirements. The compared criteria are the Harris-Stephen and the Shi-Tomasi, ones, which are the most widely used in literature among those based on intensity. Experimental results which illustrate the performance of the developed algorithm and demonstrate that the detection time is fully compliant with the requirements of the real-time system are discussed

Real-Time Hardware-in-the-Loop Laboratory Testing for Multisensor Sense and Avoid Systems

This paper focuses on a hardware-in-the-loop facility aimed at real-time testing of architectures and algorithms of multisensor sense and avoid systems. It was developed within a research project aimed at flight demonstration of autonomous non-cooperative collision avoidance for Unmanned Aircraft Systems. In this framework, an optionally piloted Very Light Aircraft was used as experimental platform. The flight system is based on multiple-sensor data integration and it includes a Ka-band radar, four electro-optical sensors, and two dedicated processing units. The laboratory test system was developed with the primary aim of prototype validation before multi-sensor tracking and collision avoidance flight tests. System concept, hardware/software components, and operating modes are described in the paper. The facility has been built with a modular approach including both flight hardware and simulated systems and can work on the basis of experimentally tested or synthetically generated scenarios. Indeed, hybrid operating modes are also foreseen which enable performance assessment also in the case of alternative sensing architectures and flight scenarios that are hardly reproducible during flight tests. Real-time multisensor tracking results based on flight data are reported, which demonstrate reliability of the laboratory simulation while also showing the effectiveness of radar/electro-optical fusion in a non-cooperative collision avoidance architecture

Multisensor based fully autonomous non-cooperative collision avoidance system for UAVs

Unmanned Aerial Vehicles will have a safe access to the Civil Airspace only when they will be able to avoid collisions even with non cooperative flying obstacles. Thus, they need to replace the capability of human eye to detect potential mid-air collisions with other airframes and the pilot experience to find an adequate avoidance trajectory. This thesis deals with development and test of a fully autonomous system devoted to avoidance of non cooperative intruders. In particular, it focuses on sensors, and processing logics and hardware, required on the unmanned system to acquire situational awareness. The study was carried out in collaboration with the Italian Aerospace Research Center within a research project named TECVOL, funded in the frame of National Aerospace Research Program. The performed activities covered all the steps in the development process from the analysis of requirements deriving from the application, to the real time implementation of designed logics. Designed prototype system is based on a multi-sensor architecture with a Ka-band pulsed radar as the main sensor, and four electro-optical cameras as aiding sensors. Proper logics and algorithms for real time sensor fusion have been developed, tested in off-line simulations, and later implemented on embedded systems to enable technology flight demonstration. Numerical results and flight data have shown the potential of the developed system. Also on the basis of the international scenario, this technology demonstration has gained a significant scientific value