Fast automatic airport detection in remote sensing images using convolutional neural networks

Fast and automatic detection of airports from remote sensing images is useful for many military and civilian applications. In this paper, a fast automatic detection method is proposed to detect airports from remote sensing images based on convolutional neural networks using the Faster R-CNN algorithm. This method first applies a convolutional neural network to generate candidate airport regions. Based on the features extracted from these proposals, it then uses another convolutional neural network to perform airport detection. By taking the typical elongated linear geometric shape of airports into consideration, some specific improvements to the method are proposed. These approaches successfully improve the quality of positive samples and achieve a better accuracy in the final detection results. Experimental results on an airport dataset, Landsat 8 images, and a Gaofen-1 satellite scene demonstrate the effectiveness and efficiency of the proposed method

Satellite Imagery Multiscale Rapid Detection with Windowed Networks

Detecting small objects over large areas remains a significant challenge in satellite imagery analytics. Among the challenges is the sheer number of pixels and geographical extent per image: a single DigitalGlobe satellite image encompasses over 64 km2 and over 250 million pixels. Another challenge is that objects of interest are often minuscule (~pixels in extent even for the highest resolution imagery), which complicates traditional computer vision techniques. To address these issues, we propose a pipeline (SIMRDWN) that evaluates satellite images of arbitrarily large size at native resolution at a rate of > 0.2 km2/s. Building upon the tensorflow object detection API paper, this pipeline offers a unified approach to multiple object detection frameworks that can run inference on images of arbitrary size. The SIMRDWN pipeline includes a modified version of YOLO (known as YOLT), along with the models of the tensorflow object detection API: SSD, Faster R-CNN, and R-FCN. The proposed approach allows comparison of the performance of these four frameworks, and can rapidly detect objects of vastly different scales with relatively little training data over multiple sensors. For objects of very different scales (e.g. airplanes versus airports) we find that using two different detectors at different scales is very effective with negligible runtime cost.We evaluate large test images at native resolution and find mAP scores of 0.2 to 0.8 for vehicle localization, with the YOLT architecture achieving both the highest mAP and fastest inference speed.Comment: 8 pages, 7 figures, 2 tables, 1 appendix. arXiv admin note: substantial text overlap with arXiv:1805.0951

Randomize to Generalize: Domain Randomization for Runway FOD Detection

Tiny Object Detection is challenging due to small size, low resolution, occlusion, background clutter, lighting conditions and small object-to-image ratio. Further, object detection methodologies often make underlying assumption that both training and testing data remain congruent. However, this presumption often leads to decline in performance when model is applied to out-of-domain(unseen) data. Techniques like synthetic image generation are employed to improve model performance by leveraging variations in input data. Such an approach typically presumes access to 3D-rendered datasets. In contrast, we propose a novel two-stage methodology Synthetic Randomized Image Augmentation (SRIA), carefully devised to enhance generalization capabilities of models encountering 2D datasets, particularly with lower resolution which is more practical in real-world scenarios. The first stage employs a weakly supervised technique to generate pixel-level segmentation masks. Subsequently, the second stage generates a batch-wise synthesis of artificial images, carefully designed with an array of diverse augmentations. The efficacy of proposed technique is illustrated on challenging foreign object debris (FOD) detection. We compare our results with several SOTA models including CenterNet, SSD, YOLOv3, YOLOv4, YOLOv5, and Outer Vit on a publicly available FOD-A dataset. We also construct an out-of-distribution test set encompassing 800 annotated images featuring a corpus of ten common categories. Notably, by harnessing merely 1.81% of objects from source training data and amalgamating with 29 runway background images, we generate 2227 synthetic images. Subsequent model retraining via transfer learning, utilizing enriched dataset generated by domain randomization, demonstrates significant improvement in detection accuracy. We report that detection accuracy improved from an initial 41% to 92% for OOD test set.Comment: 29 pages, 9 figure

Automatic Update of Airport GIS by Remote Sensing Image Analysis

This project investigates ways to automatically update Geographic Information Systems (GIS) for airports by analysis of Very High Resolution (VHR) remote sensing images. These GIS databases map the physical layout of an airport by representing a broad range of features (such as runways, taxiways and roads) as georeferenced vector objects. Updating such systems therefore involves both automatic detection of relevant objects from remotely sensed images, and comparison of these objects between bi-temporal images. The size of the VHR images and the diversity of the object types to be captured in the GIS databases makes this a very large and complex problem. Therefore we must split it into smaller parts which can be framed as instances of image processing problems. The aim of this project is to apply a range of methodologies to these problems and compare their results, providing quantitative data where possible. In this report, we devote a chapter to each sub-problem that was focussed on. Chapter 1 begins by introducing the background and motivation of the project, and describes the problem in more detail. Chapter 2 presents a method for detecting and segmenting runways, by detecting their distinctive markings and feeding them into a modified Hough transform. The algorithm was tested on a dataset of six bi-temporal remote sensing image pairs and validated against manually generated ground-truth GIS data, provided by Jeppesen. Chapter 3 investigates co-registration of bi-temporal images, as a necessary precursor to most direct change detection algorithms. Chapter 4 then tests a range of bi-temporal change detection algorithms (some standard, some novel) on co-registered images of airports, with the aim of producing a change heat-map which may assist a human operator in rapidly focussing attention on areas that have changed significantly. Chapter 5 explores a number of approaches to detecting curvilinear AMDB features such as taxilines and stopbars, by means of enhancing such features and suppressing others, prior to thresholding. Finally in Chapter 6 we develop a method for distinguishing between AMDB lines and other curvilinear structures that may occur in an image, by analysing the connectivity between such features and the runways

Machine Vision Identification of Airport Runways With Visible and Infrared Videos

A widely used machine vision pipeline based on the Speeded-Up Robust Features feature detector was applied to the problem of identifying a runway from a universe of known runways, which was constructed using video records of 19 straight-in glidepath approaches to nine runways. The recordings studied included visible, short-wave infrared, and long-wave infrared videos in clear conditions, rain, and fog. Both daytime and nighttime runway approaches were used. High detection specificity (identification of the runway approached and rejection of the other runways in the universe) was observed in all conditions (greater than 90% Bayesian posterior probability). In the visible band, repeatability (identification of a given runway across multiple videos of it) was observed only if illumination (day versus night) was the same and approach visibility was good. Some repeatability was found across visible and shortwave sensor bands. Camera-based geolocation during aircraft landing was compared to the standard Charted Visual Approach Procedure

Fusion of enhanced and synthetic vision system images for runway and horizon detection

Networked operation of unmanned air vehicles (UAVs) demands fusion of information from disparate sources for accurate flight control. In this investigation, a novel sensor fusion architecture for detecting aircraft runway and horizons as well as enhancing the awareness of surrounding terrain is introduced based on fusion of enhanced vision system (EVS) and synthetic vision system (SVS) images. EVS and SVS image fusion has yet to be implemented in real-world situations due to signal misalignment. We address this through a registration step to align EVS and SVS images. Four fusion rules combining discrete wavelet transform (DWT) sub-bands are formulated, implemented, and evaluated. The resulting procedure is tested on real EVS-SVS image pairs and pairs containing simulated turbulence. Evaluations reveal that runways and horizons can be detected accurately even in poor visibility. Furthermore, it is demonstrated that different aspects of EVS and SVS images can be emphasized by using different DWT fusion rules. The procedure is autonomous throughout landing, irrespective of weather. The fusion architecture developed in this study holds promise for incorporation into manned heads-up displays (HUDs) and UAV remote displays to assist pilots landing aircraft in poor lighting and varying weather. The algorithm also provides a basis for rule selection in other signal fusion applications

Testing sentinel-1 SAR interferometry data for airport runway monitoring: a geostatistical analysis

Multi-Temporal Interferometric Synthetic Aperture Radar (MT-InSAR) techniques are gaining momentum in the assessment and health monitoring of infrastructure assets. Amongst others, the Persistent Scatterers Interferometry (PSI) technique has proven to be viable for the long-term evaluation of ground scatterers. However, its effectiveness as a routine tool for certain critical application areas, such as the assessment of millimetre-scale differential displacements in airport runways, is still debated. This research aims to demonstrate the viability of using medium-resolution Copernicus ESA Sentinel-1A (C-Band) SAR products and their contribution to improve current maintenance strategies in case of localised foundation settlements in airport runways. To this purpose, “Runway n.3” of the “Leonardo Da Vinci International Airport” in Fiumicino, Rome, Italy was investigated as an explanatory case study, in view of historical geotechnical settlements affecting the runway area. In this context, a geostatistical study is developed for the exploratory spatial data analysis and the interpolation of the Sentinel-1A SAR data. The geostatistical analysis provided ample information on the spatial continuity of the Sentinel 1 data in comparison with the high-resolution COSMO-SkyMed data and the ground-based topographic levelling data. Furthermore, a comparison between the PSI outcomes from the Sentinel-1A SAR data—interpolated through Ordinary Kriging—and the ground-truth topographic levelling data demonstrated the high accuracy of the Sentinel 1 data. This is proven by the high values of the correlation coefficient (r = 0.94), the multiple R-squared coefficient (R2 = 0.88) and the Slope value (0.96). The results of this study clearly support the effectiveness of using Sentinel-1A SAR data as a continuous and long-term routine monitoring tool for millimetre-scale displacements in airport runways, paving the way for the development of more efficient and sustainable maintenance strategies for inclusion in next generation Airport Pavement Management Systems (APMSs)

Detection of Obstacles in Monocular Image Sequences

The ability to detect and locate runways/taxiways and obstacles in images captured using on-board sensors is an essential first step in the automation of low-altitude flight, landing, takeoff, and taxiing phase of aircraft navigation. Automation of these functions under different weather and lighting situations, can be facilitated by using sensors of different modalities. An aircraft-based Synthetic Vision System (SVS), with sensors of different modalities mounted on-board, complements the current ground-based systems in functions such as detection and prevention of potential runway collisions, airport surface navigation, and landing and takeoff in all weather conditions. In this report, we address the problem of detection of objects in monocular image sequences obtained from two types of sensors, a Passive Millimeter Wave (PMMW) sensor and a video camera mounted on-board a landing aircraft. Since the sensors differ in their spatial resolution, and the quality of the images obtained using these sensors is not the same, different approaches are used for detecting obstacles depending on the sensor type. These approaches are described separately in two parts of this report. The goal of the first part of the report is to develop a method for detecting runways/taxiways and objects on the runway in a sequence of images obtained from a moving PMMW sensor. Since the sensor resolution is low and the image quality is very poor, we propose a model-based approach for detecting runways/taxiways. We use the approximate runway model and the position information of the camera provided by the Global Positioning System (GPS) to define regions of interest in the image plane to search for the image features corresponding to the runway markers. Once the runway region is identified, we use histogram-based thresholding to detect obstacles on the runway and regions outside the runway. This algorithm is tested using image sequences simulated from a single real PMMW image

Application of Wavelet Analysis in Detecting Runway Foreign Object Debris

Foreign Object Debris (FOD) is dangerous for aircraft safety. And it can be suggested to use image processing technology on the FOD’s detection. Depending on image processing system, a major sub-system in FOD detecting system on the runway, FOD image will be observed efficiently and rapidly with few economy costs and highly accuracy and reliability. The paper analyses the characteristics and principles of wavelet transformation algorithm and applies wavelet theory on FOD’s identification and detection. Identifying the FOD’s shape and marking characteristic point on the runway under poor visual background would be accomplished by programming in MATLAB using wavelet algorithm. The results show that the plan is applicable. Besides that, it brings about profound significance for realizing the real-time detecting on the FOD and testing with more feasibility and efficiency.