A Comprehensive Review on Computer Vision Analysis of Aerial Data

With the emergence of new technologies in the field of airborne platforms and imaging sensors, aerial data analysis is becoming very popular, capitalizing on its advantages over land data. This paper presents a comprehensive review of the computer vision tasks within the domain of aerial data analysis. While addressing fundamental aspects such as object detection and tracking, the primary focus is on pivotal tasks like change detection, object segmentation, and scene-level analysis. The paper provides the comparison of various hyper parameters employed across diverse architectures and tasks. A substantial section is dedicated to an in-depth discussion on libraries, their categorization, and their relevance to different domain expertise. The paper encompasses aerial datasets, the architectural nuances adopted, and the evaluation metrics associated with all the tasks in aerial data analysis. Applications of computer vision tasks in aerial data across different domains are explored, with case studies providing further insights. The paper thoroughly examines the challenges inherent in aerial data analysis, offering practical solutions. Additionally, unresolved issues of significance are identified, paving the way for future research directions in the field of aerial data analysis.Comment: 112 page

Utilization of Deep Learning for Mapping Land Use Change Base on Geographic Information System: A Case Study of Liquefaction

This study aims to extract buildings and roads and determine the extent of changes before and after the liquefaction disaster. The research method used is automatic extraction. The data used are Google Earth images for 2017 and 2018. The data analysis technique uses the Deep Learning Geography Information System. The results showed that the extraction results of the built-up area were 23.61 ha and the undeveloped area was 147.53 ha. The total length of the road before the liquefaction disaster occurred was 35.50 km. The extraction result after the liquefaction disaster was that the area built up was 1.20 ha, while the buildings lost due to the disaster were 22.41 ha. The total road length prior to the liquefaction disaster was 35.50 km, only 11.20 km of roads were lost, 24.30 km. Deep Learning in Geographic Information Systems (GIS) is proliferating and has many advantages in all aspects of life, including technology, geography, health, education, social life, and disasters

SpaceNet MVOI: a Multi-View Overhead Imagery Dataset

Detection and segmentation of objects in overheard imagery is a challenging task. The variable density, random orientation, small size, and instance-to-instance heterogeneity of objects in overhead imagery calls for approaches distinct from existing models designed for natural scene datasets. Though new overhead imagery datasets are being developed, they almost universally comprise a single view taken from directly overhead ("at nadir"), failing to address a critical variable: look angle. By contrast, views vary in real-world overhead imagery, particularly in dynamic scenarios such as natural disasters where first looks are often over 40 degrees off-nadir. This represents an important challenge to computer vision methods, as changing view angle adds distortions, alters resolution, and changes lighting. At present, the impact of these perturbations for algorithmic detection and segmentation of objects is untested. To address this problem, we present an open source Multi-View Overhead Imagery dataset, termed SpaceNet MVOI, with 27 unique looks from a broad range of viewing angles (-32.5 degrees to 54.0 degrees). Each of these images cover the same 665 square km geographic extent and are annotated with 126,747 building footprint labels, enabling direct assessment of the impact of viewpoint perturbation on model performance. We benchmark multiple leading segmentation and object detection models on: (1) building detection, (2) generalization to unseen viewing angles and resolutions, and (3) sensitivity of building footprint extraction to changes in resolution. We find that state of the art segmentation and object detection models struggle to identify buildings in off-nadir imagery and generalize poorly to unseen views, presenting an important benchmark to explore the broadly relevant challenge of detecting small, heterogeneous target objects in visually dynamic contexts.Comment: Accepted into IEEE International Conference on Computer Vision (ICCV) 201

Disaster Site Structure Analysis: Examining Effective Remote Sensing Techniques in Blue Tarpaulin Inspection

This thesis aimed to evaluate three methods of analyzing blue roofing tarpaulin (tarp) placed on homes in post natural disaster zones with remote sensing techniques by assessing the different methods- image segmentation, machine learning (ML), and supervised classification. One can determine which is the most efficient and accurate way of detecting blue tarps. The concept here was that using the most efficient and accurate way to locate blue tarps can aid federal, state, and local emergency management (EM) operations and homeowners. In the wake of a natural disaster such as a tornado, hurricane, thunderstorm, or similar weather events, roofs are the most likely to be damaged (Esri Events., 2019). Severe roof damage needs to be mitigated as fast as possible: which in the United States is often done at no cost by the Federal Emergency Management Agency (FEMA). This research aimed to find the most efficient and accurate way of detecting blue tarps with three different remote sensing practices. The first method, image segmentation, separates parts of a whole image into smaller areas or categories that correspond to distinct items or parts of objects. Each pixel in a remotely sensed image is then classified into categories set by the user. A successful segmentation will result when pixels in the same category have comparable multivariate, grayscale values and form a linked area, whereas nearby pixels in other categories have distinct values. Machine Learning, ML, a second method, is a technique that processes data depending on many layers for feature v identification and pattern recognition. ArcGIS Pro mapping software processes data with ML classification methods to classify remote sensing imagery. Deep learning models may be used to recognize objects, classify images, and in this example, classify pixels. The resultant model definition file or deep learning software package is used to run the inference geoprocessing tools to extract particular item positions, categorize or label the objects, or classify the pixels in the picture. Finally, supervised classification is based on a system in which a user picks sample pixel in an image that are indicative of certain classes and then tells image-processing software to categorize the other pixels in the picture using these training sites as references. To group pixels together, the user also specifies the limits for how similar they must be. The number of classifications into which the image is categorized is likewise determined by the user. The importance of tracking blue roofs is multifaceted. Structures with roof damage from natural disasters face many immediate dangers, such as further water and wind damage. These communities are at a critical moment as responding to the damage efficiently and effectively should occur in the immediate aftermath of a disaster. In part due to strategies such as FEMA and the United States Army Corps of Engineers’ (USACE) Operation Blue Roof, most often blue tarpaulins are installed on structures to prevent further damage caused by wind and rain. From a Unmanned Arial Vehicles (UAV) perspective, these blue tarps stand out amid the downed trees, devastated infrastructure, and other debris that will populate the area. Understanding that recovery can be one of the most important stages of Emergency Management, testing techniques vi for speed, accuracy, and effectiveness will assist in creating more effective Emergency Management (EM) specialists

Understanding High Resolution Aerial Imagery Using Computer Vision Techniques

Computer vision can make important contributions to the analysis of remote sensing satellite or aerial imagery. However, the resolution of early satellite imagery was not sufficient to provide useful spatial features. The situation is changing with the advent of very-high-spatial-resolution (VHR) imaging sensors. This change makes it possible to use computer vision techniques to perform analysis of man-made structures. Meanwhile, the development of multi-view imaging techniques allows the generation of accurate point clouds as ancillary knowledge. This dissertation aims at developing computer vision and machine learning algorithms for high resolution aerial imagery analysis in the context of application problems including debris detection, building detection and roof condition assessment. High resolution aerial imagery and point clouds were provided by Pictometry International for this study. Debris detection after natural disasters such as tornadoes, hurricanes or tsunamis, is needed for effective debris removal and allocation of limited resources. Significant advances in aerial image acquisition have greatly enabled the possibilities for rapid and automated detection of debris. In this dissertation, a robust debris detection algorithm is proposed. Large scale aerial images are partitioned into homogeneous regions by interactive segmentation. Debris areas are identified based on extracted texture features. Robust building detection is another important part of high resolution aerial imagery understanding. This dissertation develops a 3D scene classification algorithm for building detection using point clouds derived from multi-view imagery. Point clouds are divided into point clusters using Euclidean clustering. Individual point clusters are identified based on extracted spectral and 3D structural features. The inspection of roof condition is an important step in damage claim processing in the insurance industry. Automated roof condition assessment from remotely sensed images is proposed in this dissertation. Initially, texture classification and a bag-of-words model were applied to assess the roof condition using features derived from the whole rooftop. However, considering the complexity of residential rooftop, a more sophisticated method is proposed to divide the task into two stages: 1) roof segmentation, followed by 2) classification of segmented roof regions. Deep learning techniques are investigated for both segmentation and classification. A deep learned feature is proposed and applied in a region merging segmentation algorithm. A fine-tuned deep network is adopted for roof segment classification and found to achieve higher accuracy than traditional methods using hand-crafted features. Contributions of this study include the development of algorithms for debris detection using 2D images and building detection using 3D point clouds. For roof condition assessment, the solutions to this problem are explored in two directions: features derived from the whole rooftop and features extracted from each roof segments. Through our research, roof segmentation followed by segments classification was found to be a more promising method and the workflow processing developed and tested. Deep learning techniques are also investigated for both roof segmentation and segments classification. More unsupervised feature extraction techniques using deep learning can be explored in future work

Transformer-based Flood Scene Segmentation for Developing Countries

Floods are large-scale natural disasters that often induce a massive number of deaths, extensive material damage, and economic turmoil. The effects are more extensive and longer-lasting in high-population and low-resource developing countries. Early Warning Systems (EWS) constantly assess water levels and other factors to forecast floods, to help minimize damage. Post-disaster, disaster response teams undertake a Post Disaster Needs Assessment (PDSA) to assess structural damage and determine optimal strategies to respond to highly affected neighbourhoods. However, even today in developing countries, EWS and PDSA analysis of large volumes of image and video data is largely a manual process undertaken by first responders and volunteers. We propose FloodTransformer, which to the best of our knowledge, is the first visual transformer-based model to detect and segment flooded areas from aerial images at disaster sites. We also propose a custom metric, Flood Capacity (FC) to measure the spatial extent of water coverage and quantify the segmented flooded area for EWS and PDSA analyses. We use the SWOC Flood segmentation dataset and achieve 0.93 mIoU, outperforming all other methods. We further show the robustness of this approach by validating across unseen flood images from other flood data sources.Comment: Presented at NeurIPS 2021 Workshop on Machine Learning for the Developing Worl