On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances and Million-AID

The past years have witnessed great progress on remote sensing (RS) image interpretation and its wide applications. With RS images becoming more accessible than ever before, there is an increasing demand for the automatic interpretation of these images. In this context, the benchmark datasets serve as essential prerequisites for developing and testing intelligent interpretation algorithms. After reviewing existing benchmark datasets in the research community of RS image interpretation, this article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. Specifically, we first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. We then present the general guidances on creating benchmark datasets in efficient manners. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million-AID, a new large-scale benchmark dataset containing a million instances for RS image scene classification. Several challenges and perspectives in RS image annotation are finally discussed to facilitate the research in benchmark dataset construction. We do hope this paper will provide the RS community an overall perspective on constructing large-scale and practical image datasets for further research, especially data-driven ones

Challenges and Opportunities of Multimodality and Data Fusion in Remote Sensing

International audience—Remote sensing is one of the most common ways to extract relevant information about the Earth and our environment. Remote sensing acquisitions can be done by both active (synthetic aperture radar, LiDAR) and passive (optical and thermal range, multispectral and hyperspectral) devices. According to the sensor, a variety of information about the Earth's surface can be obtained. The data acquired by these sensors can provide information about the structure (optical, synthetic aperture radar), elevation (LiDAR) and material content (multi and hyperspectral) of the objects in the image. Once considered together their comple-mentarity can be helpful for characterizing land use (urban analysis, precision agriculture), damage detection (e.g., in natural disasters such as floods, hurricanes, earthquakes, oil-spills in seas), and give insights to potential exploitation of resources (oil fields, minerals). In addition, repeated acquisitions of a scene at different times allows one to monitor natural resources and environmental variables (vegetation phenology, snow cover), anthropological effects (urban sprawl, deforestation), climate changes (desertification, coastal erosion) among others. In this paper, we sketch the current opportunities and challenges related to the exploitation of multimodal data for Earth observation. This is done by leveraging the outcomes of the Data Fusion contests, organized by the IEEE Geoscience and Remote Sensing Society since 2006. We will report on the outcomes of these contests, presenting the multimodal sets of data made available to the community each year, the targeted applications and an analysis of the submitted methods and results: How was multimodality considered and integrated in the processing chain? What were the improvements/new opportunities offered by the fusion? What were the objectives to be addressed and the reported solutions? And from this, what will be the next challenges

Spectral Optimization of Airborne Multispectral Camera for Land Cover Classification: Automatic Feature Selection and Spectral Band Clustering

Hyperspectral imagery consists of hundreds of contiguous spectral bands. However, most of them are redundant. Thus a subset of well-chosen bands is generally sufficient for a specific problem, enabling to design adapted superspectral sensors dedicated to specific land cover classification. Related both to feature selection and extraction, spectral optimization identifies the most relevant band subset for specific applications, involving a band subset relevance score as well as a method to optimize it. This study first focuses on the choice of such relevance score. Several criteria are compared through both quantitative and qualitative analyses. To have a fair comparison, all tested criteria are compared to classic hyperspectral data sets using the same optimization heuristics: an incremental one to assess the impact of the number of selected bands and a stochastic one to obtain several possible good band subsets and to derive band importance measures out of intermediate good band subsets. Last, a specific approach is proposed to cope with the optimization of bandwidth. It consists in building a hierarchy of groups of adjacent bands, according to a score to decide which adjacent bands must be merged, before band selection is performed at the different levels of this hierarchy

Regional Scale Dryland Vegetation Classification with an Integrated Lidar-Hyperspectral Approach

The sparse canopy cover and large contribution of bright background soil, along with the heterogeneous vegetation types in close proximity, are common challenges for mapping dryland vegetation with remote sensing. Consequently, the results of a single classification algorithm or one type of sensor to characterize dryland vegetation typically show low accuracy and lack robustness. In our study, we improved classification accuracy in a semi-arid ecosystem based on the use of vegetation optical (hyperspectral) and structural (lidar) information combined with the environmental characteristics of the landscape. To accomplish this goal, we used both spectral angle mapper (SAM) and multiple endmember spectral mixture analysis (MESMA) for optical vegetation classification. Lidar-derived maximum vegetation height and delineated riparian zones were then used to modify the optical classification. Incorporating the lidar information into the classification scheme increased the overall accuracy from 60% to 89%. Canopy structure can have a strong influence on spectral variability and the lidar provided complementary information for SAM’s sensitivity to shape but not magnitude of the spectra. Similar approaches to map large regions of drylands with low uncertainty may be readily implemented with unmixing algorithms applied to upcoming space-based imaging spectroscopy and lidar. This study advances our understanding of the nuances associated with mapping xeric and mesic regions, and highlights the importance of incorporating complementary algorithms and sensors to accurately characterize the heterogeneity of dryland ecosystems

Regional scale dryland vegetation classification with an integrated lidar-hyperspectral approach

The sparse canopy cover and large contribution of bright background soil, along with the heterogeneous vegetation types in close proximity, are common challenges for mapping dryland vegetation with remote sensing. Consequently, the results of a single classification algorithm or one type of sensor to characterize dryland vegetation typically show low accuracy and lack robustness. In our study, we improved classification accuracy in a semi-arid ecosystem based on the use of vegetation optical (hyperspectral) and structural (lidar) information combined with the environmental characteristics of the landscape. To accomplish this goal, we used both spectral angle mapper (SAM) and multiple endmember spectral mixture analysis (MESMA) for optical vegetation classification. Lidar-derived maximum vegetation height and delineated riparian zones were then used to modify the optical classification. Incorporating the lidar information into the classification scheme increased the overall accuracy from 60% to 89%. Canopy structure can have a strong influence on spectral variability and the lidar provided complementary information for SAM’s sensitivity to shape but not magnitude of the spectra. Similar approaches to map large regions of drylands with low uncertainty may be readily implemented with unmixing algorithms applied to upcoming space-based imaging spectroscopy and lidar. This study advances our understanding of the nuances associated with mapping xeric and mesic regions, and highlights the importance of incorporating complementary algorithms and sensors to accurately characterize the heterogeneity of dryland ecosystems

CBANet: an end-to-end cross band 2-D attention network for hyperspectral change detection in remote sensing.

As a fundamental task in remote sensing observation of the earth, change detection using hyperspectral images (HSI) features high accuracy due to the combination of the rich spectral and spatial information, especially for identifying land-cover variations in bi-temporal HSIs. Relying on the image difference, existing HSI change detection methods fail to preserve the spectral characteristics and suffer from high data dimensionality, making them extremely challenging to deal with changing areas of various sizes. To tackle these challenges, we propose a cross-band 2-D self-attention Network (CBANet) for end-to-end HSI change detection. By embedding a cross-band feature extraction module into a 2-D spatial-spectral self-attention module, CBANet is highly capable of extracting the spectral difference of matching pixels by considering the correlation between adjacent pixels. The CBANet has shown three key advantages: 1) less parameters and high efficiency; 2) high efficacy of extracting representative spectral information from bi-temporal images; and 3) high stability and accuracy for identifying both sparse sporadic changing pixels and large changing areas whilst preserving the edges. Comprehensive experiments on three publicly available datasets have fully validated the efficacy and efficiency of the proposed methodology

Region Based Data Mining on Agriculture Data

Spatial Data Mining is the process of discovering interesting and previously unknown, but potentially useful patterns from large spatial databases. Most relationships in spatial datasets are regional and there is a great need for regional regression methods that derive regional reflects different spatial characteristics of different regions. A central challenge in spatial data mining is the efficiency of spatial data mining algorithms, due to the often huge amount of spatial data and the complexity of spatial data types and spatial accessing methods. This paper proposes a regional regression technique for regions that are defined by a categorical attribute, in particular soil type. The result is a series of hierarchically grouped regions according to their similarity

Effects of Spatial Pattern of GreenSpace on Land Surface Temperature: A case study on Oasis City Aksu, northwest China

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.The urban heat island (UHI) refers to the phenomenon of higher atmospheric and surface temperatures occurring in urban areas than in the surrounding rural areas. Numerous studies have shown that increased percent cover of greenspace (PLAND) can significantly decrease land surface temperatures (LST). Fewer studies, however, have investigated the effects of configuration of greenspace on LST. This thesis aims to fill this gap using oasis city Aksu, northwest China as a case study. PLAND along with two configuration metrics were used to measure the composition and configuration of greenspace. The metrics were calculated by moving window method based on a greenspace map derived from Landsat Thematic Mapper (TM) imagery, and LST data were retrieved from Landsat TM thermal band. Normalized mutual information measure was employed to investigate the relationship between LST and the spatial pattern of greenspace. The results showed that PLAND was the most important predictor of LST. Configuration of greenspace also significantly affected LST. In addition, the variance of LST was largely explained by both composition and configuration of greenspace. Results from this study can expand our understanding of the relationship between LST and vegetation, and provide insights for improving urban greenspace planning and management

Automated Building Information Extraction and Evaluation from High-resolution Remotely Sensed Data

The two-dimensional (2D) footprints and three-dimensional (3D) structures of buildings are of great importance to city planning, natural disaster management, and virtual environmental simulation. As traditional manual methodologies for collecting 2D and 3D building information are often both time consuming and costly, automated methods are required for efficient large area mapping. It is challenging to extract building information from remotely sensed data, considering the complex nature of urban environments and their associated intricate building structures. Most 2D evaluation methods are focused on classification accuracy, while other dimensions of extraction accuracy are ignored. To assess 2D building extraction methods, a multi-criteria evaluation system has been designed. The proposed system consists of matched rate, shape similarity, and positional accuracy. Experimentation with four methods demonstrates that the proposed multi-criteria system is more comprehensive and effective, in comparison with traditional accuracy assessment metrics. Building height is critical for building 3D structure extraction. As data sources for height estimation, digital surface models (DSMs) that are derived from stereo images using existing software typically provide low accuracy results in terms of rooftop elevations. Therefore, a new image matching method is proposed by adding building footprint maps as constraints. Validation demonstrates that the proposed matching method can estimate building rooftop elevation with one third of the error encountered when using current commercial software. With an ideal input DSM, building height can be estimated by the elevation contrast inside and outside a building footprint. However, occlusions and shadows cause indistinct building edges in the DSMs generated from stereo images. Therefore, a “building-ground elevation difference model” (EDM) has been designed, which describes the trend of the elevation difference between a building and its neighbours, in order to find elevation values at bare ground. Experiments using this novel approach report that estimated building height with 1.5m residual, which out-performs conventional filtering methods. Finally, 3D buildings are digitally reconstructed and evaluated. Current 3D evaluation methods did not present the difference between 2D and 3D evaluation methods well; traditionally, wall accuracy is ignored. To address these problems, this thesis designs an evaluation system with three components: volume, surface, and point. As such, the resultant multi-criteria system provides an improved evaluation method for building reconstruction