BuildMapper: A Fully Learnable Framework for Vectorized Building Contour Extraction

Deep learning based methods have significantly boosted the study of automatic building extraction from remote sensing images. However, delineating vectorized and regular building contours like a human does remains very challenging, due to the difficulty of the methodology, the diversity of building structures, and the imperfect imaging conditions. In this paper, we propose the first end-to-end learnable building contour extraction framework, named BuildMapper, which can directly and efficiently delineate building polygons just as a human does. BuildMapper consists of two main components: 1) a contour initialization module that generates initial building contours; and 2) a contour evolution module that performs both contour vertex deformation and reduction, which removes the need for complex empirical post-processing used in existing methods. In both components, we provide new ideas, including a learnable contour initialization method to replace the empirical methods, dynamic predicted and ground truth vertex pairing for the static vertex correspondence problem, and a lightweight encoder for vertex information extraction and aggregation, which benefit a general contour-based method; and a well-designed vertex classification head for building corner vertices detection, which casts light on direct structured building contour extraction. We also built a suitable large-scale building dataset, the WHU-Mix (vector) building dataset, to benefit the study of contour-based building extraction methods. The extensive experiments conducted on the WHU-Mix (vector) dataset, the WHU dataset, and the CrowdAI dataset verified that BuildMapper can achieve a state-of-the-art performance, with a higher mask average precision (AP) and boundary AP than both segmentation-based and contour-based methods

Generalized differential morphological profiles for remote sensing image classification

Differential morphological profiles (DMPs) are widely used for the spatial/structural feature extraction and classification of remote sensing images. They can be regarded as the shape spectrum, depicting the response of the image structures related to different scales and sizes of the structural elements (SEs). DMPs are defined as the difference of morphological profiles (MPs) between consecutive scales. However, traditional DMPs can ignore discriminative information for features that are across the scales in the profiles. To solve this problem, we propose scale-span differential profiles, i.e., generalized DMPs (GDMPs), to obtain the entire differential profiles. GDMPs can describe the complete shape spectrum and measure the difference between arbitrary scales, which is more appropriate for representing the multiscale characteristics and complex landscapes of remote sensing image scenes. Subsequently, the random forest (RF) classifier is applied to interpret GDMPs considering its robustness for high-dimensional data and ability of evaluating the importance of variables. Meanwhile, the RF "out-of-bag" error can be used to quantify the importance of each channel of GDMPs and select the most discriminative information in the entire profiles. Experiments conducted on three well-known hyperspectral data sets as well as an additional World View-2 data are used to validate the effectiveness of GDMPs compared to the traditional DMPs. The results are promising as GDMPs can significantly outperform the traditional one, as it is capable of adequately exploring the multiscale morphological information

Photogrammetry and Deep Learning

Deep learning has become popular and the mainstream in types of researches related to learning,and has shown its impact on photogrammetry.According to the definition of photogrammetry,a subject that researches shapes,locations,sizes,characteristics and inter-relationships of real objects from optical images,photogrammetry considers two aspects,geometry and semantics.From the two aspects,we review the history of deep learning and discuss its current applications on photogrammetry,and forecast the future development of photogrammetry.In geometry,the deep convolutional neural network (CNN) has been widely applied in stereo matching,SLAM and 3D reconstruction,and has made some effect but needs more improvement.In semantics,conventional empirical and handcrafted methods have failed to extract the semantic information accurately and failed to produce types of “semantic thematic map” as 4D productions (DEM,DOM,DLG,DRG) of photogrammetry,which causes the semantic part of photogrammetry be ignored for a long time.The powerful generalization capacity,ability to fit any functions and stability under types of situations of deep leaning is making the automated production of thematic maps possible.We review the achievements that have been obtained in road network extraction,building detection and crop classification,etc.,and forecast that producing high-accuracy semantic thematic maps directly from optical images will become reality and these maps will become a type of standard products of photogrammetry.At last,we introduce two current researches related to geometry and semantics respectively.One is stereo matching of aerial images based on deep learning and transfer learning; the other is fine crop classification from satellite special-temporal images based on 3D CNN

Forest Type Identification with Random Forest Using Sentinel-1A, Sentinel-2A, Multi-Temporal Landsat-8 and DEM Data

Carbon sink estimation and ecological assessment of forests require accurate forest type mapping. The traditional survey method is time consuming and labor intensive, and the remote sensing method with high-resolution, multi-spectral commercial satellite images has high cost and low availability. In this study, we explore and evaluate the potential of freely-available multi-source imagery to identify forest types with an object-based random forest algorithm. These datasets included Sentinel-2A (S2), Sentinel-1A (S1) in dual polarization, one-arc-second Shuttle Radar Topographic Mission Digital Elevation (DEM) and multi-temporal Landsat-8 images (L8). We tested seven different sets of explanatory variables for classifying eight forest types in Wuhan, China. The results indicate that single-sensor (S2) or single-day data (L8) cannot obtain satisfactory results; the overall accuracy was 54.31% and 50.00%, respectively. Compared with the classification using only Sentinel-2 data, the overall accuracy increased by approximately 15.23% and 22.51%, respectively, by adding DEM and multi-temporal Landsat-8 imagery. The highest accuracy (82.78%) was achieved with fused imagery, the terrain and multi-temporal data contributing the most to forest type identification. These encouraging results demonstrate that freely-accessible multi-source remotely-sensed data have tremendous potential in forest type identification, which can effectively support monitoring and management of forest ecological resources at regional or global scales

Generalized photogrammetry of spaceborne, airborne and terrestrial multi-source remote sensing datasets

Since the 21st century, with the rapid development of cloud computing, big data, internet of things, machine learning and other information technology fields, human beings have entered a new era of artificial intelligence. The subject of photogrammetry has also followed the tide of the new round of scientific and technological revolution and developed rapidly into the brand-new generalized photogrammetry and entered the era of integrated intelligent photogrammetry. Its carrier platform, instruments and data processing theories as well as application fields have also changed significantly. The multi-sensor and multi-level integrated stereo observation technologies from spaceborne, airborne and terrestrial platforms have been greatly developed. In this paper, the novel concept of generalized photogrammetry is first put forward, and its subject connotation, development characteristics and some key technologies and applications are discussed in details. Under the brand-new generalized photogrammetry framework, data acquisition presents the characteristics of multi-angle imaging, multi-modal collaboration, multi-time integration, multi-scale linkage, while data processing presents the trends of multi-feature coupling, multi-control constraints, multi architecture processing, and multi-disciplinary intersection. The all-round development and intelligent service of the general photogrammetry still need to make greater breakthroughs in the aspects of spaceborne, airborne and terrestrial multi perspective or multi-modal image processing, intelligent information extraction and monitoring, combined 3D modeling with point cloud and image, autonomous control of unmanned system, visual inspection of intelligent manufacturing system, etc. Finally, new theories and technologies from real-time or quasi real-time intelligent geometric processing of multi-source remote sensing datasets to information extraction and intelligent service need to be established, which will make a well foundation to meet the new eara of intelligent surveying and mapping