6,210 research outputs found
Dense semantic labeling of sub-decimeter resolution images with convolutional neural networks
Semantic labeling (or pixel-level land-cover classification) in ultra-high
resolution imagery (< 10cm) requires statistical models able to learn high
level concepts from spatial data, with large appearance variations.
Convolutional Neural Networks (CNNs) achieve this goal by learning
discriminatively a hierarchy of representations of increasing abstraction.
In this paper we present a CNN-based system relying on an
downsample-then-upsample architecture. Specifically, it first learns a rough
spatial map of high-level representations by means of convolutions and then
learns to upsample them back to the original resolution by deconvolutions. By
doing so, the CNN learns to densely label every pixel at the original
resolution of the image. This results in many advantages, including i)
state-of-the-art numerical accuracy, ii) improved geometric accuracy of
predictions and iii) high efficiency at inference time.
We test the proposed system on the Vaihingen and Potsdam sub-decimeter
resolution datasets, involving semantic labeling of aerial images of 9cm and
5cm resolution, respectively. These datasets are composed by many large and
fully annotated tiles allowing an unbiased evaluation of models making use of
spatial information. We do so by comparing two standard CNN architectures to
the proposed one: standard patch classification, prediction of local label
patches by employing only convolutions and full patch labeling by employing
deconvolutions. All the systems compare favorably or outperform a
state-of-the-art baseline relying on superpixels and powerful appearance
descriptors. The proposed full patch labeling CNN outperforms these models by a
large margin, also showing a very appealing inference time.Comment: Accepted in IEEE Transactions on Geoscience and Remote Sensing, 201
Semantically Informed Multiview Surface Refinement
We present a method to jointly refine the geometry and semantic segmentation
of 3D surface meshes. Our method alternates between updating the shape and the
semantic labels. In the geometry refinement step, the mesh is deformed with
variational energy minimization, such that it simultaneously maximizes
photo-consistency and the compatibility of the semantic segmentations across a
set of calibrated images. Label-specific shape priors account for interactions
between the geometry and the semantic labels in 3D. In the semantic
segmentation step, the labels on the mesh are updated with MRF inference, such
that they are compatible with the semantic segmentations in the input images.
Also, this step includes prior assumptions about the surface shape of different
semantic classes. The priors induce a tight coupling, where semantic
information influences the shape update and vice versa. Specifically, we
introduce priors that favor (i) adaptive smoothing, depending on the class
label; (ii) straightness of class boundaries; and (iii) semantic labels that
are consistent with the surface orientation. The novel mesh-based
reconstruction is evaluated in a series of experiments with real and synthetic
data. We compare both to state-of-the-art, voxel-based semantic 3D
reconstruction, and to purely geometric mesh refinement, and demonstrate that
the proposed scheme yields improved 3D geometry as well as an improved semantic
segmentation
Relation Network for Multi-label Aerial Image Classification
Multi-label classification plays a momentous role in perceiving intricate
contents of an aerial image and triggers several related studies over the last
years. However, most of them deploy few efforts in exploiting label relations,
while such dependencies are crucial for making accurate predictions. Although
an LSTM layer can be introduced to modeling such label dependencies in a chain
propagation manner, the efficiency might be questioned when certain labels are
improperly inferred. To address this, we propose a novel aerial image
multi-label classification network, attention-aware label relational reasoning
network. Particularly, our network consists of three elemental modules: 1) a
label-wise feature parcel learning module, 2) an attentional region extraction
module, and 3) a label relational inference module. To be more specific, the
label-wise feature parcel learning module is designed for extracting high-level
label-specific features. The attentional region extraction module aims at
localizing discriminative regions in these features and yielding attentional
label-specific features. The label relational inference module finally predicts
label existences using label relations reasoned from outputs of the previous
module. The proposed network is characterized by its capacities of extracting
discriminative label-wise features in a proposal-free way and reasoning about
label relations naturally and interpretably. In our experiments, we evaluate
the proposed model on the UCM multi-label dataset and a newly produced dataset,
AID multi-label dataset. Quantitative and qualitative results on these two
datasets demonstrate the effectiveness of our model. To facilitate progress in
the multi-label aerial image classification, the AID multi-label dataset will
be made publicly available
An Adaptive Sampling Scheme to Efficiently Train Fully Convolutional Networks for Semantic Segmentation
Deep convolutional neural networks (CNNs) have shown excellent performance in
object recognition tasks and dense classification problems such as semantic
segmentation. However, training deep neural networks on large and sparse
datasets is still challenging and can require large amounts of computation and
memory. In this work, we address the task of performing semantic segmentation
on large data sets, such as three-dimensional medical images. We propose an
adaptive sampling scheme that uses a-posterior error maps, generated throughout
training, to focus sampling on difficult regions, resulting in improved
learning. Our contribution is threefold: 1) We give a detailed description of
the proposed sampling algorithm to speed up and improve learning performance on
large images. We propose a deep dual path CNN that captures information at fine
and coarse scales, resulting in a network with a large field of view and high
resolution outputs. We show that our method is able to attain new
state-of-the-art results on the VISCERAL Anatomy benchmark
Deep learning in remote sensing: a review
Standing at the paradigm shift towards data-intensive science, machine
learning techniques are becoming increasingly important. In particular, as a
major breakthrough in the field, deep learning has proven as an extremely
powerful tool in many fields. Shall we embrace deep learning as the key to all?
Or, should we resist a 'black-box' solution? There are controversial opinions
in the remote sensing community. In this article, we analyze the challenges of
using deep learning for remote sensing data analysis, review the recent
advances, and provide resources to make deep learning in remote sensing
ridiculously simple to start with. More importantly, we advocate remote sensing
scientists to bring their expertise into deep learning, and use it as an
implicit general model to tackle unprecedented large-scale influential
challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin
Exploring semantic relationships for hierarchical land use classification based on convolutional neural networks
Land use (LU) is an important information source commonly stored in geospatial databases. Most current work on automatic LU classification for updating topographic databases considers only one category level (e.g. residential or agricultural) consisting of a small number of classes. However, LU databases frequently contain very detailed information, using a hierarchical object catalogue where the number of categories differs depending on the hierarchy level. This paper presents a method for the classification of LU on the basis of aerial images that differentiates a fine-grained class structure, exploiting the hierarchical relationship between categories at different levels of the class catalogue. Starting from a convolutional neural network (CNN) for classifying the categories of all levels, we propose a strategy to simultaneously learn the semantic dependencies between different category levels explicitly. The input to the CNN consists of aerial images and derived data as well as land cover information derived from semantic segmentation. Its output is the class scores at three different semantic levels, based on which predictions that are consistent with the class hierarchy are made. We evaluate our method using two test sites and show how the classification accuracy depends on the semantic category level. While at the coarsest level, an overall accuracy in the order of 90% can be achieved, at the finest level, this accuracy is reduced to around 65%. Our experiments also show which classes are particularly hard to differentiate. © 2020 Copernicus GmbH. All rights reserved
- …