4,615 research outputs found
Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles
We present a new method to adapt an RGB-trained water segmentation network to
target-domain aerial thermal imagery using online self-supervision by
leveraging texture and motion cues as supervisory signals. This new thermal
capability enables current autonomous aerial robots operating in near-shore
environments to perform tasks such as visual navigation, bathymetry, and flow
tracking at night. Our method overcomes the problem of scarce and
difficult-to-obtain near-shore thermal data that prevents the application of
conventional supervised and unsupervised methods. In this work, we curate the
first aerial thermal near-shore dataset, show that our approach outperforms
fully-supervised segmentation models trained on limited target-domain thermal
data, and demonstrate real-time capabilities onboard an Nvidia Jetson embedded
computing platform. Code and datasets used in this work will be available at:
https://github.com/connorlee77/uav-thermal-water-segmentation.Comment: 8 pages, 4 figures, 3 table
Interpreting Deep Visual Representations via Network Dissection
The success of recent deep convolutional neural networks (CNNs) depends on
learning hidden representations that can summarize the important factors of
variation behind the data. However, CNNs often criticized as being black boxes
that lack interpretability, since they have millions of unexplained model
parameters. In this work, we describe Network Dissection, a method that
interprets networks by providing labels for the units of their deep visual
representations. The proposed method quantifies the interpretability of CNN
representations by evaluating the alignment between individual hidden units and
a set of visual semantic concepts. By identifying the best alignments, units
are given human interpretable labels across a range of objects, parts, scenes,
textures, materials, and colors. The method reveals that deep representations
are more transparent and interpretable than expected: we find that
representations are significantly more interpretable than they would be under a
random equivalently powerful basis. We apply the method to interpret and
compare the latent representations of various network architectures trained to
solve different supervised and self-supervised training tasks. We then examine
factors affecting the network interpretability such as the number of the
training iterations, regularizations, different initializations, and the
network depth and width. Finally we show that the interpreted units can be used
to provide explicit explanations of a prediction given by a CNN for an image.
Our results highlight that interpretability is an important property of deep
neural networks that provides new insights into their hierarchical structure.Comment: *B. Zhou and D. Bau contributed equally to this work. 15 pages, 27
figure
Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Network
This work presents a new method for unsupervised thermal image classification
and semantic segmentation by transferring knowledge from the RGB domain using a
multi-domain attention network. Our method does not require any thermal
annotations or co-registered RGB-thermal pairs, enabling robots to perform
visual tasks at night and in adverse weather conditions without incurring
additional costs of data labeling and registration. Current unsupervised domain
adaptation methods look to align global images or features across domains.
However, when the domain shift is significantly larger for cross-modal data,
not all features can be transferred. We solve this problem by using a shared
backbone network that promotes generalization, and domain-specific attention
that reduces negative transfer by attending to domain-invariant and
easily-transferable features. Our approach outperforms the state-of-the-art
RGB-to-thermal adaptation method in classification benchmarks, and is
successfully applied to thermal river scene segmentation using only synthetic
RGB images. Our code is made publicly available at
https://github.com/ganlumomo/thermal-uda-attention
Sea Ice Extraction via Remote Sensed Imagery: Algorithms, Datasets, Applications and Challenges
The deep learning, which is a dominating technique in artificial
intelligence, has completely changed the image understanding over the past
decade. As a consequence, the sea ice extraction (SIE) problem has reached a
new era. We present a comprehensive review of four important aspects of SIE,
including algorithms, datasets, applications, and the future trends. Our review
focuses on researches published from 2016 to the present, with a specific focus
on deep learning-based approaches in the last five years. We divided all
relegated algorithms into 3 categories, including classical image segmentation
approach, machine learning-based approach and deep learning-based methods. We
reviewed the accessible ice datasets including SAR-based datasets, the
optical-based datasets and others. The applications are presented in 4 aspects
including climate research, navigation, geographic information systems (GIS)
production and others. It also provides insightful observations and inspiring
future research directions.Comment: 24 pages, 6 figure
- …