1,113 research outputs found
HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification
Hyperspectral image (HSI) classification is widely used for the analysis of
remotely sensed images. Hyperspectral imagery includes varying bands of images.
Convolutional Neural Network (CNN) is one of the most frequently used deep
learning based methods for visual data processing. The use of CNN for HSI
classification is also visible in recent works. These approaches are mostly
based on 2D CNN. Whereas, the HSI classification performance is highly
dependent on both spatial and spectral information. Very few methods have
utilized the 3D CNN because of increased computational complexity. This letter
proposes a Hybrid Spectral Convolutional Neural Network (HybridSN) for HSI
classification. Basically, the HybridSN is a spectral-spatial 3D-CNN followed
by spatial 2D-CNN. The 3D-CNN facilitates the joint spatial-spectral feature
representation from a stack of spectral bands. The 2D-CNN on top of the 3D-CNN
further learns more abstract level spatial representation. Moreover, the use of
hybrid CNNs reduces the complexity of the model compared to 3D-CNN alone. To
test the performance of this hybrid approach, very rigorous HSI classification
experiments are performed over Indian Pines, Pavia University and Salinas Scene
remote sensing datasets. The results are compared with the state-of-the-art
hand-crafted as well as end-to-end deep learning based methods. A very
satisfactory performance is obtained using the proposed HybridSN for HSI
classification. The source code can be found at
\url{https://github.com/gokriznastic/HybridSN}.Comment: Published in IEEE Geoscience and Remote Sensing Letter
Deep Neural Network Based Hyperspectral Pixel Classification With Factorized Spectral-Spatial Feature Representation
Deep learning has been widely used for hyperspectral pixel classification due
to its ability of generating deep feature representation. However, how to
construct an efficient and powerful network suitable for hyperspectral data is
still under exploration. In this paper, a novel neural network model is
designed for taking full advantage of the spectral-spatial structure of
hyperspectral data. Firstly, we extract pixel-based intrinsic features from
rich yet redundant spectral bands by a subnetwork with supervised pre-training
scheme. Secondly, in order to utilize the local spatial correlation among
pixels, we share the previous subnetwork as a spectral feature extractor for
each pixel in a patch of image, after which the spectral features of all pixels
in a patch are combined and feeded into the subsequent classification
subnetwork. Finally, the whole network is further fine-tuned to improve its
classification performance. Specially, the spectral-spatial factorization
scheme is applied in our model architecture, making the network size and the
number of parameters great less than the existing spectral-spatial deep
networks for hyperspectral image classification. Experiments on the
hyperspectral data sets show that, compared with some state-of-art deep
learning methods, our method achieves better classification results while
having smaller network size and less parameters.Comment: 12 pages, 10 figure
A CNN-based Spatial Feature Fusion Algorithm for Hyperspectral Imagery Classification
The shortage of training samples remains one of the main obstacles in
applying the artificial neural networks (ANN) to the hyperspectral images
classification. To fuse the spatial and spectral information, pixel patches are
often utilized to train a model, which may further aggregate this problem. In
the existing works, an ANN model supervised by center-loss (ANNC) was
introduced. Training merely with spectral information, the ANNC yields
discriminative spectral features suitable for the subsequent classification
tasks. In this paper, a CNN-based spatial feature fusion (CSFF) algorithm is
proposed, which allows a smart fusion of the spatial information to the
spectral features extracted by ANNC. As a critical part of CSFF, a CNN-based
discriminant model is introduced to estimate whether two paring pixels belong
to the same class. At the testing stage, by applying the discriminant model to
the pixel-pairs generated by the test pixel and its neighbors, the local
structure is estimated and represented as a customized convolutional kernel.
The spectral-spatial feature is obtained by a convolutional operation between
the estimated kernel and the corresponding spectral features within a
neighborhood. At last, the label of the test pixel is predicted by classifying
the resulting spectral-spatial feature. Without increasing the number of
training samples or involving pixel patches at the training stage, the CSFF
framework achieves the state-of-the-art by declining classification
failures in experiments on three well-known hyperspectral images
Conditional Random Field and Deep Feature Learning for Hyperspectral Image Segmentation
Image segmentation is considered to be one of the critical tasks in
hyperspectral remote sensing image processing. Recently, convolutional neural
network (CNN) has established itself as a powerful model in segmentation and
classification by demonstrating excellent performances. The use of a graphical
model such as a conditional random field (CRF) contributes further in capturing
contextual information and thus improving the segmentation performance. In this
paper, we propose a method to segment hyperspectral images by considering both
spectral and spatial information via a combined framework consisting of CNN and
CRF. We use multiple spectral cubes to learn deep features using CNN, and then
formulate deep CRF with CNN-based unary and pairwise potential functions to
effectively extract the semantic correlations between patches consisting of
three-dimensional data cubes. Effective piecewise training is applied in order
to avoid the computationally expensive iterative CRF inference. Furthermore, we
introduce a deep deconvolution network that improves the segmentation masks. We
also introduce a new dataset and experimented our proposed method on it along
with several widely adopted benchmark datasets to evaluate the effectiveness of
our method. By comparing our results with those from several state-of-the-art
models, we show the promising potential of our method.Comment: Submitted for Journal (Version 2
Object Tracking in Hyperspectral Videos with Convolutional Features and Kernelized Correlation Filter
Target tracking in hyperspectral videos is a new research topic. In this
paper, a novel method based on convolutional network and Kernelized Correlation
Filter (KCF) framework is presented for tracking objects of interest in
hyperspectral videos. We extract a set of normalized three-dimensional cubes
from the target region as fixed convolution filters which contain spectral
information surrounding a target. The feature maps generated by convolutional
operations are combined to form a three-dimensional representation of an
object, thereby providing effective encoding of local spectral-spatial
information. We show that a simple two-layer convolutional networks is
sufficient to learn robust representations without the need of offline training
with a large dataset. In the tracking step, KCF is adopted to distinguish
targets from neighboring environment. Experimental results demonstrate that the
proposed method performs well on sample hyperspectral videos, and outperforms
several state-of-the-art methods tested on grayscale and color videos in the
same scene.Comment: Accepted by ICSM 201
Machine learning based hyperspectral image analysis: A survey
Hyperspectral sensors enable the study of the chemical properties of scene
materials remotely for the purpose of identification, detection, and chemical
composition analysis of objects in the environment. Hence, hyperspectral images
captured from earth observing satellites and aircraft have been increasingly
important in agriculture, environmental monitoring, urban planning, mining, and
defense. Machine learning algorithms due to their outstanding predictive power
have become a key tool for modern hyperspectral image analysis. Therefore, a
solid understanding of machine learning techniques have become essential for
remote sensing researchers and practitioners. This paper reviews and compares
recent machine learning-based hyperspectral image analysis methods published in
literature. We organize the methods by the image analysis task and by the type
of machine learning algorithm, and present a two-way mapping between the image
analysis tasks and the types of machine learning algorithms that can be applied
to them. The paper is comprehensive in coverage of both hyperspectral image
analysis tasks and machine learning algorithms. The image analysis tasks
considered are land cover classification, target detection, unmixing, and
physical parameter estimation. The machine learning algorithms covered are
Gaussian models, linear regression, logistic regression, support vector
machines, Gaussian mixture model, latent linear models, sparse linear models,
Gaussian mixture models, ensemble learning, directed graphical models,
undirected graphical models, clustering, Gaussian processes, Dirichlet
processes, and deep learning. We also discuss the open challenges in the field
of hyperspectral image analysis and explore possible future directions
Hyperspectral Images Classification Based on Multi-scale Residual Network
Because hyperspectral remote sensing images contain a lot of redundant
information and the data structure is highly non-linear, leading to low
classification accuracy of traditional machine learning methods. The latest
research shows that hyperspectral image classification based on deep
convolutional neural network has high accuracy. However, when a small amount of
data is used for training, the classification accuracy of deep learning methods
is greatly reduced. In order to solve the problem of low classification
accuracy of existing algorithms on small samples of hyperspectral images, a
multi-scale residual network is proposed. The multi-scale extraction and fusion
of spatial and spectral features is realized by adding a branch structure into
the residual block and using convolution kernels of different sizes in the
branch. The spatial and spectral information contained in hyperspectral images
are fully utilized to improve the classification accuracy. In addition, in
order to improve the speed and prevent overfitting, the model uses dynamic
learning rate, BN and Dropout strategies. The experimental results show that
the overall classification accuracy of this method is 99.07% and 99.96%
respectively in the data set of Indian Pines and Pavia University, which is
better than other algorithms
Tracking in Aerial Hyperspectral Videos using Deep Kernelized Correlation Filters
Hyperspectral imaging holds enormous potential to improve the
state-of-the-art in aerial vehicle tracking with low spatial and temporal
resolutions. Recently, adaptive multi-modal hyperspectral sensors have
attracted growing interest due to their ability to record extended data quickly
from aerial platforms. In this study, we apply popular concepts from
traditional object tracking, namely (1) Kernelized Correlation Filters (KCF)
and (2) Deep Convolutional Neural Network (CNN) features to aerial tracking in
hyperspectral domain. We propose the Deep Hyperspectral Kernelized Correlation
Filter based tracker (DeepHKCF) to efficiently track aerial vehicles using an
adaptive multi-modal hyperspectral sensor. We address low temporal resolution
by designing a single KCF-in-multiple Regions-of-Interest (ROIs) approach to
cover a reasonably large area. To increase the speed of deep convolutional
features extraction from multiple ROIs, we design an effective ROI mapping
strategy. The proposed tracker also provides flexibility to couple with the
more advanced correlation filter trackers. The DeepHKCF tracker performs
exceptionally well with deep features set up in a synthetic hyperspectral video
generated by the Digital Imaging and Remote Sensing Image Generation (DIRSIG)
software. Additionally, we generate a large, synthetic, single-channel dataset
using DIRSIG to perform vehicle classification in the Wide Area Motion Imagery
(WAMI) platform. This way, the high-fidelity of the DIRSIG software is proved
and a large scale aerial vehicle classification dataset is released to support
studies on vehicle detection and tracking in the WAMI platform
Hyperspectral Image Classification with Attention Aided CNNs
Convolutional neural networks (CNNs) have been widely used for hyperspectral
image classification. As a common process, small cubes are firstly cropped from
the hyperspectral image and then fed into CNNs to extract spectral and spatial
features. It is well known that different spectral bands and spatial positions
in the cubes have different discriminative abilities. If fully explored, this
prior information will help improve the learning capacity of CNNs. Along this
direction, we propose an attention aided CNN model for spectral-spatial
classification of hyperspectral images. Specifically, a spectral attention
sub-network and a spatial attention sub-network are proposed for spectral and
spatial classification, respectively. Both of them are based on the traditional
CNN model, and incorporate attention modules to aid networks focus on more
discriminative channels or positions. In the final classification phase, the
spectral classification result and the spatial classification result are
combined together via an adaptively weighted summation method. To evaluate the
effectiveness of the proposed model, we conduct experiments on three standard
hyperspectral datasets. The experimental results show that the proposed model
can achieve superior performance compared to several state-of-the-art
CNN-related models
Multisource and Multitemporal Data Fusion in Remote Sensing
The sharp and recent increase in the availability of data captured by
different sensors combined with their considerably heterogeneous natures poses
a serious challenge for the effective and efficient processing of remotely
sensed data. Such an increase in remote sensing and ancillary datasets,
however, opens up the possibility of utilizing multimodal datasets in a joint
manner to further improve the performance of the processing approaches with
respect to the application at hand. Multisource data fusion has, therefore,
received enormous attention from researchers worldwide for a wide variety of
applications. Moreover, thanks to the revisit capability of several spaceborne
sensors, the integration of the temporal information with the spatial and/or
spectral/backscattering information of the remotely sensed data is possible and
helps to move from a representation of 2D/3D data to 4D data structures, where
the time variable adds new information as well as challenges for the
information extraction algorithms. There are a huge number of research works
dedicated to multisource and multitemporal data fusion, but the methods for the
fusion of different modalities have expanded in different paths according to
each research community. This paper brings together the advances of multisource
and multitemporal data fusion approaches with respect to different research
communities and provides a thorough and discipline-specific starting point for
researchers at different levels (i.e., students, researchers, and senior
researchers) willing to conduct novel investigations on this challenging topic
by supplying sufficient detail and references
- …