312 research outputs found
Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
By considering the spectral signature as a sequence, recurrent neural
networks (RNNs) have been successfully used to learn discriminative features
from hyperspectral images (HSIs) recently. However, most of these models only
input the whole spectral bands into RNNs directly, which may not fully explore
the specific properties of HSIs. In this paper, we propose a cascaded RNN model
using gated recurrent units (GRUs) to explore the redundant and complementary
information of HSIs. It mainly consists of two RNN layers. The first RNN layer
is used to eliminate redundant information between adjacent spectral bands,
while the second RNN layer aims to learn the complementary information from
non-adjacent spectral bands. To improve the discriminative ability of the
learned features, we design two strategies for the proposed model. Besides,
considering the rich spatial information contained in HSIs, we further extend
the proposed model to its spectral-spatial counterpart by incorporating some
convolutional layers. To test the effectiveness of our proposed models, we
conduct experiments on two widely used HSIs. The experimental results show that
our proposed models can achieve better results than the compared models
Spectral feature fusion networks with dual attention for hyperspectral image classification
Recent progress in spectral classification is largely attributed to the use of convolutional neural networks (CNN).
While a variety of successful architectures have been proposed, they all extract spectral features from various portions of adjacent spectral bands. In this paper, we take a different approach and develop a deep spectral feature fusion method, which extracts both local and interlocal spectral features, capturing thus also the correlations among non-adjacent bands. To our knowledge, this is the first reported deep spectral feature fusion method. Our model is a two-stream architecture, where an intergroup and a groupwise spectral classifiers operate in parallel. The interlocal spectral correlation feature extraction is achieved elegantly, by reshaping the input spectral vectors to form the socalled non-adjacent spectral matrices. We introduce the concept of groupwise band convolution to enable efficient extraction of
discriminative local features with multiple kernels adopting to the local spectral content. Another important contribution of this work is a novel dual-channel attention mechanism to identify the most informative spectral features. The model is trained in an end-to-end fashion with a joint loss. Experimental results on real data sets demonstrate excellent performance compared to the current state-of-the-art
SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers
Hyperspectral (HS) images are characterized by approximately contiguous
spectral information, enabling the fine identification of materials by
capturing subtle spectral discrepancies. Owing to their excellent locally
contextual modeling ability, convolutional neural networks (CNNs) have been
proven to be a powerful feature extractor in HS image classification. However,
CNNs fail to mine and represent the sequence attributes of spectral signatures
well due to the limitations of their inherent network backbone. To solve this
issue, we rethink HS image classification from a sequential perspective with
transformers, and propose a novel backbone network called \ul{SpectralFormer}.
Beyond band-wise representations in classic transformers, SpectralFormer is
capable of learning spectrally local sequence information from neighboring
bands of HS images, yielding group-wise spectral embeddings. More
significantly, to reduce the possibility of losing valuable information in the
layer-wise propagation process, we devise a cross-layer skip connection to
convey memory-like components from shallow to deep layers by adaptively
learning to fuse "soft" residuals across layers. It is worth noting that the
proposed SpectralFormer is a highly flexible backbone network, which can be
applicable to both pixel- and patch-wise inputs. We evaluate the classification
performance of the proposed SpectralFormer on three HS datasets by conducting
extensive experiments, showing the superiority over classic transformers and
achieving a significant improvement in comparison with state-of-the-art
backbone networks. The codes of this work will be available at
https://github.com/danfenghong/IEEE_TGRS_SpectralFormer for the sake of
reproducibility
SaaFormer: Spectral-spatial Axial Aggregation Transformer for Hyperspectral Image Classification
Hyperspectral images (HSI) captured from earth observing satellites and
aircraft is becoming increasingly important for applications in agriculture,
environmental monitoring, mining, etc. Due to the limited available
hyperspectral datasets, the pixel-wise random sampling is the most commonly
used training-test dataset partition approach, which has significant overlap
between samples in training and test datasets. Furthermore, our experimental
observations indicates that regions with larger overlap often exhibit higher
classification accuracy. Consequently, the pixel-wise random sampling approach
poses a risk of data leakage. Thus, we propose a block-wise sampling method to
minimize the potential for data leakage. Our experimental findings also confirm
the presence of data leakage in models such as 2DCNN. Further, We propose a
spectral-spatial axial aggregation transformer model, namely SaaFormer, to
address the challenges associated with hyperspectral image classifier that
considers HSI as long sequential three-dimensional images. The model comprises
two primary components: axial aggregation attention and multi-level
spectral-spatial extraction. The axial aggregation attention mechanism
effectively exploits the continuity and correlation among spectral bands at
each pixel position in hyperspectral images, while aggregating spatial
dimension features. This enables SaaFormer to maintain high precision even
under block-wise sampling. The multi-level spectral-spatial extraction
structure is designed to capture the sensitivity of different material
components to specific spectral bands, allowing the model to focus on a broader
range of spectral details. The results on six publicly available datasets
demonstrate that our model exhibits comparable performance when using random
sampling, while significantly outperforming other methods when employing
block-wise sampling partition.Comment: arXiv admin note: text overlap with arXiv:2107.02988 by other author
Deep Intrinsic Decomposition with Adversarial Learning for Hyperspectral Image Classification
Convolutional neural networks (CNNs) have been demonstrated their powerful
ability to extract discriminative features for hyperspectral image
classification. However, general deep learning methods for CNNs ignore the
influence of complex environmental factor which enlarges the intra-class
variance and decreases the inter-class variance. This multiplies the difficulty
to extract discriminative features. To overcome this problem, this work
develops a novel deep intrinsic decomposition with adversarial learning, namely
AdverDecom, for hyperspectral image classification to mitigate the negative
impact of environmental factors on classification performance. First, we
develop a generative network for hyperspectral image (HyperNet) to extract the
environmental-related feature and category-related feature from the image.
Then, a discriminative network is constructed to distinguish different
environmental categories. Finally, a environmental and category joint learning
loss is developed for adversarial learning to make the deep model learn
discriminative features. Experiments are conducted over three commonly used
real-world datasets and the comparison results show the superiority of the
proposed method. The implementation of the proposed method and other compared
methods could be accessed at https://github.com/shendu-sw/Adversarial Learning
Intrinsic Decomposition for the sake of reproducibility.Comment: Submitted to IEEE TI
- …