2,539 research outputs found
P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation
Recently, Transformer-based models have achieved promising results in various
vision tasks, due to their ability to model long-range dependencies. However,
transformers are computationally expensive, which limits their applications in
real-time tasks such as autonomous driving. In addition, an efficient local and
global feature selection and fusion are vital for accurate dense prediction,
especially driving scene understanding tasks. In this paper, we propose a
real-time semantic segmentation architecture named Pyramid Pooling Axial
Transformer (P2AT). The proposed P2AT takes a coarse feature from the CNN
encoder to produce scale-aware contextual features, which are then combined
with the multi-level feature aggregation scheme to produce enhanced contextual
features. Specifically, we introduce a pyramid pooling axial transformer to
capture intricate spatial and channel dependencies, leading to improved
performance on semantic segmentation. Then, we design a Bidirectional Fusion
module (BiF) to combine semantic information at different levels. Meanwhile, a
Global Context Enhancer is introduced to compensate for the inadequacy of
concatenating different semantic levels. Finally, a decoder block is proposed
to help maintain a larger receptive field. We evaluate P2AT variants on three
challenging scene-understanding datasets. In particular, our P2AT variants
achieve state-of-art results on the Camvid dataset 80.5%, 81.0%, 81.1% for
P2AT-S, P2ATM, and P2AT-L, respectively. Furthermore, our experiment on
Cityscapes and Pascal VOC 2012 have demonstrated the efficiency of the proposed
architecture, with results showing that P2AT-M, achieves 78.7% on Cityscapes.
The source code will be available a
StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection
One-stage object detectors such as SSD or YOLO already have shown promising
accuracy with small memory footprint and fast speed. However, it is widely
recognized that one-stage detectors have difficulty in detecting small objects
while they are competitive with two-stage methods on large objects. In this
paper, we investigate how to alleviate this problem starting from the SSD
framework. Due to their pyramidal design, the lower layer that is responsible
for small objects lacks strong semantics(e.g contextual information). We
address this problem by introducing a feature combining module that spreads out
the strong semantics in a top-down manner. Our final model StairNet detector
unifies the multi-scale representations and semantic distribution effectively.
Experiments on PASCAL VOC 2007 and PASCAL VOC 2012 datasets demonstrate that
StairNet significantly improves the weakness of SSD and outperforms the other
state-of-the-art one-stage detectors
Attention guided global enhancement and local refinement network for semantic segmentation
The encoder-decoder architecture is widely used as a lightweight semantic
segmentation network. However, it struggles with a limited performance compared
to a well-designed Dilated-FCN model for two major problems. First, commonly
used upsampling methods in the decoder such as interpolation and deconvolution
suffer from a local receptive field, unable to encode global contexts. Second,
low-level features may bring noises to the network decoder through skip
connections for the inadequacy of semantic concepts in early encoder layers. To
tackle these challenges, a Global Enhancement Method is proposed to aggregate
global information from high-level feature maps and adaptively distribute them
to different decoder layers, alleviating the shortage of global contexts in the
upsampling process. Besides, a Local Refinement Module is developed by
utilizing the decoder features as the semantic guidance to refine the noisy
encoder features before the fusion of these two (the decoder features and the
encoder features). Then, the two methods are integrated into a Context Fusion
Block, and based on that, a novel Attention guided Global enhancement and Local
refinement Network (AGLN) is elaborately designed. Extensive experiments on
PASCAL Context, ADE20K, and PASCAL VOC 2012 datasets have demonstrated the
effectiveness of the proposed approach. In particular, with a vanilla
ResNet-101 backbone, AGLN achieves the state-of-the-art result (56.23% mean
IoU) on the PASCAL Context dataset. The code is available at
https://github.com/zhasen1996/AGLN.Comment: 12 pages, 6 figure
- …