36 research outputs found
GFF: Gated Fully Fusion for Semantic Segmentation
Semantic segmentation generates comprehensive understanding of scenes through
densely predicting the category for each pixel. High-level features from Deep
Convolutional Neural Networks already demonstrate their effectiveness in
semantic segmentation tasks, however the coarse resolution of high-level
features often leads to inferior results for small/thin objects where detailed
information is important. It is natural to consider importing low level
features to compensate for the lost detailed information in high-level
features.Unfortunately, simply combining multi-level features suffers from the
semantic gap among them. In this paper, we propose a new architecture, named
Gated Fully Fusion (GFF), to selectively fuse features from multiple levels
using gates in a fully connected way. Specifically, features at each level are
enhanced by higher-level features with stronger semantics and lower-level
features with more details, and gates are used to control the propagation of
useful information which significantly reduces the noises during fusion. We
achieve the state of the art results on four challenging scene parsing datasets
including Cityscapes, Pascal Context, COCO-stuff and ADE20K.Comment: accepted by AAAI-2020(oral
The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification
Fine-grained classification is challenging because categories can only be
discriminated by subtle and local differences. Variances in the pose, scale or
rotation usually make the problem more difficult. Most fine-grained
classification systems follow the pipeline of finding foreground object or
object parts (where) to extract discriminative features (what).
In this paper, we propose to apply visual attention to fine-grained
classification task using deep neural network. Our pipeline integrates three
types of attention: the bottom-up attention that propose candidate patches, the
object-level top-down attention that selects relevant patches to a certain
object, and the part-level top-down attention that localizes discriminative
parts. We combine these attentions to train domain-specific deep nets, then use
it to improve both the what and where aspects. Importantly, we avoid using
expensive annotations like bounding box or part information from end-to-end.
The weak supervision constraint makes our work easier to generalize.
We have verified the effectiveness of the method on the subsets of ILSVRC2012
dataset and CUB200_2011 dataset. Our pipeline delivered significant
improvements and achieved the best accuracy under the weakest supervision
condition. The performance is competitive against other methods that rely on
additional annotations
SFNet: Faster and Accurate Semantic Segmentation via Semantic Flow
In this paper, we focus on exploring effective methods for faster and
accurate semantic segmentation. A common practice to improve the performance is
to attain high-resolution feature maps with strong semantic representation. Two
strategies are widely used: atrous convolutions and feature pyramid fusion,
while both are either computationally intensive or ineffective. Inspired by the
Optical Flow for motion alignment between adjacent video frames, we propose a
Flow Alignment Module (FAM) to learn \textit{Semantic Flow} between feature
maps of adjacent levels and broadcast high-level features to high-resolution
features effectively and efficiently. Furthermore, integrating our FAM to a
standard feature pyramid structure exhibits superior performance over other
real-time methods, even on lightweight backbone networks, such as ResNet-18 and
DFNet. Then to further speed up the inference procedure, we also present a
novel Gated Dual Flow Alignment Module to directly align high-resolution
feature maps and low-resolution feature maps where we term the improved version
network as SFNet-Lite. Extensive experiments are conducted on several
challenging datasets, where results show the effectiveness of both SFNet and
SFNet-Lite. In particular, when using Cityscapes test set, the SFNet-Lite
series achieve 80.1 mIoU while running at 60 FPS using ResNet-18 backbone and
78.8 mIoU while running at 120 FPS using STDC backbone on RTX-3090. Moreover,
we unify four challenging driving datasets into one large dataset, which we
named Unified Driving Segmentation (UDS) dataset. It contains diverse domain
and style information. We benchmark several representative works on UDS. Both
SFNet and SFNet-Lite still achieve the best speed and accuracy trade-off on
UDS, which serves as a strong baseline in such a challenging setting. The code
and models are publicly available at https://github.com/lxtGH/SFSegNets.Comment: IJCV-2023; Extension of Previous work arXiv:2002.1012