294 research outputs found
Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation
Weakly supervised semantic segmentation is receiving great attention due to
its low human annotation cost. In this paper, we aim to tackle bounding box
supervised semantic segmentation, i.e., training accurate semantic segmentation
models using bounding box annotations as supervision. To this end, we propose
Affinity Attention Graph Neural Network (GNN). Following previous
practices, we first generate pseudo semantic-aware seeds, which are then formed
into semantic graphs based on our newly proposed affinity Convolutional Neural
Network (CNN). Then the built graphs are input to our GNN, in which an
affinity attention layer is designed to acquire the short- and long- distance
information from soft graph edges to accurately propagate semantic labels from
the confident seeds to the unlabeled pixels. However, to guarantee the
precision of the seeds, we only adopt a limited number of confident pixel seed
labels for GNN, which may lead to insufficient supervision for training.
To alleviate this issue, we further introduce a new loss function and a
consistency-checking mechanism to leverage the bounding box constraint, so that
more reliable guidance can be included for the model optimization. Experiments
show that our approach achieves new state-of-the-art performances on Pascal VOC
2012 datasets (val: 76.5\%, test: 75.2\%). More importantly, our approach can
be readily applied to bounding box supervised instance segmentation task or
other weakly supervised semantic segmentation tasks, with state-of-the-art or
comparable performance among almot all weakly supervised tasks on PASCAL VOC or
COCO dataset. Our source code will be available at
https://github.com/zbf1991/A2GNN.Comment: Accepted by IEEE Transactions on Pattern Analysis and Machine
Intelligence (TAPMI 2021
A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction
The rapid development of deep learning has made a great progress in image
segmentation, one of the fundamental tasks of computer vision. However, the
current segmentation algorithms mostly rely on the availability of pixel-level
annotations, which are often expensive, tedious, and laborious. To alleviate
this burden, the past years have witnessed an increasing attention in building
label-efficient, deep-learning-based image segmentation algorithms. This paper
offers a comprehensive review on label-efficient image segmentation methods. To
this end, we first develop a taxonomy to organize these methods according to
the supervision provided by different types of weak labels (including no
supervision, inexact supervision, incomplete supervision and inaccurate
supervision) and supplemented by the types of segmentation problems (including
semantic segmentation, instance segmentation and panoptic segmentation). Next,
we summarize the existing label-efficient image segmentation methods from a
unified perspective that discusses an important question: how to bridge the gap
between weak supervision and dense prediction -- the current methods are mostly
based on heuristic priors, such as cross-pixel similarity, cross-label
constraint, cross-view consistency, and cross-image relation. Finally, we share
our opinions about the future research directions for label-efficient deep
image segmentation.Comment: Accepted to IEEE TPAM
Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation on Point Clouds
Point clouds provide intrinsic geometric information and surface context for
scene understanding. Existing methods for point cloud segmentation require a
large amount of fully labeled data. Using advanced depth sensors, collection of
large scale 3D dataset is no longer a cumbersome process. However, manually
producing point-level label on the large scale dataset is time and
labor-intensive. In this paper, we propose a weakly supervised approach to
predict point-level results using weak labels on 3D point clouds. We introduce
our multi-path region mining module to generate pseudo point-level label from a
classification network trained with weak labels. It mines the localization cues
for each class from various aspects of the network feature using different
attention modules. Then, we use the point-level pseudo labels to train a point
cloud segmentation network in a fully supervised manner. To the best of our
knowledge, this is the first method that uses cloud-level weak labels on raw 3D
space to train a point cloud semantic segmentation network. In our setting, the
3D weak labels only indicate the classes that appeared in our input sample. We
discuss both scene- and subcloud-level weakly labels on raw 3D point cloud data
and perform in-depth experiments on them. On ScanNet dataset, our result
trained with subcloud-level labels is compatible with some fully supervised
methods.Comment: Accepted by CVPR202
- …