242 research outputs found
Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation
We propose an approach to discover class-specific pixels for the
weakly-supervised semantic segmentation task. We show that properly combining
saliency and attention maps allows us to obtain reliable cues capable of
significantly boosting the performance. First, we propose a simple yet powerful
hierarchical approach to discover the class-agnostic salient regions, obtained
using a salient object detector, which otherwise would be ignored. Second, we
use fully convolutional attention maps to reliably localize the class-specific
regions in a given image. We combine these two cues to discover class-specific
pixels which are then used as an approximate ground truth for training a CNN.
While solving the weakly supervised semantic segmentation task, we ensure that
the image-level classification task is also solved in order to enforce the CNN
to assign at least one pixel to each object present in the image.
Experimentally, on the PASCAL VOC12 val and test sets, we obtain the mIoU of
60.8% and 61.9%, achieving the performance gains of 5.1% and 5.2% compared to
the published state-of-the-art results. The code is made publicly available
Stable Rank Normalization for Improved Generalization in Neural Networks and GANs
Exciting new work on the generalization bounds for neural networks (NN) given
by Neyshabur et al. , Bartlett et al. closely depend on two
parameter-depenedent quantities: the Lipschitz constant upper-bound and the
stable rank (a softer version of the rank operator). This leads to an
interesting question of whether controlling these quantities might improve the
generalization behaviour of NNs. To this end, we propose stable rank
normalization (SRN), a novel, optimal, and computationally efficient
weight-normalization scheme which minimizes the stable rank of a linear
operator. Surprisingly we find that SRN, inspite of being non-convex problem,
can be shown to have a unique optimal solution. Moreover, we show that SRN
allows control of the data-dependent empirical Lipschitz constant, which in
contrast to the Lipschitz upper-bound, reflects the true behaviour of a model
on a given dataset. We provide thorough analyses to show that SRN, when applied
to the linear layers of a NN for classification, provides striking
improvements-11.3% on the generalization gap compared to the standard NN along
with significant reduction in memorization. When applied to the discriminator
of GANs (called SRN-GAN) it improves Inception, FID, and Neural divergence
scores on the CIFAR 10/100 and CelebA datasets, while learning mappings with
low empirical Lipschitz constants.Comment: Accepted at the International Conference in Learning Representations,
2020, Addis Ababa, Ethiopi
Continual Learning in Low-rank Orthogonal Subspaces
In continual learning (CL), a learner is faced with a sequence of tasks,
arriving one after the other, and the goal is to remember all the tasks once
the continual learning experience is finished. The prior art in CL uses
episodic memory, parameter regularization or extensible network structures to
reduce interference among tasks, but in the end, all the approaches learn
different tasks in a joint vector space. We believe this invariably leads to
interference among different tasks. We propose to learn tasks in different
(low-rank) vector subspaces that are kept orthogonal to each other in order to
minimize interference. Further, to keep the gradients of different tasks coming
from these subspaces orthogonal to each other, we learn isometric mappings by
posing network training as an optimization problem over the Stiefel manifold.
To the best of our understanding, we report, for the first time, strong results
over experience-replay baseline with and without memory on standard
classification benchmarks in continual learning. The code is made publicly
available.Comment: The paper is accepted at NeurIPS'2
Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentation
Referring Image Segmentation (RIS) - the problem of identifying objects in
images through natural language sentences - is a challenging task currently
mostly solved through supervised learning. However, while collecting referred
annotation masks is a time-consuming process, the few existing
weakly-supervised and zero-shot approaches fall significantly short in
performance compared to fully-supervised learning ones. To bridge the
performance gap without mask annotations, we propose a novel weakly-supervised
framework that tackles RIS by decomposing it into three steps: obtaining
instance masks for the object mentioned in the referencing instruction
(segment), using zero-shot learning to select a potentially correct mask for
the given instruction (select), and bootstrapping a model which allows for
fixing the mistakes of zero-shot selection (correct). In our experiments, using
only the first two steps (zero-shot segment and select) outperforms other
zero-shot baselines by as much as 19%, while our full method improves upon this
much stronger baseline and sets the new state-of-the-art for weakly-supervised
RIS, reducing the gap between the weakly-supervised and fully-supervised
methods in some cases from around 33% to as little as 14%. Code is available at
https://github.com/fgirbal/segment-select-correct
- …