20 research outputs found
Energy-Based Generative Cooperative Saliency Prediction
Conventional saliency prediction models typically learn a deterministic
mapping from images to the corresponding ground truth saliency maps. In this
paper, we study the saliency prediction problem from the perspective of
generative models by learning a conditional probability distribution over
saliency maps given an image, and treating the prediction as a sampling
process. Specifically, we propose a generative cooperative saliency prediction
framework based on the generative cooperative networks, where a conditional
latent variable model and a conditional energy-based model are jointly trained
to predict saliency in a cooperative manner. We call our model the SalCoopNets.
The latent variable model serves as a fast but coarse predictor to efficiently
produce an initial prediction, which is then refined by the iterative Langevin
revision of the energy-based model that serves as a fine predictor. Such a
coarse-to-fine cooperative saliency prediction strategy offers the best of both
worlds. Moreover, we generalize our framework to the scenario of weakly
supervised saliency prediction, where saliency annotation of training images is
partially observed, by proposing a cooperative learning while recovering
strategy. Lastly, we show that the learned energy function can serve as a
refinement module that can refine the results of other pre-trained saliency
prediction models. Experimental results show that our generative model can
achieve state-of-the-art performance. Our code is publicly available at:
\url{https://github.com/JingZhang617/SalCoopNets}
Structure-Consistent Weakly Supervised Salient Object Detection with Local Saliency Coherence
Sparse labels have been attracting much attention in recent years. However,
the performance gap between weakly supervised and fully supervised salient
object detection methods is huge, and most previous weakly supervised works
adopt complex training methods with many bells and whistles. In this work, we
propose a one-round end-to-end training approach for weakly supervised salient
object detection via scribble annotations without pre/post-processing
operations or extra supervision data. Since scribble labels fail to offer
detailed salient regions, we propose a local coherence loss to propagate the
labels to unlabeled regions based on image features and pixel distance, so as
to predict integral salient regions with complete object structures. We design
a saliency structure consistency loss as self-consistent mechanism to ensure
consistent saliency maps are predicted with different scales of the same image
as input, which could be viewed as a regularization technique to enhance the
model generalization ability. Additionally, we design an aggregation module
(AGGM) to better integrate high-level features, low-level features and global
context information for the decoder to aggregate various information. Extensive
experiments show that our method achieves a new state-of-the-art performance on
six benchmarks (e.g. for the ECSSD dataset: F_\beta = 0.8995, E_\xi = 0.9079
and MAE = 0.0489$), with an average gain of 4.60\% for F-measure, 2.05\% for
E-measure and 1.88\% for MAE over the previous best method on this task. Source
code is available at http://github.com/siyueyu/SCWSSOD.Comment: Accepted by AAAI202
Few-Cost Salient Object Detection with Adversarial-Paced Learning
Detecting and segmenting salient objects from given image scenes has received
great attention in recent years. A fundamental challenge in training the
existing deep saliency detection models is the requirement of large amounts of
annotated data. While gathering large quantities of training data becomes cheap
and easy, annotating the data is an expensive process in terms of time, labor
and human expertise. To address this problem, this paper proposes to learn the
effective salient object detection model based on the manual annotation on a
few training images only, thus dramatically alleviating human labor in training
models. To this end, we name this task as the few-cost salient object detection
and propose an adversarial-paced learning (APL)-based framework to facilitate
the few-cost learning scenario. Essentially, APL is derived from the self-paced
learning (SPL) regime but it infers the robust learning pace through the
data-driven adversarial learning mechanism rather than the heuristic design of
the learning regularizer. Comprehensive experiments on four widely-used
benchmark datasets demonstrate that the proposed method can effectively
approach to the existing supervised deep salient object detection models with
only 1k human-annotated training images. The project page is available at
https://github.com/hb-stone/FC-SOD