25,140 research outputs found
DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
A key problem in salient object detection is how to effectively model the
semantic properties of salient objects in a data-driven manner. In this paper,
we propose a multi-task deep saliency model based on a fully convolutional
neural network (FCNN) with global input (whole raw images) and global output
(whole saliency maps). In principle, the proposed saliency model takes a
data-driven strategy for encoding the underlying saliency prior information,
and then sets up a multi-task learning scheme for exploring the intrinsic
correlations between saliency detection and semantic image segmentation.
Through collaborative feature learning from such two correlated tasks, the
shared fully convolutional layers produce effective features for object
perception. Moreover, it is capable of capturing the semantic information on
salient objects across different levels using the fully convolutional layers,
which investigate the feature-sharing properties of salient object detection
with great feature redundancy reduction. Finally, we present a graph Laplacian
regularized nonlinear regression model for saliency refinement. Experimental
results demonstrate the effectiveness of our approach in comparison with the
state-of-the-art approaches.Comment: To appear in IEEE Transactions on Image Processing (TIP), Project
Website: http://www.zhaoliming.net/research/deepsalienc
Co-salient Object Detection Based on Deep Saliency Networks and Seed Propagation over an Integrated Graph
This paper presents a co-salient object detection method to find common
salient regions in a set of images. We utilize deep saliency networks to
transfer co-saliency prior knowledge and better capture high-level semantic
information, and the resulting initial co-saliency maps are enhanced by seed
propagation steps over an integrated graph. The deep saliency networks are
trained in a supervised manner to avoid online weakly supervised learning and
exploit them not only to extract high-level features but also to produce both
intra- and inter-image saliency maps. Through a refinement step, the initial
co-saliency maps can uniformly highlight co-salient regions and locate accurate
object boundaries. To handle input image groups inconsistent in size, we
propose to pool multi-regional descriptors including both within-segment and
within-group information. In addition, the integrated multilayer graph is
constructed to find the regions that the previous steps may not detect by seed
propagation with low-level descriptors. In this work, we utilize the useful
complementary components of high-, low-level information, and several
learning-based steps. Our experiments have demonstrated that the proposed
approach outperforms comparable co-saliency detection methods on widely used
public databases and can also be directly applied to co-segmentation tasks.Comment: 13 pages, 10 figures, 3 table
Salient Object Detection: A Discriminative Regional Feature Integration Approach
Salient object detection has been attracting a lot of interest, and recently
various heuristic computational models have been designed. In this paper, we
formulate saliency map computation as a regression problem. Our method, which
is based on multi-level image segmentation, utilizes the supervised learning
approach to map the regional feature vector to a saliency score. Saliency
scores across multiple levels are finally fused to produce the saliency map.
The contributions lie in two-fold. One is that we propose a discriminate
regional feature integration approach for salient object detection. Compared
with existing heuristic models, our proposed method is able to automatically
integrate high-dimensional regional saliency features and choose discriminative
ones. The other is that by investigating standard generic region properties as
well as two widely studied concepts for salient object detection, i.e.,
regional contrast and backgroundness, our approach significantly outperforms
state-of-the-art methods on six benchmark datasets. Meanwhile, we demonstrate
that our method runs as fast as most existing algorithms
Review of Visual Saliency Detection with Comprehensive Information
Visual saliency detection model simulates the human visual system to perceive
the scene, and has been widely used in many vision tasks. With the acquisition
technology development, more comprehensive information, such as depth cue,
inter-image correspondence, or temporal relationship, is available to extend
image saliency detection to RGBD saliency detection, co-saliency detection, or
video saliency detection. RGBD saliency detection model focuses on extracting
the salient regions from RGBD images by combining the depth information.
Co-saliency detection model introduces the inter-image correspondence
constraint to discover the common salient object in an image group. The goal of
video saliency detection model is to locate the motion-related salient object
in video sequences, which considers the motion cue and spatiotemporal
constraint jointly. In this paper, we review different types of saliency
detection algorithms, summarize the important issues of the existing methods,
and discuss the existent problems and future works. Moreover, the evaluation
datasets and quantitative measurements are briefly introduced, and the
experimental analysis and discission are conducted to provide a holistic
overview of different saliency detection methods.Comment: 18 pages, 11 figures, 7 tables, Accepted by IEEE Transactions on
Circuits and Systems for Video Technology 2018, https://rmcong.github.io
Multi-interactive Dual-decoder for RGB-thermal Salient Object Detection
RGB-thermal salient object detection (SOD) aims to segment the common
prominent regions of visible image and corresponding thermal infrared image
that we call it RGBT SOD. Existing methods don't fully explore and exploit the
potentials of complementarity of different modalities and multi-type cues of
image contents, which play a vital role in achieving accurate results. In this
paper, we propose a multi-interactive dual-decoder to mine and model the
multi-type interactions for accurate RGBT SOD. In specific, we first encode two
modalities into multi-level multi-modal feature representations. Then, we
design a novel dual-decoder to conduct the interactions of multi-level
features, two modalities and global contexts. With these interactions, our
method works well in diversely challenging scenarios even in the presence of
invalid modality. Finally, we carry out extensive experiments on public RGBT
and RGBD SOD datasets, and the results show that the proposed method achieves
the outstanding performance against state-of-the-art algorithms. The source
code has been released
at:https://github.com/lz118/Multi-interactive-Dual-decoder.Comment: Accepted by IEEE TI
Video Salient Object Detection Using Spatiotemporal Deep Features
This paper presents a method for detecting salient objects in videos where
temporal information in addition to spatial information is fully taken into
account. Following recent reports on the advantage of deep features over
conventional hand-crafted features, we propose a new set of SpatioTemporal Deep
(STD) features that utilize local and global contexts over frames. We also
propose new SpatioTemporal Conditional Random Field (STCRF) to compute saliency
from STD features. STCRF is our extension of CRF to the temporal domain and
describes the relationships among neighboring regions both in a frame and over
frames. STCRF leads to temporally consistent saliency maps over frames,
contributing to the accurate detection of salient objects' boundaries and noise
reduction during detection. Our proposed method first segments an input video
into multiple scales and then computes a saliency map at each scale level using
STD features with STCRF. The final saliency map is computed by fusing saliency
maps at different scale levels. Our experiments, using publicly available
benchmark datasets, confirm that the proposed method significantly outperforms
state-of-the-art methods. We also applied our saliency computation to the video
object segmentation task, showing that our method outperforms existing video
object segmentation methods.Comment: accepted at TI
A dense subgraph based algorithm for compact salient image region detection
We present an algorithm for graph based saliency computation that utilizes
the underlying dense subgraphs in finding visually salient regions in an image.
To compute the salient regions, the model first obtains a saliency map using
random walks on a Markov chain. Next, k-dense subgraphs are detected to further
enhance the salient regions in the image. Dense subgraphs convey more
information about local graph structure than simple centrality measures. To
generate the Markov chain, intensity and color features of an image in addition
to region compactness is used. For evaluating the proposed model, we do
extensive experiments on benchmark image data sets. The proposed method
performs comparable to well-known algorithms in salient region detection.Comment: 33 pages, 18 figures, Single column manuscript pre-print, Accepted at
Computer Vision and Image Understanding, Elsevie
A novel graph structure for salient object detection based on divergence background and compact foreground
In this paper, we propose an efficient and discriminative model for salient
object detection. Our method is carried out in a stepwise mechanism based on
both divergence background and compact foreground cues. In order to effectively
enhance the distinction between nodes along object boundaries and the
similarity among object regions, a graph is constructed by introducing the
concept of virtual node. To remove incorrect outputs, a scheme for selecting
background seeds and a method for generating compactness foreground regions are
introduced, respectively. Different from prior methods, we calculate the
saliency value of each node based on the relationship between the corresponding
node and the virtual node. In order to achieve significant performance
improvement consistently, we propose an Extended Manifold Ranking (EMR)
algorithm, which subtly combines suppressed / active nodes and mid-level
information. Extensive experimental results demonstrate that the proposed
algorithm performs favorably against the state-of-art saliency detection
methods in terms of different evaluation metrics on several benchmark datasets.Comment: 22 pages,16 figures, 2 table
Hierarchical Saliency Detection on Extended CSSD
Complex structures commonly exist in natural images. When an image contains
small-scale high-contrast patterns either in the background or foreground,
saliency detection could be adversely affected, resulting erroneous and
non-uniform saliency assignment. The issue forms a fundamental challenge for
prior methods. We tackle it from a scale point of view and propose a
multi-layer approach to analyze saliency cues. Different from varying patch
sizes or downsizing images, we measure region-based scales. The final saliency
values are inferred optimally combining all the saliency cues in different
scales using hierarchical inference. Through our inference model, single-scale
information is selected to obtain a saliency map. Our method improves detection
quality on many images that cannot be handled well traditionally. We also
construct an extended Complex Scene Saliency Dataset (ECSSD) to include complex
but general natural images.Comment: 14 pages, 15 figure
Automatic Salient Object Detection for Panoramic Images Using Region Growing and Fixation Prediction Model
Almost all previous works on saliency detection have been dedicated to
conventional images, however, with the outbreak of panoramic images due to the
rapid development of VR or AR technology, it is becoming more challenging,
meanwhile valuable for extracting salient contents in panoramic images.
In this paper, we propose a novel bottom-up salient object detection
framework for panoramic images. First, we employ a spatial density estimation
method to roughly extract object proposal regions, with the help of region
growing algorithm. Meanwhile, an eye fixation model is utilized to predict
visually attractive parts in the image from the perspective of the human visual
search mechanism. Then, the previous results are combined by the maxima
normalization to get the coarse saliency map. Finally, a refinement step based
on geodesic distance is utilized for post-processing to derive the final
saliency map.
To fairly evaluate the performance of the proposed approach, we propose a
high-quality dataset of panoramic images (SalPan). Extensive evaluations
demonstrate the effectiveness of our proposed method on panoramic images and
the superiority of the proposed method against other methods.Comment: Previous Project website: https://github.com/ChunbiaoZhu/DCC-201
- …