10 research outputs found
Adaptive Graph Convolutional Network with Attention Graph Clustering for Co-saliency Detection
Co-saliency detection aims to discover the common and salient foregrounds
from a group of relevant images. For this task, we present a novel adaptive
graph convolutional network with attention graph clustering (GCAGC). Three
major contributions have been made, and are experimentally shown to have
substantial practical merits. First, we propose a graph convolutional network
design to extract information cues to characterize the intra- and interimage
correspondence. Second, we develop an attention graph clustering algorithm to
discriminate the common objects from all the salient foreground objects in an
unsupervised fashion. Third, we present a unified framework with
encoder-decoder structure to jointly train and optimize the graph convolutional
network, attention graph cluster, and co-saliency detection decoder in an
end-to-end manner. We evaluate our proposed GCAGC method on three cosaliency
detection benchmark datasets (iCoseg, Cosal2015 and COCO-SEG). Our GCAGC method
obtains significant improvements over the state-of-the-arts on most of them.Comment: CVPR202
Co-Salient Object Detection with Co-Representation Purification
Co-salient object detection (Co-SOD) aims at discovering the common objects
in a group of relevant images. Mining a co-representation is essential for
locating co-salient objects. Unfortunately, the current Co-SOD method does not
pay enough attention that the information not related to the co-salient object
is included in the co-representation. Such irrelevant information in the
co-representation interferes with its locating of co-salient objects. In this
paper, we propose a Co-Representation Purification (CoRP) method aiming at
searching noise-free co-representation. We search a few pixel-wise embeddings
probably belonging to co-salient regions. These embeddings constitute our
co-representation and guide our prediction. For obtaining purer
co-representation, we use the prediction to iteratively reduce irrelevant
embeddings in our co-representation. Experiments on three datasets demonstrate
that our CoRP achieves state-of-the-art performances on the benchmark datasets.
Our source code is available at https://github.com/ZZY816/CoRP.Comment: Accepted by TPAMI 202
Video Desnowing and Deraining via Saliency and Dual Adaptive Spatiotemporal Filtering
Outdoor vision sensing systems often struggle with poor weather conditions, such as snow and rain, which poses a great challenge to existing video desnowing and deraining methods. In this paper, we propose a novel video desnowing and deraining model that utilizes the salience information of moving objects to address this problem. First, we remove the snow and rain from the video by low-rank tensor decomposition, which makes full use of the spatial location information and the correlation between the three channels of the color video. Second, because existing algorithms often regard sparse snowflakes and rain streaks as moving objects, this paper injects salience information into moving object detection, which reduces the false alarms and missed alarms of moving objects. At the same time, feature point matching is used to mine the redundant information of moving objects in continuous frames, and a dual adaptive minimum filtering algorithm in the spatiotemporal domain is proposed by us to remove snow and rain in front of moving objects. Both qualitative and quantitative experimental results show that the proposed algorithm is more competitive than other state-of-the-art snow and rain removal methods
Re-thinking Co-Salient Object Detection
In this paper, we conduct a comprehensive study on the co-salient object
detection (CoSOD) problem for images. CoSOD is an emerging and rapidly growing
extension of salient object detection (SOD), which aims to detect the
co-occurring salient objects in a group of images. However, existing CoSOD
datasets often have a serious data bias, assuming that each group of images
contains salient objects of similar visual appearances. This bias can lead to
the ideal settings and effectiveness of models trained on existing datasets,
being impaired in real-life situations, where similarities are usually semantic
or conceptual. To tackle this issue, we first introduce a new benchmark, called
CoSOD3k in the wild, which requires a large amount of semantic context, making
it more challenging than existing CoSOD datasets. Our CoSOD3k consists of 3,316
high-quality, elaborately selected images divided into 160 groups with
hierarchical annotations. The images span a wide range of categories, shapes,
object sizes, and backgrounds. Second, we integrate the existing SOD techniques
to build a unified, trainable CoSOD framework, which is long overdue in this
field. Specifically, we propose a novel CoEG-Net that augments our prior model
EGNet with a co-attention projection strategy to enable fast common information
learning. CoEG-Net fully leverages previous large-scale SOD datasets and
significantly improves the model scalability and stability. Third, we
comprehensively summarize 40 cutting-edge algorithms, benchmarking 18 of them
over three challenging CoSOD datasets (iCoSeg, CoSal2015, and our CoSOD3k), and
reporting more detailed (i.e., group-level) performance analysis. Finally, we
discuss the challenges and future works of CoSOD. We hope that our study will
give a strong boost to growth in the CoSOD community. The benchmark toolbox and
results are available on our project page at http://dpfan.net/CoSOD3K/.Comment: 22pages, 18 figures. CVPR2020-CoSOD3K extension. Code:
https://github.com/DengPingFan/CoEGNe
Robust Deep Co-Saliency Detection with Group Semantic
High-level semantic knowledge in addition to low-level visual cues is essentially crucial for co-saliency detection. This paper proposes a novel end-to-end deep learning approach for robust co-saliency detection by simultaneously learning highlevel group-wise semantic representation as well as deep visual features of a given image group. The inter-image interaction at semantic-level as well as the complementarity between group semantics and visual features are exploited to boost the inferring of co-salient regions. Specifically, the proposed approach consists of a co-category learning branch and a co-saliency detection branch. While the former is proposed to learn group-wise semantic vector using co-category association of an image group as supervision, the latter is to infer precise co-salient maps based on the ensemble of group semantic knowledge and deep visual cues. The group semantic vector is broadcasted to each spatial location of multi-scale visual feature maps and is used as a top-down semantic guidance for boosting the bottom-up inferring of co-saliency. The co-category learning and co-saliency detection branches are jointly optimized in a multi-task learning manner, further improving the robustness of the approach. Moreover, we construct a new large-scale co-saliency dataset COCO-SEG to facilitate research of co-saliency detection. Extensive experimental results on COCO-SEG and a widely used benchmark Cosal2015 have demonstrated the superiority of the proposed approach as compared to the state-of-the-art methods