4,361 research outputs found
Graph Distillation for Action Detection with Privileged Modalities
We propose a technique that tackles action detection in multimodal videos
under a realistic and challenging condition in which only limited training data
and partially observed modalities are available. Common methods in transfer
learning do not take advantage of the extra modalities potentially available in
the source domain. On the other hand, previous work on multimodal learning only
focuses on a single domain or task and does not handle the modality discrepancy
between training and testing. In this work, we propose a method termed graph
distillation that incorporates rich privileged information from a large-scale
multimodal dataset in the source domain, and improves the learning in the
target domain where training data and modalities are scarce. We evaluate our
approach on action classification and detection tasks in multimodal videos, and
show that our model outperforms the state-of-the-art by a large margin on the
NTU RGB+D and PKU-MMD benchmarks. The code is released at
http://alan.vision/eccv18_graph/.Comment: ECCV 201
Learning Segmentation Masks with the Independence Prior
An instance with a bad mask might make a composite image that uses it look
fake. This encourages us to learn segmentation by generating realistic
composite images. To achieve this, we propose a novel framework that exploits a
new proposed prior called the independence prior based on Generative
Adversarial Networks (GANs). The generator produces an image with multiple
category-specific instance providers, a layout module and a composition module.
Firstly, each provider independently outputs a category-specific instance image
with a soft mask. Then the provided instances' poses are corrected by the
layout module. Lastly, the composition module combines these instances into a
final image. Training with adversarial loss and penalty for mask area, each
provider learns a mask that is as small as possible but enough to cover a
complete category-specific instance. Weakly supervised semantic segmentation
methods widely use grouping cues modeling the association between image parts,
which are either artificially designed or learned with costly segmentation
labels or only modeled on local pairs. Unlike them, our method automatically
models the dependence between any parts and learns instance segmentation. We
apply our framework in two cases: (1) Foreground segmentation on
category-specific images with box-level annotation. (2) Unsupervised learning
of instance appearances and masks with only one image of homogeneous object
cluster (HOC). We get appealing results in both tasks, which shows the
independence prior is useful for instance segmentation and it is possible to
unsupervisedly learn instance masks with only one image.Comment: 7+5 pages, 13 figures, Accepted to AAAI 201
On Representation Knowledge Distillation for Graph Neural Networks
Knowledge distillation is a learning paradigm for boosting resource-efficient
graph neural networks (GNNs) using more expressive yet cumbersome teacher
models. Past work on distillation for GNNs proposed the Local Structure
Preserving loss (LSP), which matches local structural relationships defined
over edges across the student and teacher's node embeddings. This paper studies
whether preserving the global topology of how the teacher embeds graph data can
be a more effective distillation objective for GNNs, as real-world graphs often
contain latent interactions and noisy edges. We propose Graph Contrastive
Representation Distillation (G-CRD), which uses contrastive learning to
implicitly preserve global topology by aligning the student node embeddings to
those of the teacher in a shared representation space. Additionally, we
introduce an expanded set of benchmarks on large-scale real-world datasets
where the performance gap between teacher and student GNNs is non-negligible.
Experiments across 4 datasets and 14 heterogeneous GNN architectures show that
G-CRD consistently boosts the performance and robustness of lightweight GNNs,
outperforming LSP (and a global structure preserving variant of LSP) as well as
baselines from 2D computer vision. An analysis of the representational
similarity among teacher and student embedding spaces reveals that G-CRD
balances preserving local and global relationships, while structure preserving
approaches are best at preserving one or the other
Neural Motifs: Scene Graph Parsing with Global Context
We investigate the problem of producing structured graph representations of
visual scenes. Our work analyzes the role of motifs: regularly appearing
substructures in scene graphs. We present new quantitative insights on such
repeated structures in the Visual Genome dataset. Our analysis shows that
object labels are highly predictive of relation labels but not vice-versa. We
also find that there are recurring patterns even in larger subgraphs: more than
50% of graphs contain motifs involving at least two relations. Our analysis
motivates a new baseline: given object detections, predict the most frequent
relation between object pairs with the given labels, as seen in the training
set. This baseline improves on the previous state-of-the-art by an average of
3.6% relative improvement across evaluation settings. We then introduce Stacked
Motif Networks, a new architecture designed to capture higher order motifs in
scene graphs that further improves over our strong baseline by an average 7.1%
relative gain. Our code is available at github.com/rowanz/neural-motifs.Comment: CVPR 2018 camera read
FGAD: Self-boosted Knowledge Distillation for An Effective Federated Graph Anomaly Detection Framework
Graph anomaly detection (GAD) aims to identify anomalous graphs that
significantly deviate from other ones, which has raised growing attention due
to the broad existence and complexity of graph-structured data in many
real-world scenarios. However, existing GAD methods usually execute with
centralized training, which may lead to privacy leakage risk in some sensitive
cases, thereby impeding collaboration among organizations seeking to
collectively develop robust GAD models. Although federated learning offers a
promising solution, the prevalent non-IID problems and high communication costs
present significant challenges, particularly pronounced in collaborations with
graph data distributed among different participants. To tackle these
challenges, we propose an effective federated graph anomaly detection framework
(FGAD). We first introduce an anomaly generator to perturb the normal graphs to
be anomalous, and train a powerful anomaly detector by distinguishing generated
anomalous graphs from normal ones. Then, we leverage a student model to distill
knowledge from the trained anomaly detector (teacher model), which aims to
maintain the personality of local models and alleviate the adverse impact of
non-IID problems. Moreover, we design an effective collaborative learning
mechanism that facilitates the personalization preservation of local models and
significantly reduces communication costs among clients. Empirical results of
the GAD tasks on non-IID graphs compared with state-of-the-art baselines
demonstrate the superiority and efficiency of the proposed FGAD method
Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces
Continual graph learning routinely finds its role in a variety of real-world
applications where the graph data with different tasks come sequentially.
Despite the success of prior works, it still faces great challenges. On the one
hand, existing methods work with the zero-curvature Euclidean space, and
largely ignore the fact that curvature varies over the coming graph sequence.
On the other hand, continual learners in the literature rely on abundant
labels, but labeling graph in practice is particularly hard especially for the
continuously emerging graphs on-the-fly. To address the aforementioned
challenges, we propose to explore a challenging yet practical problem, the
self-supervised continual graph learning in adaptive Riemannian spaces. In this
paper, we propose a novel self-supervised Riemannian Graph Continual Learner
(RieGrace). In RieGrace, we first design an Adaptive Riemannian GCN (AdaRGCN),
a unified GCN coupled with a neural curvature adapter, so that Riemannian space
is shaped by the learnt curvature adaptive to each graph. Then, we present a
Label-free Lorentz Distillation approach, in which we create teacher-student
AdaRGCN for the graph sequence. The student successively performs
intra-distillation from itself and inter-distillation from the teacher so as to
consolidate knowledge without catastrophic forgetting. In particular, we
propose a theoretically grounded Generalized Lorentz Projection for the
contrastive distillation in Riemannian space. Extensive experiments on the
benchmark datasets show the superiority of RieGrace, and additionally, we
investigate on how curvature changes over the graph sequence.Comment: Accepted by AAAI 2023 (Main Track), 9 pages, 4 figure
- …