15,915 research outputs found
Dual Relation Knowledge Distillation for Object Detection
Knowledge distillation is an effective method for model compression. However,
it is still a challenging topic to apply knowledge distillation to detection
tasks. There are two key points resulting in poor distillation performance for
detection tasks. One is the serious imbalance between foreground and background
features, another one is that small object lacks enough feature representation.
To solve the above issues, we propose a new distillation method named dual
relation knowledge distillation (DRKD), including pixel-wise relation
distillation and instance-wise relation distillation. The pixel-wise relation
distillation embeds pixel-wise features in the graph space and applies graph
convolution to capture the global pixel relation. By distilling the global
pixel relation, the student detector can learn the relation between foreground
and background features, and avoid the difficulty of distilling features
directly for the feature imbalance issue. Besides, we find that instance-wise
relation supplements valuable knowledge beyond independent features for small
objects. Thus, the instance-wise relation distillation is designed, which
calculates the similarity of different instances to obtain a relation matrix.
More importantly, a relation filter module is designed to highlight valuable
instance relations. The proposed dual relation knowledge distillation is
general and can be easily applied for both one-stage and two-stage detectors.
Our method achieves state-of-the-art performance, which improves Faster R-CNN
based on ResNet50 from 38.4% to 41.6% mAP and improves RetinaNet based on
ResNet50 from 37.4% to 40.3% mAP on COCO 2017.Comment: Accepted by IJCAI-202
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
Many recent works on knowledge distillation have provided ways to transfer
the knowledge of a trained network for improving the learning process of a new
one, but finding a good technique for knowledge distillation is still an open
problem. In this paper, we provide a new perspective based on a decision
boundary, which is one of the most important component of a classifier. The
generalization performance of a classifier is closely related to the adequacy
of its decision boundary, so a good classifier bears a good decision boundary.
Therefore, transferring information closely related to the decision boundary
can be a good attempt for knowledge distillation. To realize this goal, we
utilize an adversarial attack to discover samples supporting a decision
boundary. Based on this idea, to transfer more accurate information about the
decision boundary, the proposed algorithm trains a student classifier based on
the adversarial samples supporting the decision boundary. Experiments show that
the proposed method indeed improves knowledge distillation and achieves the
state-of-the-arts performance.Comment: Accepted to AAAI 201
Joint Topic-Semantic-aware Social Recommendation for Online Voting
Online voting is an emerging feature in social networks, in which users can
express their attitudes toward various issues and show their unique interest.
Online voting imposes new challenges on recommendation, because the propagation
of votings heavily depends on the structure of social networks as well as the
content of votings. In this paper, we investigate how to utilize these two
factors in a comprehensive manner when doing voting recommendation. First, due
to the fact that existing text mining methods such as topic model and semantic
model cannot well process the content of votings that is typically short and
ambiguous, we propose a novel Topic-Enhanced Word Embedding (TEWE) method to
learn word and document representation by jointly considering their topics and
semantics. Then we propose our Joint Topic-Semantic-aware social Matrix
Factorization (JTS-MF) model for voting recommendation. JTS-MF model calculates
similarity among users and votings by combining their TEWE representation and
structural information of social networks, and preserves this
topic-semantic-social similarity during matrix factorization. To evaluate the
performance of TEWE representation and JTS-MF model, we conduct extensive
experiments on real online voting dataset. The results prove the efficacy of
our approach against several state-of-the-art baselines.Comment: The 26th ACM International Conference on Information and Knowledge
Management (CIKM 2017
- …