292,624 research outputs found
Pseudo Labels for Single Positive Multi-Label Learning
The cost of data annotation is a substantial impediment for multi-label image
classification: in every image, every category must be labeled as present or
absent. Single positive multi-label (SPML) learning is a cost-effective
solution, where models are trained on a single positive label per image. Thus,
SPML is a more challenging domain, since it requires dealing with missing
labels. In this work, we propose a method to turn single positive data into
fully-labeled data: Pseudo Multi-Labels. Basically, a teacher network is
trained on single positive labels. Then, we treat the teacher model's
predictions on the training data as ground-truth labels to train a student
network on fully-labeled images. With this simple approach, we show that the
performance achieved by the student model approaches that of a model trained on
the actual fully-labeled images.Comment: ICLR 2023, Tiny Papers Trac
Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification
Object re-identification (re-id) aims to identify a specific object across
times or camera views, with the person re-id and vehicle re-id as the most
widely studied applications. Re-id is challenging because of the variations in
viewpoints, (human) poses, and occlusions. Multi-shots of the same object can
cover diverse viewpoints/poses and thus provide more comprehensive information.
In this paper, we propose exploiting the multi-shots of the same identity to
guide the feature learning of each individual image. Specifically, we design an
Uncertainty-aware Multi-shot Teacher-Student (UMTS) Network. It consists of a
teacher network (T-net) that learns the comprehensive features from multiple
images of the same object, and a student network (S-net) that takes a single
image as input. In particular, we take into account the data dependent
heteroscedastic uncertainty for effectively transferring the knowledge from the
T-net to S-net. To the best of our knowledge, we are the first to make use of
multi-shots of an object in a teacher-student learning manner for effectively
boosting the single image based re-id. We validate the effectiveness of our
approach on the popular vehicle re-id and person re-id datasets. In inference,
the S-net alone significantly outperforms the baselines and achieves the
state-of-the-art performance.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20
Explainable Action Advising for Multi-Agent Reinforcement Learning
Action advising is a knowledge transfer technique for reinforcement learning
based on the teacher-student paradigm. An expert teacher provides advice to a
student during training in order to improve the student's sample efficiency and
policy performance. Such advice is commonly given in the form of state-action
pairs. However, it makes it difficult for the student to reason with and apply
to novel states. We introduce Explainable Action Advising, in which the teacher
provides action advice as well as associated explanations indicating why the
action was chosen. This allows the student to self-reflect on what it has
learned, enabling advice generalization and leading to improved sample
efficiency and learning performance - even in environments where the teacher is
sub-optimal. We empirically show that our framework is effective in both
single-agent and multi-agent scenarios, yielding improved policy returns and
convergence rates when compared to state-of-the-art methodsComment: This work has been accepted to ICRA 202
MEAL: Multi-Model Ensemble via Adversarial Learning
Often the best performing deep neural models are ensembles of multiple
base-level networks. Unfortunately, the space required to store these many
networks, and the time required to execute them at test-time, prohibits their
use in applications where test sets are large (e.g., ImageNet). In this paper,
we present a method for compressing large, complex trained ensembles into a
single network, where knowledge from a variety of trained deep neural networks
(DNNs) is distilled and transferred to a single DNN. In order to distill
diverse knowledge from different trained (teacher) models, we propose to use
adversarial-based learning strategy where we define a block-wise training loss
to guide and optimize the predefined student network to recover the knowledge
in teacher models, and to promote the discriminator network to distinguish
teacher vs. student features simultaneously. The proposed ensemble method
(MEAL) of transferring distilled knowledge with adversarial learning exhibits
three important advantages: (1) the student network that learns the distilled
knowledge with discriminators is optimized better than the original model; (2)
fast inference is realized by a single forward pass, while the performance is
even better than traditional ensembles from multi-original models; (3) the
student network can learn the distilled knowledge from a teacher model that has
arbitrary structures. Extensive experiments on CIFAR-10/100, SVHN and ImageNet
datasets demonstrate the effectiveness of our MEAL method. On ImageNet, our
ResNet-50 based MEAL achieves top-1/5 21.79%/5.99% val error, which outperforms
the original model by 2.06%/1.14%. Code and models are available at:
https://github.com/AaronHeee/MEALComment: To appear in AAAI 2019. Code and models are available at:
https://github.com/AaronHeee/MEA
On the Uncertain Single-View Depths in Colonoscopies
Estimating depth information from endoscopic images is a prerequisite for a
wide set of AI-assisted technologies, such as accurate localization and
measurement of tumors, or identification of non-inspected areas. As the domain
specificity of colonoscopies -- deformable low-texture environments with
fluids, poor lighting conditions and abrupt sensor motions -- pose challenges
to multi-view 3D reconstructions, single-view depth learning stands out as a
promising line of research. Depth learning can be extended in a Bayesian
setting, which enables continual learning, improves decision making and can be
used to compute confidence intervals or quantify uncertainty for in-body
measurements. In this paper, we explore for the first time Bayesian deep
networks for single-view depth estimation in colonoscopies. Our specific
contribution is two-fold: 1) an exhaustive analysis of scalable Bayesian
networks for depth learning in different datasets, highlighting challenges and
conclusions regarding synthetic-to-real domain changes and supervised vs.
self-supervised methods; and 2) a novel teacher-student approach to deep depth
learning that takes into account the teacher uncertainty.Comment: 11 page
- …