563 research outputs found
Auxiliary Learning as an Asymmetric Bargaining Game
Auxiliary learning is an effective method for enhancing the generalization
capabilities of trained models, particularly when dealing with small datasets.
However, this approach may present several difficulties: (i) optimizing
multiple objectives can be more challenging, and (ii) how to balance the
auxiliary tasks to best assist the main task is unclear. In this work, we
propose a novel approach, named AuxiNash, for balancing tasks in auxiliary
learning by formalizing the problem as generalized bargaining game with
asymmetric task bargaining power. Furthermore, we describe an efficient
procedure for learning the bargaining power of tasks based on their
contribution to the performance of the main task and derive theoretical
guarantees for its convergence. Finally, we evaluate AuxiNash on multiple
multi-task benchmarks and find that it consistently outperforms competing
methods.Comment: ICML 202
Improving spiking neural network performance with auxiliary learning
The use of back propagation through the time learning rule enabled the supervised training of deep spiking neural networks to process temporal neuromorphic data. However, their performance is still below non-spiking neural networks. Previous work pointed out that one of the main causes is the limited number of neuromorphic data currently available, which are also difficult to generate. With the goal of overcoming this problem, we explore the usage of auxiliary learning as a means of helping spiking neural networks to identify more general features. Tests are performed on neuromorphic DVS-CIFAR10 and DVS128-Gesture datasets. The results indicate that training with auxiliary learning tasks improves their accuracy, albeit slightly. Different scenarios, including manual and automatic combination losses using implicit differentiation, are explored to analyze the usage of auxiliary tasks
Meta-Auxiliary Learning for Adaptive Human Pose Prediction
Predicting high-fidelity future human poses, from a historically observed
sequence, is decisive for intelligent robots to interact with humans. Deep
end-to-end learning approaches, which typically train a generic pre-trained
model on external datasets and then directly apply it to all test samples,
emerge as the dominant solution to solve this issue. Despite encouraging
progress, they remain non-optimal, as the unique properties (e.g., motion
style, rhythm) of a specific sequence cannot be adapted. More generally, at
test-time, once encountering unseen motion categories (out-of-distribution),
the predicted poses tend to be unreliable. Motivated by this observation, we
propose a novel test-time adaptation framework that leverages two
self-supervised auxiliary tasks to help the primary forecasting network adapt
to the test sequence. In the testing phase, our model can adjust the model
parameters by several gradient updates to improve the generation quality.
However, due to catastrophic forgetting, both auxiliary tasks typically tend to
the low ability to automatically present the desired positive incentives for
the final prediction performance. For this reason, we also propose a
meta-auxiliary learning scheme for better adaptation. In terms of general
setup, our approach obtains higher accuracy, and under two new experimental
designs for out-of-distribution data (unseen subjects and categories), achieves
significant improvements.Comment: 10 pages, 6 figures, AAAI 2023 accepte
Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learning
In deep learning, auxiliary objectives are often used to facilitate learning
in situations where data is scarce, or the principal task is extremely complex.
This idea is primarily inspired by the improved generalization capability
induced by solving multiple tasks simultaneously, which leads to a more robust
shared representation. Nevertheless, finding optimal auxiliary tasks that give
rise to the desired improvement is a crucial problem that often requires
hand-crafted solutions or expensive meta-learning approaches. In this paper, we
propose a novel framework, dubbed Detaux, whereby a weakly supervised
disentanglement procedure is used to discover new unrelated classification
tasks and the associated labels that can be exploited with the principal task
in any Multi-Task Learning (MTL) model. The disentanglement procedure works at
a representation level, isolating a subspace related to the principal task,
plus an arbitrary number of orthogonal subspaces. In the most disentangled
subspaces, through a clustering procedure, we generate the additional
classification tasks, and the associated labels become their representatives.
Subsequently, the original data, the labels associated with the principal task,
and the newly discovered ones can be fed into any MTL framework. Extensive
validation on both synthetic and real data, along with various ablation
studies, demonstrate promising results, revealing the potential in what has
been, so far, an unexplored connection between learning disentangled
representations and MTL. The code will be made publicly available upon
acceptance.Comment: Under review in Pattern Recognition Letter
- …