27 research outputs found
Recommended from our members
Predictive Complexity Priors
Specifying a Bayesian prior is notoriously difficult for complex models such as neural networks. Reasoning about parameters is made challenging by the high-dimensionality and over-parameterization of the space. Priors that seem benign and uninformative can have unintuitive and detrimental effects on a model's predictions. For this reason, we propose predictive complexity priors: a functional prior that is defined by comparing the model's predictions to those of a reference model. Although originally defined on the model outputs, we transfer the prior to the model parameters via a change of variables. The traditional Bayesian workflow can then proceed as usual. We apply our predictive complexity prior to high-dimensional regression, reasoning over neural network depth, and sharing of statistical strength for few-shot learning
Spectral Diffusion Processes
Score-based generative modelling (SGM) has proven to be a very effective
method for modelling densities on finite-dimensional spaces. In this work we
propose to extend this methodology to learn generative models over functional
spaces. To do so, we represent functional data in spectral space to dissociate
the stochastic part of the processes from their space-time part. Using
dimensionality reduction techniques we then sample from their stochastic
component using finite dimensional SGM. We demonstrate our method's
effectiveness for modelling various multimodal datasets.Comment: 17 pages, 11 figures, Score-based Method Workshop at 36th Conference
on Neural Information Processing Systems (NeurIPS 2022
What Matters for Meta-Learning Vision Regression Tasks?
Meta-learning is widely used in few-shot classification and function
regression due to its ability to quickly adapt to unseen tasks. However, it has
not yet been well explored on regression tasks with high dimensional inputs
such as images. This paper makes two main contributions that help understand
this barely explored area. \emph{First}, we design two new types of
cross-category level vision regression tasks, namely object discovery and pose
estimation of unprecedented complexity in the meta-learning domain for computer
vision. To this end, we (i) exhaustively evaluate common meta-learning
techniques on these tasks, and (ii) quantitatively analyze the effect of
various deep learning techniques commonly used in recent meta-learning
algorithms in order to strengthen the generalization capability: data
augmentation, domain randomization, task augmentation and meta-regularization.
Finally, we (iii) provide some insights and practical recommendations for
training meta-learning algorithms on vision regression tasks. \emph{Second}, we
propose the addition of functional contrastive learning (FCL) over the task
representations in Conditional Neural Processes (CNPs) and train in an
end-to-end fashion. The experimental results show that the results of prior
work are misleading as a consequence of a poor choice of the loss function as
well as too small meta-training sets. Specifically, we find that CNPs
outperform MAML on most tasks without fine-tuning. Furthermore, we observe that
naive task augmentation without a tailored design results in underfitting.Comment: Accepted at CVPR 202