166 research outputs found
Small steps and giant leaps: Minimal Newton solvers for Deep Learning
We propose a fast second-order method that can be used as a drop-in
replacement for current deep learning solvers. Compared to stochastic gradient
descent (SGD), it only requires two additional forward-mode automatic
differentiation operations per iteration, which has a computational cost
comparable to two standard forward passes and is easy to implement. Our method
addresses long-standing issues with current second-order solvers, which invert
an approximate Hessian matrix every iteration exactly or by conjugate-gradient
methods, a procedure that is both costly and sensitive to noise. Instead, we
propose to keep a single estimate of the gradient projected by the inverse
Hessian matrix, and update it once per iteration. This estimate has the same
size and is similar to the momentum variable that is commonly used in SGD. No
estimate of the Hessian is maintained. We first validate our method, called
CurveBall, on small problems with known closed-form solutions (noisy Rosenbrock
function and degenerate 2-layer linear networks), where current deep learning
solvers seem to struggle. We then train several large models on CIFAR and
ImageNet, including ResNet and VGG-f networks, where we demonstrate faster
convergence with no hyperparameter tuning. Code is available
End-to-end representation learning for Correlation Filter based tracking
The Correlation Filter is an algorithm that trains a linear template to
discriminate between images and their translations. It is well suited to object
tracking because its formulation in the Fourier domain provides a fast
solution, enabling the detector to be re-trained once per frame. Previous works
that use the Correlation Filter, however, have adopted features that were
either manually designed or trained for a different task. This work is the
first to overcome this limitation by interpreting the Correlation Filter
learner, which has a closed-form solution, as a differentiable layer in a deep
neural network. This enables learning deep features that are tightly coupled to
the Correlation Filter. Experiments illustrate that our method has the
important practical benefit of allowing lightweight architectures to achieve
state-of-the-art performance at high framerates.Comment: To appear at CVPR 201
Learning feed-forward one-shot learners
One-shot learning is usually tackled by using generative models or
discriminative embeddings. Discriminative methods based on deep learning, which
are very effective in other learning scenarios, are ill-suited for one-shot
learning as they need large amounts of training data. In this paper, we propose
a method to learn the parameters of a deep model in one shot. We construct the
learner as a second deep network, called a learnet, which predicts the
parameters of a pupil network from a single exemplar. In this manner we obtain
an efficient feed-forward one-shot learner, trained end-to-end by minimizing a
one-shot classification objective in a learning to learn formulation. In order
to make the construction feasible, we propose a number of factorizations of the
parameters of the pupil network. We demonstrate encouraging results by learning
characters from single exemplars in Omniglot, and by tracking visual objects
from a single initial exemplar in the Visual Object Tracking benchmark.Comment: The first three authors contributed equally, and are listed in
alphabetical orde
Extracting Reward Functions from Diffusion Models
Diffusion models have achieved remarkable results in image generation, and
have similarly been used to learn high-performing policies in sequential
decision-making tasks. Decision-making diffusion models can be trained on
lower-quality data, and then be steered with a reward function to generate
near-optimal trajectories. We consider the problem of extracting a reward
function by comparing a decision-making diffusion model that models low-reward
behavior and one that models high-reward behavior; a setting related to inverse
reinforcement learning. We first define the notion of a relative reward
function of two diffusion models and show conditions under which it exists and
is unique. We then devise a practical learning algorithm for extracting it by
aligning the gradients of a reward function -- parametrized by a neural network
-- to the difference in outputs of both diffusion models. Our method finds
correct reward functions in navigation environments, and we demonstrate that
steering the base model with the learned reward functions results in
significantly increased performance in standard locomotion benchmarks. Finally,
we demonstrate that our approach generalizes beyond sequential decision-making
by learning a reward-like function from two large-scale image generation
diffusion models. The extracted reward function successfully assigns lower
rewards to harmful images
Invariant Information Clustering for Unsupervised Image Classification and Segmentation
We present a novel clustering objective that learns a neural network
classifier from scratch, given only unlabelled data samples. The model
discovers clusters that accurately match semantic classes, achieving
state-of-the-art results in eight unsupervised clustering benchmarks spanning
image classification and segmentation. These include STL10, an unsupervised
variant of ImageNet, and CIFAR10, where we significantly beat the accuracy of
our closest competitors by 6.6 and 9.5 absolute percentage points respectively.
The method is not specialised to computer vision and operates on any paired
dataset samples; in our experiments we use random transforms to obtain a pair
from each image. The trained network directly outputs semantic labels, rather
than high dimensional representations that need external processing to be
usable for semantic clustering. The objective is simply to maximise mutual
information between the class assignments of each pair. It is easy to implement
and rigorously grounded in information theory, meaning we effortlessly avoid
degenerate solutions that other clustering methods are susceptible to. In
addition to the fully unsupervised mode, we also test two semi-supervised
settings. The first achieves 88.8% accuracy on STL10 classification, setting a
new global state-of-the-art over all existing methods (whether supervised,
semi-supervised or unsupervised). The second shows robustness to 90% reductions
in label coverage, of relevance to applications that wish to make use of small
amounts of labels. github.com/xu-ji/IICComment: International Conference on Computer Vision 201
Análise Económica - Financeira das 3 Grandes Sociedades Desportivas (Sporting, Benfica e Porto)
O futebol é um desporto, considerado como um fenómeno a nÃvel mundial, porque molda culturas, atrai multidões e quebra barreiras culturais e sociais. Contudo, nos últimos anos, aumentou a importância a nÃvel económico e, por isso, a Union of European Football Associations (UEFA) foi forçada a implementar mudanças desportivas, em geral, com exigências ao nÃvel do Fair Play Financeiro (FFP), em particular, tendo como foco os aspetos económicos e financeiros.
Face ao referido justifica-se que o projeto aplicado tenha como objetivo a análise económica e financeira das três Sociedades Anónimas Desportivas que detêm em Portugal mais tÃtulos desportivos, isto é: Sporting Clube de Portugal – Futebol, SAD, Sport Lisboa e Benfica – Futebol SAD e o Futebol Clube do Porto – Futebol, SAD Metodologicamente, a primeira parte do projeto aplicado centrou-se numa revisão da literatura sobre o futebol e as Sociedades Anónimas Desportiva (SAD), bem como a análise de leis e normas que regem estas sociedades e, ainda, a prestação de contas das mesmas. Na segunda parte do projeto aplicado desenvolve-se uma análise empÃrica que permitiu realizar uma avaliação individual longitudinal com uma comparação das três SAD, justificadas num perÃodo de estudo de oito épocas desportivas que compreende a época de 2007/2008 a 2014/2015.
Os resultados do projeto aplicado permitiram concluir que cada Sociedade Anónima Desportiva representa e interpreta a sua própria prestação de contas, de uma maneira relevante e face aos resultados desportivos alcançados. Assim, com base na classificação desportiva dos clubes na UEFA, foi elaborado um indicador para comparar os resultados desportivos e económicos e o modo como uns podem influenciar os outros. Paralelamente, foram realizadas análises económicas e financeiras das três sociedades obtendo assim uma perspetiva da posição financeira e da sua performance no perÃodo em estudo de cada uma das SAD, de modo a identificar o equilÃbrio e a sua sustentabilidade
Influence of Drying Treatment on Physical Properties of Pumpkin
The aim of this work was to evaluate the properties of pumpkin (Cucurbita maxima L.) exposed to convective air drying and freeze-drying. The samples were analyzed in terms of physical properties (colour and texture). The trials in the convective chamber were done at 40 ºC and 60 ºC, in the drying tunnel at 60 ºC and in the freeze dryer at -50 ºC. It was concluded that the freeze drying and the air drying at 40 ºC produced smaller changes in the colour while the drying in the tunnel originated more intense colour changes. With respect to texture, it was possible to deduce that the pulp in the fresh product at 2 cm off from the skin is harder than the pulp at 4 cm off from the skin. As to the effect of drying in the texture of the pumpkin, it was observed that all dryings affected texture considerably when compared to the fresh product. In fact, hardness
varied from 75 % in the drying in chamber at 40 ºC to 90 % in the tunnel drying, when compared to the fresh product. As to springiness, it was changed more in the drying at 40 ºC, while cohesiveness showed the higher change in the freeze drying treatment
- …