30 research outputs found
Learning Local Invariant Mahalanobis Distances
Abstract For many tasks and data types, there are natural transformations to which the data should be invariant or insensitive. For instance, in visual recognition, natural images should be insensitive to rotation and translation. This requirement and its implications have been important in many machine learning applications, and tolerance for image transformations was primarily achieved by using robust feature vectors. In this paper we propose a novel and computationally efficient way to learn a local Mahalanobis metric per datum, and show how we can learn a local invariant metric to any transformation in order to improve performance
Learning Discrete Weights and Activations Using the Local Reparameterization Trick
In computer vision and machine learning, a crucial challenge is to lower the
computation and memory demands for neural network inference. A commonplace
solution to address this challenge is through the use of binarization. By
binarizing the network weights and activations, one can significantly reduce
computational complexity by substituting the computationally expensive floating
operations with faster bitwise operations. This leads to a more efficient
neural network inference that can be deployed on low-resource devices. In this
work, we extend previous approaches that trained networks with discrete weights
using the local reparameterization trick to also allow for discrete
activations. The original approach optimized a distribution over the discrete
weights and uses the central limit theorem to approximate the pre-activation
with a continuous Gaussian distribution. Here we show that the probabilistic
modeling can also allow effective training of networks with discrete activation
as well. This further reduces runtime and memory footprint at inference time
with state-of-the-art results for networks with binary activations