3 research outputs found
Adversarial Fisher Vectors for Unsupervised Representation Learning
We examine Generative Adversarial Networks (GANs) through the lens of deep
Energy Based Models (EBMs), with the goal of exploiting the density model that
follows from this formulation. In contrast to a traditional view where the
discriminator learns a constant function when reaching convergence, here we
show that it can provide useful information for downstream tasks, e.g., feature
extraction for classification. To be concrete, in the EBM formulation, the
discriminator learns an unnormalized density function (i.e., the negative
energy term) that characterizes the data manifold. We propose to evaluate both
the generator and the discriminator by deriving corresponding Fisher Score and
Fisher Information from the EBM. We show that by assuming that the generated
examples form an estimate of the learned density, both the Fisher Information
and the normalized Fisher Vectors are easy to compute. We also show that we are
able to derive a distance metric between examples and between sets of examples.
We conduct experiments showing that the GAN-induced Fisher Vectors demonstrate
competitive performance as unsupervised feature extractors for classification
and perceptual similarity tasks. Code is available at
\url{https://github.com/apple/ml-afv}.Comment: Accepted as spotlight presentation to NeurIPS 201
Set Distribution Networks: a Generative Model for Sets of Images
Images with shared characteristics naturally form sets. For example, in a
face verification benchmark, images of the same identity form sets. For
generative models, the standard way of dealing with sets is to represent each
as a one hot vector, and learn a conditional generative model
. This representation assumes that the number of sets
is limited and known, such that the distribution over sets reduces to a simple
multinomial distribution. In contrast, we study a more generic problem where
the number of sets is large and unknown. We introduce Set Distribution Networks
(SDNs), a novel framework that learns to autoencode and freely generate sets.
We achieve this by jointly learning a set encoder, set discriminator, set
generator, and set prior. We show that SDNs are able to reconstruct image sets
that preserve salient attributes of the inputs in our benchmark datasets, and
are also able to generate novel objects/identities. We examine the sets
generated by SDN with a pre-trained 3D reconstruction network and a face
verification network, respectively, as a novel way to evaluate the quality of
generated sets of images
Fast Adaptation with Linearized Neural Networks
The inductive biases of trained neural networks are difficult to understand
and, consequently, to adapt to new settings. We study the inductive biases of
linearizations of neural networks, which we show to be surprisingly good
summaries of the full network functions. Inspired by this finding, we propose a
technique for embedding these inductive biases into Gaussian processes through
a kernel designed from the Jacobian of the network. In this setting, domain
adaptation takes the form of interpretable posterior inference, with
accompanying uncertainty estimation. This inference is analytic and free of
local optima issues found in standard techniques such as fine-tuning neural
network weights to a new task. We develop significant computational speed-ups
based on matrix multiplies, including a novel implementation for scalable
Fisher vector products. Our experiments on both image classification and
regression demonstrate the promise and convenience of this framework for
transfer learning, compared to neural network fine-tuning. Code is available at
https://github.com/amzn/xfer/tree/master/finite_ntk.Comment: AISTATS 202