83 research outputs found
Adaptive Density Estimation for Generative Models
Unsupervised learning of generative models has seen tremendous progress over
recent years, in particular due to generative adversarial networks (GANs),
variational autoencoders, and flow-based models. GANs have dramatically
improved sample quality, but suffer from two drawbacks: (i) they mode-drop,
i.e., do not cover the full support of the train data, and (ii) they do not
allow for likelihood evaluations on held-out data. In contrast,
likelihood-based training encourages models to cover the full support of the
train data, but yields poorer samples. These mutual shortcomings can in
principle be addressed by training generative latent variable models in a
hybrid adversarial-likelihood manner. However, we show that commonly made
parametric assumptions create a conflict between them, making successful hybrid
models non trivial. As a solution, we propose to use deep invertible
transformations in the latent variable decoder. This approach allows for
likelihood computations in image space, is more efficient than fully invertible
models, and can take full advantage of adversarial training. We show that our
model significantly improves over existing hybrid models: offering GAN-like
samples, IS and FID scores that are competitive with fully adversarial models,
and improved likelihood scores
PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification
We propose using an adversarial autoencoder (AAE) to replace generative
adversarial network (GAN) in the private aggregation of teacher ensembles
(PATE), a solution for ensuring differential privacy in speech applications.
The AAE architecture allows us to obtain good synthetic speech leveraging upon
a discriminative training of latent vectors. Such synthetic speech is used to
build a privacy-preserving classifier when non-sensitive data is not
sufficiently available in the public domain. This classifier follows the PATE
scheme that uses an ensemble of noisy outputs to label the synthetic samples
and guarantee -differential privacy (DP) on its derived
classifiers. Our proposed framework thus consists of an AAE-based generator and
a PATE-based classifier (PATE-AAE). Evaluated on the Google Speech Commands
Dataset Version II, the proposed PATE-AAE improves the average classification
accuracy by + and +, respectively, when compared with
alternative privacy-preserving solutions, namely PATE-GAN and DP-GAN, while
maintaining a strong level of privacy target at =0.01 with a fixed
=10.Comment: Accepted to Interspeech 202
- …