6 research outputs found
Kernel Mean Matching for Content Addressability of GANs
We propose a novel procedure which adds "content-addressability" to any given
unconditional implicit model e.g., a generative adversarial network (GAN). The
procedure allows users to control the generative process by specifying a set
(arbitrary size) of desired examples based on which similar samples are
generated from the model. The proposed approach, based on kernel mean matching,
is applicable to any generative models which transform latent vectors to
samples, and does not require retraining of the model. Experiments on various
high-dimensional image generation problems (CelebA-HQ, LSUN bedroom, bridge,
tower) show that our approach is able to generate images which are consistent
with the input set, while retaining the image quality of the original model. To
our knowledge, this is the first work that attempts to construct, at test time,
a content-addressable generative model from a trained marginal model.Comment: Wittawat Jitkrittum and Patsorn Sangkloy contributed equally to this
wor
Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images
State-of-the-art (SOTA) Generative Models (GMs) can synthesize
photo-realistic images that are hard for humans to distinguish from genuine
photos. We propose to perform reverse engineering of GMs to infer the model
hyperparameters from the images generated by these models. We define a novel
problem, "model parsing", as estimating GM network architectures and training
loss functions by examining their generated images -- a task seemingly
impossible for human beings. To tackle this problem, we propose a framework
with two components: a Fingerprint Estimation Network (FEN), which estimates a
GM fingerprint from a generated image by training with four constraints to
encourage the fingerprint to have desired properties, and a Parsing Network
(PN), which predicts network architecture and loss functions from the estimated
fingerprints. To evaluate our approach, we collect a fake image dataset with
K images generated by GMs. Extensive experiments show encouraging
results in parsing the hyperparameters of the unseen models. Finally, our
fingerprint estimation can be leveraged for deepfake detection and image
attribution, as we show by reporting SOTA results on both the recent Celeb-DF
and image attribution benchmarks