505 research outputs found
Manifold-valued Image Generation with Wasserstein Generative Adversarial Nets
Generative modeling over natural images is one of the most fundamental
machine learning problems. However, few modern generative models, including
Wasserstein Generative Adversarial Nets (WGANs), are studied on manifold-valued
images that are frequently encountered in real-world applications. To fill the
gap, this paper first formulates the problem of generating manifold-valued
images and exploits three typical instances: hue-saturation-value (HSV) color
image generation, chromaticity-brightness (CB) color image generation, and
diffusion-tensor (DT) image generation. For the proposed generative modeling
problem, we then introduce a theorem of optimal transport to derive a new
Wasserstein distance of data distributions on complete manifolds, enabling us
to achieve a tractable objective under the WGAN framework. In addition, we
recommend three benchmark datasets that are CIFAR-10 HSV/CB color images,
ImageNet HSV/CB color images, UCL DT image datasets. On the three datasets, we
experimentally demonstrate the proposed manifold-aware WGAN model can generate
more plausible manifold-valued images than its competitors.Comment: Accepted by AAAI 201
Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets
In this work, we propose a novel approach for generating videos of the six
basic facial expressions given a neutral face image. We propose to exploit the
face geometry by modeling the facial landmarks motion as curves encoded as
points on a hypersphere. By proposing a conditional version of manifold-valued
Wasserstein generative adversarial network (GAN) for motion generation on the
hypersphere, we learn the distribution of facial expression dynamics of
different classes, from which we synthesize new facial expression motions. The
resulting motions can be transformed to sequences of landmarks and then to
images sequences by editing the texture information using another conditional
Generative Adversarial Network. To the best of our knowledge, this is the first
work that explores manifold-valued representations with GAN to address the
problem of dynamic facial expression generation. We evaluate our proposed
approach both quantitatively and qualitatively on two public datasets;
Oulu-CASIA and MUG Facial Expression. Our experimental results demonstrate the
effectiveness of our approach in generating realistic videos with continuous
motion, realistic appearance and identity preservation. We also show the
efficiency of our framework for dynamic facial expressions generation, dynamic
facial expression transfer and data augmentation for training improved emotion
recognition models
Sliced Wasserstein Generative Models
In generative modeling, the Wasserstein distance (WD) has emerged as a useful
metric to measure the discrepancy between generated and real data
distributions. Unfortunately, it is challenging to approximate the WD of
high-dimensional distributions. In contrast, the sliced Wasserstein distance
(SWD) factorizes high-dimensional distributions into their multiple
one-dimensional marginal distributions and is thus easier to approximate. In
this paper, we introduce novel approximations of the primal and dual SWD.
Instead of using a large number of random projections, as it is done by
conventional SWD approximation methods, we propose to approximate SWDs with a
small number of parameterized orthogonal projections in an end-to-end deep
learning fashion. As concrete applications of our SWD approximations, we design
two types of differentiable SWD blocks to equip modern generative
frameworks---Auto-Encoders (AE) and Generative Adversarial Networks (GAN). In
the experiments, we not only show the superiority of the proposed generative
models on standard image synthesis benchmarks, but also demonstrate the
state-of-the-art performance on challenging high resolution image and video
generation in an unsupervised manner.Comment: This paper is accepted by CVPR 2019, accidentally uploaded as a new
submission (arXiv:1904.05408, which has been withdrawn). The code is
available at this https URL https:// github.com/musikisomorphie/swd.gi
Manifold-Aware CycleGAN for High-Resolution Structural-to-DTI Synthesis
Unpaired image-to-image translation has been applied successfully to natural
images but has received very little attention for manifold-valued data such as
in diffusion tensor imaging (DTI). The non-Euclidean nature of DTI prevents
current generative adversarial networks (GANs) from generating plausible images
and has mainly limited their application to diffusion MRI scalar maps, such as
fractional anisotropy (FA) or mean diffusivity (MD). Even if these scalar maps
are clinically useful, they mostly ignore fiber orientations and therefore have
limited applications for analyzing brain fibers. Here, we propose a
manifold-aware CycleGAN that learns the generation of high-resolution DTI from
unpaired T1w images. We formulate the objective as a Wasserstein distance
minimization problem of data distributions on a Riemannian manifold of
symmetric positive definite 3x3 matrices SPD(3), using adversarial and
cycle-consistency losses. To ensure that the generated diffusion tensors lie on
the SPD(3) manifold, we exploit the theoretical properties of the exponential
and logarithm maps of the Log-Euclidean metric. We demonstrate that, unlike
standard GANs, our method is able to generate realistic high-resolution DTI
that can be used to compute diffusion-based metrics and potentially run fiber
tractography algorithms. To evaluate our model's performance, we compute the
cosine similarity between the generated tensors principal orientation and their
ground-truth orientation, the mean squared error (MSE) of their derived FA
values and the Log-Euclidean distance between the tensors. We demonstrate that
our method produces 2.5 times better FA MSE than a standard CycleGAN and up to
30% better cosine similarity than a manifold-aware Wasserstein GAN while
synthesizing sharp high-resolution DTI.Comment: Accepted at MICCAI 2020 International Workshop on Computational
Diffusion MR
Graph-Regularized Manifold-Aware Conditional Wasserstein GAN for Brain Functional Connectivity Generation
Common measures of brain functional connectivity (FC) including covariance
and correlation matrices are semi-positive definite (SPD) matrices residing on
a cone-shape Riemannian manifold. Despite its remarkable success for
Euclidean-valued data generation, use of standard generative adversarial
networks (GANs) to generate manifold-valued FC data neglects its inherent SPD
structure and hence the inter-relatedness of edges in real FC. We propose a
novel graph-regularized manifold-aware conditional Wasserstein GAN (GR-SPD-GAN)
for FC data generation on the SPD manifold that can preserve the global FC
structure. Specifically, we optimize a generalized Wasserstein distance between
the real and generated SPD data under an adversarial training, conditioned on
the class labels. The resulting generator can synthesize new SPD-valued FC
matrices associated with different classes of brain networks, e.g., brain
disorder or healthy control. Furthermore, we introduce additional population
graph-based regularization terms on both the SPD manifold and its tangent space
to encourage the generator to respect the inter-subject similarity of FC
patterns in the real data. This also helps in avoiding mode collapse and
produces more stable GAN training. Evaluated on resting-state functional
magnetic resonance imaging (fMRI) data of major depressive disorder (MDD),
qualitative and quantitative results show that the proposed GR-SPD-GAN clearly
outperforms several state-of-the-art GANs in generating more realistic
fMRI-based FC samples. When applied to FC data augmentation for MDD
identification, classification models trained on augmented data generated by
our approach achieved the largest margin of improvement in classification
accuracy among the competing GANs over baselines without data augmentation.Comment: 10 pages, 4 figure
PHom-GeM: Persistent Homology for Generative Models
Generative neural network models, including Generative Adversarial Network
(GAN) and Auto-Encoders (AE), are among the most popular neural network models
to generate adversarial data. The GAN model is composed of a generator that
produces synthetic data and of a discriminator that discriminates between the
generator's output and the true data. AE consist of an encoder which maps the
model distribution to a latent manifold and of a decoder which maps the latent
manifold to a reconstructed distribution. However, generative models are known
to provoke chaotically scattered reconstructed distribution during their
training, and consequently, incomplete generated adversarial distributions.
Current distance measures fail to address this problem because they are not
able to acknowledge the shape of the data manifold, i.e. its topological
features, and the scale at which the manifold should be analyzed. We propose
Persistent Homology for Generative Models, PHom-GeM, a new methodology to
assess and measure the distribution of a generative model. PHom-GeM minimizes
an objective function between the true and the reconstructed distributions and
uses persistent homology, the study of the topological features of a space at
different spatial resolutions, to compare the nature of the true and the
generated distributions. Our experiments underline the potential of persistent
homology for Wasserstein GAN in comparison to Wasserstein AE and Variational
AE. The experiments are conducted on a real-world data set particularly
challenging for traditional distance measures and generative neural network
models. PHom-GeM is the first methodology to propose a topological distance
measure, the bottleneck distance, for generative models used to compare
adversarial samples in the context of credit card transactions
- …