Max-Sliced Wasserstein Distance and its use for GANs

Generative adversarial nets (GANs) and variational auto-encoders have significantly improved our distribution modeling capabilities, showing promise for dataset augmentation, image-to-image translation and feature learning. However, to model high-dimensional distributions, sequential training and stacked architectures are common, increasing the number of tunable hyper-parameters as well as the training time. Nonetheless, the sample complexity of the distance metrics remains one of the factors affecting GAN training. We first show that the recently proposed sliced Wasserstein distance has compelling sample complexity properties when compared to the Wasserstein distance. To further improve the sliced Wasserstein distance we then analyze its `projection complexity' and develop the max-sliced Wasserstein distance which enjoys compelling sample complexity while reducing projection complexity, albeit necessitating a max estimation. We finally illustrate that the proposed distance trains GANs on high-dimensional images up to a resolution of 256x256 easily.Comment: Accepted to CVPR 201

Harmonic mappings valued in the Wasserstein space

We propose a definition of the Dirichlet energy (which is roughly speaking the integral of the square of the gradient) for mappings mu : Omega -> (P(D), W\_2) defined over a subset Omega of R^p and valued in the space P(D) of probability measures on a compact convex subset D of R^q endowed with the quadratic Wasserstein distance. Our definition relies on a straightforward generalization of the Benamou-Brenier formula (already introduced by Brenier) but is also equivalent to the definition of Koorevaar, Schoen and Jost as limit of approximate Dirichlet energies, and to the definition of Reshetnyak of Sobolev spaces valued in metric spaces. We study harmonic mappings, i.e. minimizers of the Dirichlet energy provided that the values on the boundary d Omega are fixed. The notion of constant-speed geodesics in the Wasserstein space is recovered by taking for Omega a segment of R. As the Wasserstein space (P(D), W\_2) is positively curved in the sense of Alexandrov we cannot apply the theory of Koorevaar, Schoen and Jost and we use instead arguments based on optimal transport. We manage to get existence of harmonic mappings provided that the boundary values are Lipschitz on d Omega, uniqueness is an open question. If Omega is a segment of R, it is known that a curve valued in the Wasserstein space P(D) can be seen as a superposition of curves valued in D. We show that it is no longer the case in higher dimensions: a generic mapping Omega -> P(D) cannot be represented as the superposition of mappings Omega -> D. We are able to show the validity of a maximum principle: the composition F(mu) of a function F : P(D) -> R convex along generalized geodesics and a harmonic mapping mu : Omega -> P(D) is a subharmonic real-valued function. We also study the special case where we restrict ourselves to a given family of elliptically contoured distributions (a finite-dimensional and geodesically convex submanifold of (P(D), W\_2) which generalizes the case of Gaussian measures) and show that it boils down to harmonic mappings valued in the Riemannian manifold of symmetric matrices endowed with the distance coming from optimal transport

Approximate reasoning for real-time probabilistic processes

We develop a pseudo-metric analogue of bisimulation for generalized semi-Markov processes. The kernel of this pseudo-metric corresponds to bisimulation; thus we have extended bisimulation for continuous-time probabilistic processes to a much broader class of distributions than exponential distributions. This pseudo-metric gives a useful handle on approximate reasoning in the presence of numerical information -- such as probabilities and time -- in the model. We give a fixed point characterization of the pseudo-metric. This makes available coinductive reasoning principles for reasoning about distances. We demonstrate that our approach is insensitive to potentially ad hoc articulations of distance by showing that it is intrinsic to an underlying uniformity. We provide a logical characterization of this uniformity using a real-valued modal logic. We show that several quantitative properties of interest are continuous with respect to the pseudo-metric. Thus, if two processes are metrically close, then observable quantitative properties of interest are indeed close.Comment: Preliminary version appeared in QEST 0