324 research outputs found
Generalized Sliced Wasserstein Distances
The Wasserstein distance and its variations, e.g., the sliced-Wasserstein
(SW) distance, have recently drawn attention from the machine learning
community. The SW distance, specifically, was shown to have similar properties
to the Wasserstein distance, while being much simpler to compute, and is
therefore used in various applications including generative modeling and
general supervised/unsupervised learning. In this paper, we first clarify the
mathematical connection between the SW distance and the Radon transform. We
then utilize the generalized Radon transform to define a new family of
distances for probability measures, which we call generalized
sliced-Wasserstein (GSW) distances. We also show that, similar to the SW
distance, the GSW distance can be extended to a maximum GSW (max-GSW) distance.
We then provide the conditions under which GSW and max-GSW distances are indeed
distances. Finally, we compare the numerical performance of the proposed
distances on several generative modeling tasks, including SW flows and SW
auto-encoders
Amortized Projection Optimization for Sliced Wasserstein Generative Models
Seeking informative projecting directions has been an important task in
utilizing sliced Wasserstein distance in applications. However, finding these
directions usually requires an iterative optimization procedure over the space
of projecting directions, which is computationally expensive. Moreover, the
computational issue is even more severe in deep learning applications, where
computing the distance between two mini-batch probability measures is repeated
several times. This nested loop has been one of the main challenges that
prevent the usage of sliced Wasserstein distances based on good projections in
practice. To address this challenge, we propose to utilize the
learning-to-optimize technique or amortized optimization to predict the
informative direction of any given two mini-batch probability measures. To the
best of our knowledge, this is the first work that bridges amortized
optimization and sliced Wasserstein generative models. In particular, we derive
linear amortized models, generalized linear amortized models, and non-linear
amortized models which are corresponding to three types of novel mini-batch
losses, named amortized sliced Wasserstein. We demonstrate the favorable
performance of the proposed sliced losses in deep generative modeling on
standard benchmark datasets.Comment: Accepted to NeurIPS 2022, 22 pages, 6 figures, 8 table
Sliced Wasserstein Kernel for Persistence Diagrams
Persistence diagrams (PDs) play a key role in topological data analysis
(TDA), in which they are routinely used to describe topological properties of
complicated shapes. PDs enjoy strong stability properties and have proven their
utility in various learning contexts. They do not, however, live in a space
naturally endowed with a Hilbert structure and are usually compared with
specific distances, such as the bottleneck distance. To incorporate PDs in a
learning pipeline, several kernels have been proposed for PDs with a strong
emphasis on the stability of the RKHS distance w.r.t. perturbations of the PDs.
In this article, we use the Sliced Wasserstein approximation SW of the
Wasserstein distance to define a new kernel for PDs, which is not only provably
stable but also provably discriminative (depending on the number of points in
the PDs) w.r.t. the Wasserstein distance between PDs. We also demonstrate
its practicality, by developing an approximation technique to reduce kernel
computation time, and show that our proposal compares favorably to existing
kernels for PDs on several benchmarks.Comment: Minor modification
- …