1 research outputs found
Star-Shaped Denoising Diffusion Probabilistic Models
Methods based on Denoising Diffusion Probabilistic Models (DDPM) became a
ubiquitous tool in generative modeling. However, they are mostly limited to
Gaussian and discrete diffusion processes. We propose Star-Shaped Denoising
Diffusion Probabilistic Models (SS-DDPM), a model with a non-Markovian
diffusion-like noising process. In the case of Gaussian distributions, this
model is equivalent to Markovian DDPMs. However, it can be defined and applied
with arbitrary noising distributions, and admits efficient training and
sampling algorithms for a wide range of distributions that lie in the
exponential family. We provide a simple recipe for designing diffusion-like
models with distributions like Beta, von Mises--Fisher, Dirichlet, Wishart and
others, which can be especially useful when data lies on a constrained manifold
such as the unit sphere, the space of positive semi-definite matrices, the
probabilistic simplex, etc. We evaluate the model in different settings and
find it competitive even on image data, where Beta SS-DDPM achieves results
comparable to a Gaussian DDPM