Recently, diffusion probabilistic models (DPMs) have achieved promising
results in diverse generative tasks. A typical DPM framework includes a forward
process that gradually diffuses the data distribution and a reverse process
that recovers the data distribution from time-dependent data scores. In this
work, we observe that the stochastic reverse process of data scores is a
martingale, from which concentration bounds and the optional stopping theorem
for data scores can be derived. Then, we discover a simple way for calibrating
an arbitrary pretrained DPM, with which the score matching loss can be reduced
and the lower bounds of model likelihood can consequently be increased. We
provide general calibration guidelines under various model parametrizations.
Our calibration method is performed only once and the resulting models can be
used repeatedly for sampling. We conduct experiments on multiple datasets to
empirically validate our proposal. Our code is at
https://github.com/thudzj/Calibrated-DPMs.Comment: NeurIPS 202