822 research outputs found
Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting
Diffusion models have achieved state-of-the-art performance in generative
modeling tasks across various domains. Prior works on time series diffusion
models have primarily focused on developing conditional models tailored to
specific forecasting or imputation tasks. In this work, we explore the
potential of task-agnostic, unconditional diffusion models for several time
series applications. We propose TSDiff, an unconditionally trained diffusion
model for time series. Our proposed self-guidance mechanism enables
conditioning TSDiff for downstream tasks during inference, without requiring
auxiliary networks or altering the training procedure. We demonstrate the
effectiveness of our method on three different time series tasks: forecasting,
refinement, and synthetic data generation. First, we show that TSDiff is
competitive with several task-specific conditional forecasting methods
(predict). Second, we leverage the learned implicit probability density of
TSDiff to iteratively refine the predictions of base forecasters with reduced
computational overhead over reverse diffusion (refine). Notably, the generative
performance of the model remains intact -- downstream forecasters trained on
synthetic samples from TSDiff outperform forecasters that are trained on
samples from other state-of-the-art generative time series models, occasionally
even outperforming models trained on real data (synthesize)
AdaCat: Adaptive Categorical Discretization for Autoregressive Models
Autoregressive generative models can estimate complex continuous data
distributions, like trajectory rollouts in an RL environment, image
intensities, and audio. Most state-of-the-art models discretize continuous data
into several bins and use categorical distributions over the bins to
approximate the continuous data distribution. The advantage is that the
categorical distribution can easily express multiple modes and are
straightforward to optimize. However, such approximation cannot express sharp
changes in density without using significantly more bins, making it parameter
inefficient. We propose an efficient, expressive, multimodal parameterization
called Adaptive Categorical Discretization (AdaCat). AdaCat discretizes each
dimension of an autoregressive model adaptively, which allows the model to
allocate density to fine intervals of interest, improving parameter efficiency.
AdaCat generalizes both categoricals and quantile-based regression. AdaCat is a
simple add-on to any discretization-based distribution estimator. In
experiments, AdaCat improves density estimation for real-world tabular data,
images, audio, and trajectories, and improves planning in model-based offline
RL.Comment: Uncertainty in Artificial Intelligence (UAI) 2022 13 pages, 4 figure
- …