42 research outputs found
SPI-GAN: Distilling Score-based Generative Models with Straight-Path Interpolations
Score-based generative models (SGMs) are a recently proposed paradigm for
deep generative tasks and now show the state-of-the-art sampling performance.
It is known that the original SGM design solves the two problems of the
generative trilemma: i) sampling quality, and ii) sampling diversity. However,
the last problem of the trilemma was not solved, i.e., their training/sampling
complexity is notoriously high. To this end, distilling SGMs into simpler
models, e.g., generative adversarial networks (GANs), is gathering much
attention currently. We present an enhanced distillation method, called
straight-path interpolation GAN (SPI-GAN), which can be compared to the
state-of-the-art shortcut-based distillation method, called denoising diffusion
GAN (DD-GAN). However, our method corresponds to an extreme method that does
not use any intermediate shortcut information of the reverse SDE path, in which
case DD-GAN fails to obtain good results. Nevertheless, our straight-path
interpolation method greatly stabilizes the overall training process. As a
result, SPI-GAN is one of the best models in terms of the sampling
quality/diversity/time for CIFAR-10, CelebA-HQ-256, and LSUN-Church-256
STaSy: Score-based Tabular data Synthesis
Tabular data synthesis is a long-standing research topic in machine learning.
Many different methods have been proposed over the past decades, ranging from
statistical methods to deep generative methods. However, it has not always been
successful due to the complicated nature of real-world tabular data. In this
paper, we present a new model named Score-based Tabular data Synthesis (STaSy)
and its training strategy based on the paradigm of score-based generative
modeling. Despite the fact that score-based generative models have resolved
many issues in generative models, there still exists room for improvement in
tabular data synthesis. Our proposed training strategy includes a self-paced
learning technique and a fine-tuning strategy, which further increases the
sampling quality and diversity by stabilizing the denoising score matching
training. Furthermore, we also conduct rigorous experimental studies in terms
of the generative task trilemma: sampling quality, diversity, and time. In our
experiments with 15 benchmark tabular datasets and 7 baselines, our method
outperforms existing methods in terms of task-dependant evaluations and
diversity. Code is available at https://github.com/JayoungKim408/STaSy.Comment: 27 pages, Accepted by ICLR 2023 for spotlight presentation, Official
code: https://github.com/JayoungKim408/STaS
Regular Time-series Generation using SGM
Score-based generative models (SGMs) are generative models that are in the
spotlight these days. Time-series frequently occurs in our daily life, e.g.,
stock data, climate data, and so on. Especially, time-series forecasting and
classification are popular research topics in the field of machine learning.
SGMs are also known for outperforming other generative models. As a result, we
apply SGMs to synthesize time-series data by learning conditional score
functions. We propose a conditional score network for the time-series
generation domain. Furthermore, we also derive the loss function between the
score matching and the denoising score matching in the time-series generation
domain. Finally, we achieve state-of-the-art results on real-world datasets in
terms of sampling diversity and quality.Comment: 9 pages, appendix 3 pages, under revie
Precursor-of-Anomaly Detection for Irregular Time Series
Anomaly detection is an important field that aims to identify unexpected
patterns or data points, and it is closely related to many real-world problems,
particularly to applications in finance, manufacturing, cyber security, and so
on. While anomaly detection has been studied extensively in various fields,
detecting future anomalies before they occur remains an unexplored territory.
In this paper, we present a novel type of anomaly detection, called
\emph{\textbf{P}recursor-of-\textbf{A}nomaly} (PoA) detection. Unlike
conventional anomaly detection, which focuses on determining whether a given
time series observation is an anomaly or not, PoA detection aims to detect
future anomalies before they happen. To solve both problems at the same time,
we present a neural controlled differential equation-based neural network and
its multi-task learning algorithm. We conduct experiments using 17 baselines
and 3 datasets, including regular and irregular time series, and demonstrate
that our presented method outperforms the baselines in almost all cases. Our
ablation studies also indicate that the multitasking training method
significantly enhances the overall performance for both anomaly and PoA
detection.Comment: KDD 2023 accepted pape