5 research outputs found
Add and Thin: Diffusion for Temporal Point Processes
Autoregressive neural networks within the temporal point process (TPP)
framework have become the standard for modeling continuous-time event data.
Even though these models can expressively capture event sequences in a
one-step-ahead fashion, they are inherently limited for long-term forecasting
applications due to the accumulation of errors caused by their sequential
nature. To overcome these limitations, we derive ADD-THIN, a principled
probabilistic denoising diffusion model for TPPs that operates on entire event
sequences. Unlike existing diffusion approaches, ADD-THIN naturally handles
data with discrete and continuous components. In experiments on synthetic and
real-world datasets, our model matches the state-of-the-art TPP models in
density estimation and strongly outperforms them in forecasting
Generative Diffusion for 3D Turbulent Flows
Turbulent flows are well known to be chaotic and hard to predict; however,
their dynamics differ between two and three dimensions. While 2D turbulence
tends to form large, coherent structures, in three dimensions vortices cascade
to smaller and smaller scales. This cascade creates many fast-changing,
small-scale structures and amplifies the unpredictability, making
regression-based methods infeasible. We propose the first generative model for
forced turbulence in arbitrary 3D geometries and introduce a sample quality
metric for turbulent flows based on the Wasserstein distance of the generated
velocity-vorticity distribution. In several experiments, we show that our
generative diffusion model circumvents the unpredictability of turbulent flows
and produces high-quality samples based solely on geometric information.
Furthermore, we demonstrate that our model beats an industrial-grade numerical
solver in the time to generate a turbulent flow field from scratch by an order
of magnitude
Assessing Robustness via Score-Based Adversarial Image Generation
Most adversarial attacks and defenses focus on perturbations within small
-norm constraints. However, threat models cannot capture all
relevant semantic-preserving perturbations, and hence, the scope of robustness
evaluations is limited. In this work, we introduce Score-Based Adversarial
Generation (ScoreAG), a novel framework that leverages the advancements in
score-based generative models to generate adversarial examples beyond
-norm constraints, so-called unrestricted adversarial examples,
overcoming their limitations. Unlike traditional methods, ScoreAG maintains the
core semantics of images while generating realistic adversarial examples,
either by transforming existing images or synthesizing new ones entirely from
scratch. We further exploit the generative capability of ScoreAG to purify
images, empirically enhancing the robustness of classifiers. Our extensive
empirical evaluation demonstrates that ScoreAG matches the performance of
state-of-the-art attacks and defenses across multiple benchmarks. This work
highlights the importance of investigating adversarial examples bounded by
semantics rather than -norm constraints. ScoreAG represents an
important step towards more encompassing robustness assessments
FLGR: Fixed Length Gists Representation Learning for RNN-HMM Hybrid-Based Neuromorphic Continuous Gesture Recognition
A neuromorphic vision sensors is a novel passive sensing modality and frameless sensors with several advantages over conventional cameras. Frame-based cameras have an average frame-rate of 30 fps, causing motion blur when capturing fast motion, e.g., hand gesture. Rather than wastefully sending entire images at a fixed frame rate, neuromorphic vision sensors only transmit the local pixel-level changes induced by the movement in a scene when they occur. This leads to advantageous characteristics, including low energy consumption, high dynamic range, a sparse event stream and low response latency. In this study, a novel representation learning method was proposed: Fixed Length Gists Representation (FLGR) learning for event-based gesture recognition. Previous methods accumulate events into video frames in a time duration (e.g., 30 ms) to make the accumulated image-level representation. However, the accumulated-frame-based representation waives the friendly event-driven paradigm of neuromorphic vision sensor. New representation are urgently needed to fill the gap in non-accumulated-frame-based representation and exploit the further capabilities of neuromorphic vision. The proposed FLGR is a sequence learned from mixture density autoencoder and preserves the nature of event-based data better. FLGR has a data format of fixed length, and it is easy to feed to sequence classifier. Moreover, an RNN-HMM hybrid was proposed to address the continuous gesture recognition problem. Recurrent neural network (RNN) was applied for FLGR sequence classification while hidden Markov model (HMM) is employed for localizing the candidate gesture and improving the result in a continuous sequence. A neuromorphic continuous hand gestures dataset (Neuro ConGD Dataset) was developed with 17 hand gestures classes for the community of the neuromorphic research. Hopefully, FLGR can inspire the study on the event-based highly efficient, high-speed, and high-dynamic-range sequence classification tasks