13 research outputs found
Predictive Uncertainty through Quantization
High-risk domains require reliable confidence estimates from predictive
models. Deep latent variable models provide these, but suffer from the rigid
variational distributions used for tractable inference, which err on the side
of overconfidence. We propose Stochastic Quantized Activation Distributions
(SQUAD), which imposes a flexible yet tractable distribution over discretized
latent variables. The proposed method is scalable, self-normalizing and sample
efficient. We demonstrate that the model fully utilizes the flexible
distribution, learns interesting non-linearities, and provides predictive
uncertainty of competitive quality
Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck
Markov processes are widely used mathematical models for describing dynamic
systems in various fields. However, accurately simulating large-scale systems
at long time scales is computationally expensive due to the short time steps
required for accurate integration. In this paper, we introduce an inference
process that maps complex systems into a simplified representational space and
models large jumps in time. To achieve this, we propose Time-lagged Information
Bottleneck (T-IB), a principled objective rooted in information theory, which
aims to capture relevant temporal features while discarding high-frequency
information to simplify the simulation task and minimize the inference error.
Our experiments demonstrate that T-IB learns information-optimal
representations for accurately modeling the statistical properties and dynamics
of the original process at a selected time lag, outperforming existing
time-lagged dimensionality reduction methods.Comment: 10 pages, 14 figure
Learning Sampling and Model-Based Signal Recovery for Compressed Sensing MRI
Compressed sensing (CS) MRI relies on adequate undersampling of the k-space
to accelerate the acquisition without compromising image quality. Consequently,
the design of optimal sampling patterns for these k-space coefficients has
received significant attention, with many CS MRI methods exploiting
variable-density probability distributions. Realizing that an optimal sampling
pattern may depend on the downstream task (e.g. image reconstruction,
segmentation, or classification), we here propose joint learning of both
task-adaptive k-space sampling and a subsequent model-based proximal-gradient
recovery network. The former is enabled through a probabilistic generative
model that leverages the Gumbel-softmax relaxation to sample across trainable
beliefs while maintaining differentiability. The proposed combination of a
highly flexible sampling model and a model-based (sampling-adaptive) image
reconstruction network facilitates exploration and efficient training, yielding
improved MR image quality compared to other sampling baselines
PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
Time-dependent partial differential equations (PDEs) are ubiquitous in
science and engineering. Recently, mostly due to the high computational cost of
traditional solution techniques, deep neural network based surrogates have
gained increased interest. The practical utility of such neural PDE solvers
relies on their ability to provide accurate, stable predictions over long time
horizons, which is a notoriously hard problem. In this work, we present a
large-scale analysis of common temporal rollout strategies, identifying the
neglect of non-dominant spatial frequency information, often associated with
high frequencies in PDE solutions, as the primary pitfall limiting stable,
accurate rollout performance. Based on these insights, we draw inspiration from
recent advances in diffusion models to introduce PDE-Refiner; a novel model
class that enables more accurate modeling of all frequency components via a
multistep refinement process. We validate PDE-Refiner on challenging benchmarks
of complex fluid dynamics, demonstrating stable and accurate rollouts that
consistently outperform state-of-the-art models, including neural, numerical,
and hybrid neural-numerical architectures. We further demonstrate that
PDE-Refiner greatly enhances data efficiency, since the denoising objective
implicitly induces a novel form of spectral data augmentation. Finally,
PDE-Refiner's connection to diffusion models enables an accurate and efficient
assessment of the model's predictive uncertainty, allowing us to estimate when
the surrogate becomes inaccurate.Comment: Project website: https://phlippe.github.io/PDERefiner
Learning Sub-Sampling and Signal Recovery with Applications in Ultrasound Imaging
Limitations on bandwidth and power consumption impose strict bounds on data
rates of diagnostic imaging systems. Consequently, the design of suitable (i.e.
task- and data-aware) compression and reconstruction techniques has attracted
considerable attention in recent years. Compressed sensing emerged as a popular
framework for sparse signal reconstruction from a small set of compressed
measurements. However, typical compressed sensing designs measure a
(non)linearly weighted combination of all input signal elements, which poses
practical challenges. These designs are also not necessarily task-optimal. In
addition, real-time recovery is hampered by the iterative and time-consuming
nature of sparse recovery algorithms. Recently, deep learning methods have
shown promise for fast recovery from compressed measurements, but the design of
adequate and practical sensing strategies remains a challenge. Here, we propose
a deep learning solution termed Deep Probabilistic Sub-sampling (DPS), that
learns a task-driven sub-sampling pattern, while jointly training a subsequent
task model. Once learned, the task-based sub-sampling patterns are fixed and
straightforwardly implementable, e.g. by non-uniform analog-to-digital
conversion, sparse array design, or slow-time ultrasound pulsing schemes. The
effectiveness of our framework is demonstrated in-silico for sparse signal
recovery from partial Fourier measurements, and in-vivo for both anatomical
image and tissue-motion (Doppler) reconstruction from sub-sampled medical
ultrasound imaging data