322 research outputs found
Challenges in Markov chain Monte Carlo for Bayesian neural networks
Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in
Bayesian neural networks (BNNs). This paper initially reviews the main
challenges in sampling from the parameter posterior of a neural network via
MCMC. Such challenges culminate to lack of convergence to the parameter
posterior. Nevertheless, this paper shows that a non-converged Markov chain,
generated via MCMC sampling from the parameter space of a neural network, can
yield via Bayesian marginalization a valuable predictive posterior of the
output of the neural network. Classification examples based on multilayer
perceptrons showcase highly accurate predictive posteriors. The postulate of
limited scope for MCMC developments in BNNs is partially valid; an
asymptotically exact parameter posterior seems less plausible, yet an accurate
predictive posterior is a tenable research avenue
Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics
Coarse-grained (CG) molecular dynamics enables the study of biological
processes at temporal and spatial scales that would be intractable at an
atomistic resolution. However, accurately learning a CG force field remains a
challenge. In this work, we leverage connections between score-based generative
models, force fields and molecular dynamics to learn a CG force field without
requiring any force inputs during training. Specifically, we train a diffusion
generative model on protein structures from molecular dynamics simulations, and
we show that its score function approximates a force field that can directly be
used to simulate CG molecular dynamics. While having a vastly simplified
training setup compared to previous work, we demonstrate that our approach
leads to improved performance across several small- to medium-sized protein
simulations, reproducing the CG equilibrium distribution, and preserving
dynamics of all-atom simulations such as protein folding events
A survey of uncertainty in deep neural networks
Over the last decade, neural networks have reached almost every field of science and become a crucial part of various real world applications. Due to the increasing spread, confidence in neural network predictions has become more and more important. However, basic neural networks do not deliver certainty estimates or suffer from over- or under-confidence, i.e. are badly calibrated. To overcome this, many researchers have been working on understanding and quantifying uncertainty in a neural network's prediction. As a result, different types and sources of uncertainty have been identified and various approaches to measure and quantify uncertainty in neural networks have been proposed. This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities. It is intended to give anyone interested in uncertainty estimation in neural networks a broad overview and introduction, without presupposing prior knowledge in this field. For that, a comprehensive introduction to the most crucial sources of uncertainty is given and their separation into reducible model uncertainty and irreducible data uncertainty is presented. The modeling of these uncertainties based on deterministic neural networks, Bayesian neural networks (BNNs), ensemble of neural networks, and test-time data augmentation approaches is introduced and different branches of these fields as well as the latest developments are discussed. For a practical application, we discuss different measures of uncertainty, approaches for calibrating neural networks, and give an overview of existing baselines and available implementations. Different examples from the wide spectrum of challenges in the fields of medical image analysis, robotics, and earth observation give an idea of the needs and challenges regarding uncertainties in the practical applications of neural networks. Additionally, the practical limitations of uncertainty quantification methods in neural networks for mission- and safety-critical real world applications are discussed and an outlook on the next steps towards a broader usage of such methods is given
Renormalizing Diffusion Models
We explain how to use diffusion models to learn inverse renormalization group
flows of statistical and quantum field theories. Diffusion models are a class
of machine learning models which have been used to generate samples from
complex distributions, such as the distribution of natural images. These models
achieve sample generation by learning the inverse process to a diffusion
process which adds noise to the data until the distribution of the data is pure
noise. Nonperturbative renormalization group schemes in physics can naturally
be written as diffusion processes in the space of fields. We combine these
observations in a concrete framework for building ML-based models for studying
field theories, in which the models learn the inverse process to an
explicitly-specified renormalization group scheme. We detail how these models
define a class of adaptive bridge (or parallel tempering) samplers for lattice
field theory. Because renormalization group schemes have a physical meaning, we
provide explicit prescriptions for how to compare results derived from models
associated to several different renormalization group schemes of interest. We
also explain how to use diffusion models in a variational method to find ground
states of quantum systems. We apply some of our methods to numerically find RG
flows of interacting statistical field theories. From the perspective of
machine learning, our work provides an interpretation of multiscale diffusion
models, and gives physically-inspired suggestions for diffusion models which
should have novel properties.Comment: 69+15 pages, 8 figures; v2: figure and references added, typos
correcte
Back to Basics: Fast Denoising Iterative Algorithm
We introduce Back to Basics (BTB), a fast iterative algorithm for noise
reduction. Our method is computationally efficient, does not require training
or ground truth data, and can be applied in the presence of independent noise,
as well as correlated (coherent) noise, where the noise level is unknown. We
examine three study cases: natural image denoising in the presence of additive
white Gaussian noise, Poisson-distributed image denoising, and speckle
suppression in optical coherence tomography (OCT). Experimental results
demonstrate that the proposed approach can effectively improve image quality,
in challenging noise settings. Theoretical guarantees are provided for
convergence stability
Phase transitions in lattice gauge theories: From the numerical sign problem to machine learning
Lattice simulations of Quantum chromodynamics (QCD) are an important tool of modern quantum field theory.
They provide high precision results from first principle computations and as such allow for comparison
between experiment and theory. At finite baryon density, such simulations are no longer possible due to the
numerical sign problem which occurs when the action of the theory becomes complex, leading to integrals over highly oscillatory functions.
We investigate two approaches to solve this problem. We employ complex Langevin method, which is a complexified stochastic process and investigate its properties.
We apply it to QCD in a region where other methods are unreliable, we go up to . We finally investigate its applicability for SU(2) real-time simulations.
We also investigate the Lefschetz Thimble method, which solves the sign problem by deforming the manifold of integration, such that there is no more oscillatory behavior.
We discuss aspects of the method in simple models and develop algorithms for higher dimensions.\\
Finally, we apply neural networks to lattice simulation data and use them to extract the order parameter for
the phase transition in the Ising model and SU(2) gauge theory. Thus, we uncover what the neural network learns
- …