4,580 research outputs found
Fast inference in nonlinear dynamical systems using gradient matching
Parameter inference in mechanistic models of
coupled differential equations is a topical problem.
We propose a new method based on kernel
ridge regression and gradient matching, and
an objective function that simultaneously encourages
goodness of fit and penalises inconsistencies
with the differential equations. Fast minimisation
is achieved by exploiting partial convexity
inherent in this function, and setting up an iterative
algorithm in the vein of the EM algorithm.
An evaluation of the proposed method on various
benchmark data suggests that it compares
favourably with state-of-the-art alternatives
Herding as a Learning System with Edge-of-Chaos Dynamics
Herding defines a deterministic dynamical system at the edge of chaos. It
generates a sequence of model states and parameters by alternating parameter
perturbations with state maximizations, where the sequence of states can be
interpreted as "samples" from an associated MRF model. Herding differs from
maximum likelihood estimation in that the sequence of parameters does not
converge to a fixed point and differs from an MCMC posterior sampling approach
in that the sequence of states is generated deterministically. Herding may be
interpreted as a"perturb and map" method where the parameter perturbations are
generated using a deterministic nonlinear dynamical system rather than randomly
from a Gumbel distribution. This chapter studies the distinct statistical
characteristics of the herding algorithm and shows that the fast convergence
rate of the controlled moments may be attributed to edge of chaos dynamics. The
herding algorithm can also be generalized to models with latent variables and
to a discriminative learning setting. The perceptron cycling theorem ensures
that the fast moment matching property is preserved in the more general
framework
Inference in Nonlinear Systems with Unscented Kalman Filters
An increasing number of scientific disciplines, most notably the life sciences and
health care, have become more quantitative, describing complex systems with coupled nonlinear
diāµerential equations. While powerful algorithms for numerical simulations from these systems
have been developed, statistical inference of the system parameters is still a challenging problem.
A promising approach is based on the unscented Kalman filter (UKF), which has seen
a variety of recent applications, from soft tissue mechanics to chemical kinetics. The present
study investigates the dependence of the accuracy of parameter estimation on the initialisation.
Based on three toy systems that capture typical features of real-world complex systems: limit
cycles, chaotic attractors and intrinsic stochasticity, we carry out repeated simulations on a large
range of independent data instantiations. Our study allows a quantification of the accuracy of
inference, measured in terms of two alternative distance measures in function and parameter
space, in dependence on the initial deviation from the ground truth
Multiphase MCMC sampling for parameter inference in nonlinear ordinary differential equations
Traditionally, ODE parameter inference relies on solving the system of ODEs and assessing fit of the estimated signal with the observations. However, nonlinear ODEs often do not permit closed form solutions. Using numerical methods to solve the equations results in prohibitive computational costs, particularly when one adopts a Bayesian approach in sampling parameters from a posterior distribution. With the introduction of gradient matching, we can abandon the need to numerically solve the system of equations. Inherent in these efficient procedures is an introduction of bias to the learning problem as we no longer sample based on the exact likelihood function. This paper presents a multiphase MCMC approach that attempts to close the gap between efficiency and accuracy. By sampling using a surrogate likelihood, we accelerate convergence to the stationary distribution before sampling using the exact likelihood. We demonstrate that this method combines the efficiency of gradient matching and the accuracy of the exact likelihood scheme
Connections Between Adaptive Control and Optimization in Machine Learning
This paper demonstrates many immediate connections between adaptive control
and optimization methods commonly employed in machine learning. Starting from
common output error formulations, similarities in update law modifications are
examined. Concepts in stability, performance, and learning, common to both
fields are then discussed. Building on the similarities in update laws and
common concepts, new intersections and opportunities for improved algorithm
analysis are provided. In particular, a specific problem related to higher
order learning is solved through insights obtained from these intersections.Comment: 18 page
Inference in Complex Systems Using Multi-Phase MCMC Sampling With Gradient Matching Burn-in
We propose a novel method for parameter inference that builds on the current research in gradient matching surrogate likelihood spaces. Adopting a three phase technique, we demonstrate that it is possible to obtain parameter estimates of limited bias whilst still adopting the paradigm of the computationally cheap surrogate approximation
Statistical inference in mechanistic models: time warping for improved gradient matching
Inference in mechanistic models of non-linear differential equations is a challenging problem in current computational statistics. Due to the high computational costs of numerically solving the differential equations in every step of an iterative parameter adaptation scheme, approximate methods based on gradient matching have become popular. However, these methods critically depend on the smoothing scheme for function interpolation. The present article adapts an idea from manifold learning and demonstrates that a time warping approach aiming to homogenize intrinsic length scales can lead to a significant improvement in parameter estimation accuracy. We demonstrate the effectiveness of this scheme on noisy data from two dynamical systems with periodic limit cycle, a biopathway, and an application from soft-tissue mechanics. Our study also provides a comparative evaluation on a wide range of signal-to-noise ratios
- ā¦