Dynamic Programming for controlled Markov families: abstractly and over Martingale Measures

We describe an abstract control-theoretic framework in which the validity of the dynamic programming principle can be established in continuous time by a verification of a small number of structural properties. As an application we treat several cases of interest, most notably the lower-hedging and utility-maximization problems of financial mathematics both of which are naturally posed over ``sets of martingale measures''

Bayesian Filtering for ODEs with Bounded Derivatives

Recently there has been increasing interest in probabilistic solvers for ordinary differential equations (ODEs) that return full probability measures, instead of point estimates, over the solution and can incorporate uncertainty over the ODE at hand, e.g. if the vector field or the initial value is only approximately known or evaluable. The ODE filter proposed in recent work models the solution of the ODE by a Gauss-Markov process which serves as a prior in the sense of Bayesian statistics. While previous work employed a Wiener process prior on the (possibly multiple times) differentiated solution of the ODE and established equivalence of the corresponding solver with classical numerical methods, this paper raises the question whether other priors also yield practically useful solvers. To this end, we discuss a range of possible priors which enable fast filtering and propose a new prior--the Integrated Ornstein Uhlenbeck Process (IOUP)--that complements the existing Integrated Wiener process (IWP) filter by encoding the property that a derivative in time of the solution is bounded in the sense that it tends to drift back to zero. We provide experiments comparing IWP and IOUP filters which support the belief that IWP approximates better divergent ODE's solutions whereas IOUP is a better prior for trajectories with bounded derivatives.Comment: 14 pages, 9 figrue

Probability Measures for Numerical Solutions of Differential Equations

In this paper, we present a formal quantification of epistemic uncertainty induced by numerical solutions of ordinary and partial differential equation models. Numerical solutions of differential equations contain inherent uncertainties due to the finite dimensional approximation of an unknown and implicitly defined function. When statistically analysing models based on differential equations describing physical, or other naturally occurring, phenomena, it is therefore important to explicitly account for the uncertainty introduced by the numerical method. This enables objective determination of its importance relative to other uncertainties, such as those caused by data contaminated with noise or model error induced by missing physical or inadequate descriptors. To this end we show that a wide variety of existing solvers can be randomised, inducing a probability measure over the solutions of such differential equations. These measures exhibit contraction to a Dirac measure around the true unknown solution, where the rates of convergence are consistent with the underlying deterministic numerical method. Ordinary differential equations and elliptic partial differential equations are used to illustrate the approach to quantifying uncertainty in both the statistical analysis of the forward and inverse problems

PHOENICS: A universal deep Bayesian optimizer

In this work we introduce PHOENICS, a probabilistic global optimization algorithm combining ideas from Bayesian optimization with concepts from Bayesian kernel density estimation. We propose an inexpensive acquisition function balancing the explorative and exploitative behavior of the algorithm. This acquisition function enables intuitive sampling strategies for an efficient parallel search of global minima. The performance of PHOENICS is assessed via an exhaustive benchmark study on a set of 15 discrete, quasi-discrete and continuous multidimensional functions. Unlike optimization methods based on Gaussian processes (GP) and random forests (RF), we show that PHOENICS is less sensitive to the nature of the co-domain, and outperforms GP and RF optimizations. We illustrate the performance of PHOENICS on the Oregonator, a difficult case-study describing a complex chemical reaction network. We demonstrate that only PHOENICS was able to reproduce qualitatively and quantitatively the target dynamic behavior of this nonlinear reaction dynamics. We recommend PHOENICS for rapid optimization of scalar, possibly non-convex, black-box unknown objective functions

On numerical density approximations of solutions of SDEs with unbounded coefficients

We study a numerical method to compute probability density functions of solutions of stochastic differential equations. The method is sometimes called the numerical path integration method and has been shown to be fast and accurate in application oriented fields. In this paper we provide a rigorous analysis of the method that covers systems of equations with unbounded coefficients. Working in a natural space for densities, $L^1$, we obtain stability, consistency, and new convergence results for the method, new well-posedness and semigroup generation results for the related Fokker-Planck-Kolmogorov equation, and a new and rigorous connection to the corresponding probability density functions for both the approximate and the exact problems. To prove the results we combine semigroup and PDE arguments in a new way that should be of independent interest

Probability & incompressible Navier-Stokes equations: An overview of some recent developments

This is largely an attempt to provide probabilists some orientation to an important class of non-linear partial differential equations in applied mathematics, the incompressible Navier-Stokes equations. Particular focus is given to the probabilistic framework introduced by LeJan and Sznitman [Probab. Theory Related Fields 109 (1997) 343-366] and extended by Bhattacharya et al. [Trans. Amer. Math. Soc. 355 (2003) 5003-5040; IMA Vol. Math. Appl., vol. 140, 2004, in press]. In particular this is an effort to provide some foundational facts about these equations and an overview of some recent results with an indication of some new directions for probabilistic consideration.Comment: Published at http://dx.doi.org/10.1214/154957805100000078 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

Overdamped limit of generalized stochastic Hamiltonian systems for singular interaction potentials

First weak solutions of generalized stochastic Hamiltonian systems (gsHs) are constructed via essential m-dissipativity of their generators on a suitable core. For a scaled gsHs we prove convergence of the corresponding semigroups and tightness of the weak solutions. This yields convergence in law of the scaled gsHs to a distorted Brownian motion. In particular, the results confirm the convergence of the Langevin dynamics in the overdamped regime to the overdamped Langevin equation. The proofs work for a large class of (singular) interaction potentials including, e.g., potentials of Lennard--Jones type

Quantitative model-checking of controlled discrete-time Markov processes

This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the solution of one of two fundamental problems: reachability and repeated reachability. We provide a comprehensive study of the former problem and an initial characterisation of the (much more involved) latter problem. A case study elucidates concepts and techniques

Nonnegative entire bounded solutions to some semilinear equations involving the fractional Laplacian

The main goal is to establish necessary and sufficient conditions under which the fractional semilinear elliptic equation $\Delta^{\frac{\alpha}{2}} u=\rho(x)\,\varphi(u)$ admits nonnegative nontrivial bounded solutions in the whole space $\mathbb{R}^N$

Multiplicatively interacting point processes and applications to neural modeling

We introduce a nonlinear modification of the classical Hawkes process, which allows inhibitory couplings between units without restrictions. The resulting system of interacting point processes provides a useful mathematical model for recurrent networks of spiking neurons with exponential transfer functions. The expected rates of all neurons in the network are approximated by a first-order differential system. We study the stability of the solutions of this equation, and use the new formalism to implement a winner-takes-all network that operates robustly for a wide range of parameters. Finally, we discuss relations with the generalised linear model that is widely used for the analysis of spike trains.Comment: 22 pages, 7 figures. Submitted to J. Comp. Neurosci. Overall changes according to suggestions of different reviewers. A conceptual error in a derivation has been correcte