Applications of the cross-entropy method to importance sampling and optimal control of diffusions

Abstract. We study the cross-entropy method for diffusions. One of the results is a versatile cross-entropy algorithm that can be used to design efficient importance sampling strategies for rare events or to solve optimal control problems. The approach is based on the minimization of a suitable cross-entropy functional, with a parametric family of exponentially tilted probability distributions. We illustrate the new algorithm with several numerical examples and discuss algorithmic issues and possible extensions of the method. Key words. importance sampling, optimal control, cross-entropy method, rare events, change of measure. AMS subject classifications. 1 Introduction This article deals with the application of the cross-entropy method to diffusion processes, specifically, with the application to importance sampling for rare events and optimal control. Generally, the cross-entropy method is a Monte-Carlo method that was originally developed for the efficient simulation of rare events in queuing models and that has been extended to, e.g., combinatorial optimization or analysis of networks in the meantim

Jarzynski's equality, fluctuation theorems, and variance reduction: Mathematical analysis and numerical algorithms

In this paper, we study Jarzynski's equality and fluctuation theorems for diffusion processes. While some of the results considered in the current work are known in the (mainly physics) literature, we review and generalize these nonequilibrium theorems using mathematical arguments, therefore enabling further investigations in the mathematical community. On the numerical side, variance reduction approaches such as importance sampling method are studied in order to compute free energy differences based on Jarzynski's equality.Comment: journal versio

Model reduction algorithms for optimal control and importance sampling of diffusions

We propose numerical algorithms for solving complex, high- dimensional control and importance sampling problems based on reduced-order models. The algorithms approach the “curse of dimensionality” by a combination of model reduction techniques for multiscale diffusions and stochastic optimization tools, with the aim of reducing the original, possibly high-dimensional problem to a lower dimensional representation of the dynamics, in which only few relevant degrees of freedom are controlled or biased. Specifically, we study situations in which either an suitable set of slow collective variables onto which the dynamics can be projected is known, or situations in which the dynamics shows strongly localized behaviour in the small noise regime. The idea is to use the solution of the reduced-order model as a predictor of the exact solution that, in a corrector step, can be used together with the original dynamics, where no explicit assumptions about small parameters or scale separation have to be made. We illustrate the approach with simple, but paradigmatic numerical examples

Variational characterization of free energy: theory and algorithms

The article surveys and extends variational formulations of the thermodynamic free energy and discusses their information-theoretic content from the perspective of mathematical statistics. We revisit the well-known Jarzynski equality for nonequilibrium free energy sampling within the framework of importance sampling and Girsanov change-of-measure transformations. The implications of the different variational formulations for designing efficient stochastic optimization and nonequilibrium simulation algorithms for computing free energies are discussed and illustrated

Theory and Algorithms

The article surveys and extends variational formulations of the thermodynamic free energy and discusses their information-theoretic content from the perspective of mathematical statistics. We revisit the well-known Jarzynski equality for nonequilibrium free energy sampling within the framework of importance sampling and Girsanov change-of-measure transformations. The implications of the different variational formulations for designing efficient stochastic optimization and nonequilibrium simulation algorithms for computing free energies are discussed and illustrated. View Full-Tex

Transport, Variational Inference and Diffusions: with Applications to Annealed Flows and Schr\"odinger Bridges

This paper explores the connections between optimal transport and variational inference, with a focus on forward and reverse time stochastic differential equations and Girsanov transformations.We present a principled and systematic framework for sampling and generative modelling centred around divergences on path space. Our work culminates in the development of a novel score-based annealed flow technique (with connections to Jarzynski and Crooks identities from statistical physics) and a regularised iterative proportional fitting (IPF)-type objective, departing from the sequential nature of standard IPF. Through a series of generative modelling examples and a double-well-based rare event task, we showcase the potential of the proposed methods.Comment: Workshop on New Frontiers in Learning, Control, and Dynamical Systems at the International Conference on Machine Learning (ICML), Honolulu, Hawaii, USA, 202