Stochastic control liaisons: Richard Sinkhorn meets gaspard monge on a schr\uf6dinger bridge

In 1931-1932, Erwin Schr\uf6dinger studied a hot gas Gedankenexperiment (an instance of large deviations of the empirical distribution). Schr\uf6dinger's problem represents an early example of a fundamental inference method, the so-called maximum entropy method, having roots in Boltzmann's work and being developed in subsequent years by Jaynes, Burg, Dempster, and Csisz\ue1r. The problem, known as the Schr\uf6dinger bridge problem (SBP) with "uniform"prior, was more recently recognized as a regularization of the Monge-Kantorovich optimal mass transport (OMT) problem, leading to effective computational schemes for the latter. Specifically, OMT with quadratic cost may be viewed as a zerotemperature limit of the problem posed by Schr\uf6dinger in the early 1930s. The latter amounts to minimization of Helmholtz's free energy over probability distributions that are constrained to possess two given marginals. The problem features a delicate compromise, mediated by a temperature parameter, between minimizing the internal energy and maximizing the entropy. These concepts are central to a rapidly expanding area of modern science dealing with the so-called Sinkhorn algorithm, which appears as a special case of an algorithm first studied in the more challenging continuous space setting by the French analyst Robert Fortet in 1938-1940 specifically for Schr\uf6dinger bridges. Due to the constraint on end-point distributions, dynamic programming is not a suitable tool to attack these problems. Instead, Fortet's iterative algorithm and its discrete counterpart, the Sinkhorn iteration, permit computation of the optimal solution by iteratively solving the so-called Schr\uf6dinger system. Convergence of the iteration is guaranteed by contraction along the steps in suitable metrics, such as Hilbert's projective metric. In both the continuous as well as the discrete time and space settings, stochastic control provides a reformulation of and a context for the dynamic versions of general Schr\uf6dinger bridge problems and of their zero-temperature limit, the OMT problem. These problems, in turn, naturally lead to steering problems for flows of one-time marginals which represent a new paradigm for controlling uncertainty. The zero-temperature problem in the continuous-time and space setting turns out to be the celebrated Benamou-Brenier characterization of the McCann displacement interpolation flow in OMT. The formalism and techniques behind these control problems on flows of probability distributions have attracted significant attention in recent years as they lead to a variety of new applications in spacecraft guidance, control of robot or biological swarms, sensing, active cooling, and network routing as well as in computer and data science. This multifaceted and versatile framework, intertwining SBP and OMT, provides the substrate for the historical and technical overview of the field given in this paper. A key motivation has been to highlight links between the classical early work in both topics and the more recent stochastic control viewpoint, which naturally lends itself to efficient computational schemes and interesting generalizations

Characterization of Segmented Large Volume, High Purity Germanium Detectors

Gamma ray tracking in future HPGe arrays like AGATA will rely on pulse shape analysis (PSA) of multiple Gamma interactions. For this purpose, a simple and fast procedure was developed which enabled the first full characterization of a segmented large volume HPGe detector. An analytical model for the hole mobility in a Ge crystal lattice was developed to describe the hole drift anisotropy with experimental velocity values along the crystal axis as parameters. The new model is based on the drifted Maxwellian hole distribution in Ge. It is verified by reproducing successfully experimental longitudinal hole anisotropy data. A comparison between electron and hole mobility shows large differences for the longitudinal and tangential velocity anisotropy as a function of the electrical field orientation. Measurements on a 12 fold segmented, n-type, large volume, irregular shaped HPGe detector were performed in order to determine the parameters of anisotropic mobility for electrons and holes as charge carriers created by gamma ray interactions. To characterize the electron mobility the complete outer detector surface was scanned in small steps employing photopeak interactions at 60keV. A precise measurement of the hole drift anisotropy was performed with 356keV rays. The drift velocity anisotropy and crystal geometry cause considerable rise time differences in pulse shapes depending on the position of the spatial charge carrier creation. Pulse shapes of direct and transient signals are reproduced by weighting potential calculations with high precision. The measured angular dependence of rise times is caused by the anisotropic mobility, crystal geometry, changing field strength and space charge effects. Preamplified signals were processed employing digital spectroscopy electronics. Response functions, crosstalk contributions and averaging procedures were taken into account implying novel methods due to the segmentation of the Ge-crystal and the digital electronics. The results are relevant for the future gamma ray tracking detectors where high precision of position information of single gamma ray interactions is required. The high accuracy in simulation enabled very high position resolution using PSA. The first application of this technique in a real experiment aimed at the correction for crystal bending imperfections in a Bragg-spectrometer. A position resolution of 1.4mm (sigma) was achieved with 184keV gamma rays employing the fully characterized detector. By careful characterization of the electronic noise, this result is expected to improve further upon use of optimized filters

Learning Dynamics from Data Using Optimal Transport Techniques and Applications

Optimal Transport has been studied widely in recent years, the concept of Wasserstein distance brings a lot of applications in computational mathematics, machine learning, engineering, even finance areas. Meanwhile, people are gradually realizing that as the amount of data as well as the needs of utilizing data increase vastly, data-driven models have great potentials in real-world applications. In this thesis, we apply the theories of OT and design data-driven algorithms to form and compute various OT problems. We also build a framework to learn inverse OT problem. Furthermore, we develop OT and deep learning based models to solve stochastic differential equations, optimal control, mean field games related problems, all in data-driven settings. In Chapter 2, we provide necessary mathematical concepts and results that form the basis of this thesis. It contains brief surveys of optimal transport, stochastic differential equations, Fokker-Planck equations, deep learning, optimal controls and mean field games. Chapter 3 to Chapter 5 present several scalable algorithms to handle optimal transport problems within different settings. Specifically, Chapter 3 shows a new saddle scheme and learning strategy for computing the Wasserstein geodesic, as well as the Wasserstein distance and OT map between two probability distributions in high dimensions. We parametrize the map and Lagrange multipliers as neural networks. We demonstrate the performance of our algorithms through a series of experiments with both synthetic and realistic data. Chapter 4 presents a scalable algorithm for computing the Monge map between two probability distributions since computing the Monge maps remains challenging, in spite of the rapid developments of the numerical methods for optimal transport problems. Similarly, we formulate the problem as a mini-max problem and solve it via deep learning. The performance of our algorithms is demonstrated through a series of experiments with both synthetic and realistic data. In Chapter 5 we study OT problem in an inverse view, which we also call Inverse OT (IOT) problem. IOT also refers to the problem of learning the cost function for OT from observed transport plan or its samples. We derive an unconstrained convex optimization formulation of the inverse OT problem. We provide a comprehensive characterization of the properties of inverse OT, including uniqueness of solutions. We also develop two numerical algorithms, one is a fast matrix scaling method based on the Sinkhorn-Knopp algorithm for discrete OT, and the other one is a learning based algorithm that parameterizes the cost function as a deep neural network for continuous OT. Our numerical results demonstrate promising efficiency and accuracy advantages of the proposed algorithms over existing state-of-the-art methods. In Chapter 6 we propose a novel method using the weak form of Fokker Planck Equation (FPE) --- a partial differential equation --- to describe the density evolution of data in a sampled form, which is then combined with Wasserstein generative adversarial network (WGAN) in the training process. In such a sample-based framework we are able to learn the nonlinear dynamics from aggregate data without explicitly solving FPE. We demonstrate our approach in the context of a series of synthetic and real-world data sets. Chapter 7 introduces the application of OT and neural networks in optimal density control. Particularly, we parametrize the control strategy via neural networks, and provide an algorithm to learn the strategy that can drive samples following one distribution to new locations following target distribution. We demonstrate our method in both synthetic and realistic experiments, where we also consider perturbation fields. Finally Chapter 8 presents applications of mean field game in generative modeling and finance area. With more details, we build a GAN framework upon mean field game to generate desired distribution starting with white noise, we also investigate its connection to OT. Moreover, we apply mean field game theories to study the equilibrium trading price in stock markets, we demonstrate the theoretical result by conducting experiments on real trading data.Ph.D