Inverse Problems and Data Assimilation

These notes are designed with the aim of providing a clear and concise introduction to the subjects of Inverse Problems and Data Assimilation, and their inter-relations, together with citations to some relevant literature in this area. The first half of the notes is dedicated to studying the Bayesian framework for inverse problems. Techniques such as importance sampling and Markov Chain Monte Carlo (MCMC) methods are introduced; these methods have the desirable property that in the limit of an infinite number of samples they reproduce the full posterior distribution. Since it is often computationally intensive to implement these methods, especially in high dimensional problems, approximate techniques such as approximating the posterior by a Dirac or a Gaussian distribution are discussed. The second half of the notes cover data assimilation. This refers to a particular class of inverse problems in which the unknown parameter is the initial condition of a dynamical system, and in the stochastic dynamics case the subsequent states of the system, and the data comprises partial and noisy observations of that (possibly stochastic) dynamical system. We will also demonstrate that methods developed in data assimilation may be employed to study generic inverse problems, by introducing an artificial time to generate a sequence of probability measures interpolating from the prior to the posterior

An optimization principle for deriving nonequilibrium statistical models of Hamiltonian dynamics

A general method for deriving closed reduced models of Hamiltonian dynamical systems is developed using techniques from optimization and statistical estimation. As in standard projection operator methods, a set of resolved variables is selected to capture the slow, macroscopic behavior of the system, and the family of quasi-equilibrium probability densities on phase space corresponding to these resolved variables is employed as a statistical model. The macroscopic dynamics of the mean resolved variables is determined by optimizing over paths of these probability densities. Specifically, a cost function is introduced that quantifies the lack-of-fit of such paths to the underlying microscopic dynamics; it is an ensemble-averaged, squared-norm of the residual that results from submitting a path of trial densities to the Liouville equation. The evolution of the macrostate is estimated by minimizing the time integral of the cost function. The value function for this optimization satisfies the associated Hamilton-Jacobi equation, and it determines the optimal relation between the statistical parameters and the irreversible fluxes of the resolved variables, thereby closing the reduced dynamics. The resulting equations for the macroscopic variables have the generic form of governing equations for nonequilibrium thermodynamics, and they furnish a rational extension of the classical equations of linear irreversible thermodynamics beyond the near-equilibrium regime. In particular, the value function is a thermodynamic potential that extends the classical dissipation function and supplies the nonlinear relation between thermodynamics forces and fluxes

Variational Inequality Approach to Stochastic Nash Equilibrium Problems with an Application to Cournot Oligopoly

In this note we investigate stochastic Nash equilibrium problems by means of monotone variational inequalities in probabilistic Lebesgue spaces. We apply our approach to a class of oligopolistic market equilibrium problems where the data are known through their probability distributions.Comment: 19 pages, 2 table

A primer on noise-induced transitions in applied dynamical systems

Noise plays a fundamental role in a wide variety of physical and biological dynamical systems. It can arise from an external forcing or due to random dynamics internal to the system. It is well established that even weak noise can result in large behavioral changes such as transitions between or escapes from quasi-stable states. These transitions can correspond to critical events such as failures or extinctions that make them essential phenomena to understand and quantify, despite the fact that their occurrence is rare. This article will provide an overview of the theory underlying the dynamics of rare events for stochastic models along with some example applications

Computational Methods for Support Vector Machine Classification and Large-Scale Kalman Filtering

The first half of this dissertation focuses on computational methods for solving the constrained quadratic program (QP) within the support vector machine (SVM) classifier. One of the SVM formulations requires the solution of bound and equality constrained QPs. We begin by describing an augmented Lagrangian approach which incorporates the equality constraint into the objective function, resulting in a bound constrained QP. Furthermore, all constraints may be incorporated into the objective function to yield an unconstrained quadratic program, allowing us to apply the conjugate gradient (CG) method. Lastly, we adapt the scaled gradient projection method of [10] to the SVM QP and compare the performance of these methods with the state-of-the-art sequential minimal optimization algorithm and MATLAB\u27s built in constrained QP solver, quadprog. The augmented Lagrangian method outperforms other state-of-the-art methods on three image test cases. The second half of this dissertation focuses on computational methods for large-scale Kalman filtering applications. The Kalman filter (KF) is a method for solving a dynamic, coupled system of equations. While these methods require only linear algebra, standard KF is often infeasible in large-scale implementations due to the storage requirements and inverse calculations of large, dense covariance matrices. We introduce the use of the CG and Lanczos methods into various forms of the Kalman filter for low-rank approximations of the covariance matrices, with low-storage requirements. We also use CG for efficient Gaussian sampling within the ensemble Kalman filter method. The CG-based KF methods perform similarly in root-mean-square error when compared to the standard KF methods, when the standard implementations are feasible, and outperform the limited-memory Broyden-Fletcher-Goldfarb-Shanno approximation method

Differential Equation Models and Numerical Methods for Reverse Engineering Genetic Regulatory Networks

This dissertation develops and analyzes differential equation-based mathematical models and efficient numerical methods and algorithms for genetic regulatory network identification. The primary objectives of the dissertation are to design, analyze, and test a general variational framework and numerical methods for seeking its approximate solutions for reverse engineering genetic regulatory networks from microarray datasets using the approach based on differential equation modeling. In the proposed variational framework, no structure assumption on the genetic network is presumed, instead, the network is solely determined by the microarray profile of the network components and is identified through a well chosen variational principle which minimizes a biological energy functional. The variational principle serves not only as a selection criterion to pick up the right biological solution of the underlying differential equation model but also provide an effective mathematical characterization of the small-world property of genetic regulatory networks which has been observed in lab experiments. Five specific models within the variational framework and efficient numerical methods and algorithms for computing their solutions are proposed and analyzed in the dissertation. Model validations using both synthetic network datasets and real world subnetwork datasets of Saccharomyces cerevisiae (yeast) and E. Coli are done on all five proposed variational models and a performance comparison vs some existing genetic regulatory network identification methods is also provided. As microarray data is typically noisy, in order to take into account the noise effect in the mathematical models, we propose a new approach based on stochastic differential equation modeling and generalize the deterministic variational framework to a stochastic variational framework which relies on stochastic optimization. Numerical algorithms are also proposed for computing solutions of the stochastic variational models. To address the important issue of post-processing computed networks to reflect the small-world property of underlying genetic regulatory networks, a novel threshholding technique based on the Random Matrix Theory is proposed and tested on various synthetic network datasets

Probabilistic finite elements for fatigue and fracture analysis

An overview of the probabilistic finite element method (PFEM) developed by the authors and their colleagues in recent years is presented. The primary focus is placed on the development of PFEM for both structural mechanics problems and fracture mechanics problems. The perturbation techniques are used as major tools for the analytical derivation. The following topics are covered: (1) representation and discretization of random fields; (2) development of PFEM for the general linear transient problem and nonlinear elasticity using Hu-Washizu variational principle; (3) computational aspects; (4) discussions of the application of PFEM to the reliability analysis of both brittle fracture and fatigue; and (5) a stochastic computational tool based on stochastic boundary element (SBEM). Results are obtained for the reliability index and corresponding probability of failure for: (1) fatigue crack growth; (2) defect geometry; (3) fatigue parameters; and (4) applied loads. These results show that initial defect is a critical parameter

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more