Hybrid PDE solver for data-driven problems and modern branching

The numerical solution of large-scale PDEs, such as those occurring in data-driven applications, unavoidably require powerful parallel computers and tailored parallel algorithms to make the best possible use of them. In fact, considerations about the parallelization and scalability of realistic problems are often critical enough to warrant acknowledgement in the modelling phase. The purpose of this paper is to spread awareness of the Probabilistic Domain Decomposition (PDD) method, a fresh approach to the parallelization of PDEs with excellent scalability properties. The idea exploits the stochastic representation of the PDE and its approximation via Monte Carlo in combination with deterministic high-performance PDE solvers. We describe the ingredients of PDD and its applicability in the scope of data science. In particular, we highlight recent advances in stochastic representations for nonlinear PDEs using branching diffusions, which have significantly broadened the scope of PDD. We envision this work as a dictionary giving large-scale PDE practitioners references on the very latest algorithms and techniques of a non-standard, yet highly parallelizable, methodology at the interface of deterministic and probabilistic numerical methods. We close this work with an invitation to the fully nonlinear case and open research questions.Comment: 23 pages, 7 figures; Final SMUR version; To appear in the European Journal of Applied Mathematics (EJAM

Approximate Bayesian computation (ABC) gives exact results under the assumption of model error

Approximate Bayesian computation (ABC) or likelihood-free inference algorithms are used to find approximations to posterior distributions without making explicit use of the likelihood function, depending instead on simulation of sample data sets from the model. In this paper we show that under the assumption of the existence of a uniform additive model error term, ABC algorithms give exact results when sufficient summaries are used. This interpretation allows the approximation made in many previous application papers to be understood, and should guide the choice of metric and tolerance in future work. ABC algorithms can be generalized by replacing the 0-1 cut-off with an acceptance probability that varies with the distance of the simulated data from the observed data. The acceptance density gives the distribution of the error term, enabling the uniform error usually used to be replaced by a general distribution. This generalization can also be applied to approximate Markov chain Monte Carlo algorithms. In light of this work, ABC algorithms can be seen as calibration techniques for implicit stochastic models, inferring parameter values in light of the computer model, data, prior beliefs about the parameter values, and any measurement or model errors.Comment: 33 pages, 1 figure, to appear in Statistical Applications in Genetics and Molecular Biology 201

Stochastic Approximation Algorithms With Applications To Particle Swarm Optimization, Adaptive Optimization, And Consensus

In this dissertation, we present three problems arising in recent applications of stochastic approximation methods. In Chapter 2, we use stochastic approximation to analyze Particle Swarm Optimization (PSO) algorithm. We introduce four coefficients and rewrite the PSO procedure as a stochastic approximation type iterative algorithm. Then we analyze its convergence using weak convergence method. It is proved that a suitably scaled sequence of swarms converge to the solution of an ordinary differential equation. We also establish certain stability results. Moreover, convergence rates are ascertained by using weak convergence method. A centered and scaled sequence of the estimation errors is shown to have a diffusion limit. In Chapter 3, we study a class of stochastic approximation algorithms with regime switching that is modulated by a discrete Markov chain having countable state spaces and two-time-scale structures. In the algorithm, the increments of a sequence of occupation measures are updated using constant step size. It is demonstrated that least squares estimations from the tracking errors can be developed. Under the assumption that the adaptation rates are of the same order of magnitude as that of times-different parameter, it is proven that the continuous-time interpolation from the iterates converges weakly to some system of ordinary differential equations (ODEs) with regime switching, and that a suitably scaled sequence of the tracking errors converges to a system of switching diffusion. This work is an extension of the work in [80]. In Chapter 4, we developed asynchronous stochastic approximation (SA) algorithms for networked systems with multi-agents and regime-switching topologies to achieve consensus control. There are several distinct features of the algorithms. (1) In contrast to the most existing consensus algorithms, the participating agents compute and communicate in an asynchronous fashion without using a global clock. (2) The agents compute and communicate at random times. (3) The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. (4) The functions involved are allowed to vary with respect to time hence nonstationarity can be handled. (5) Multi-scale formulation enriches the applicability of the algorithms. In the setup, the switching process contains a rate parameter \e \u3e 0 in the transition probability matrix that characterizes how frequently the topology switches. The algorithm uses a step-size $\mu$ that defines how fast the network states are updated. Depending on their relative values, three distinct scenarios emerge. Under suitable conditions, it is shown that a continuous-time interpolation of the iterates converges weakly to a system of randomly switching ordinary differential equations modulated by a continuous-time Markov chain, or to a system of differential equations (an average with respect to certain measure). In addition, a scaled sequence of tracking errors converges to a witching diffusion or a diffusion. Simulation results are presented to demonstrate these findings