On finite-difference approximations for normalized Bellman equations

A class of stochastic optimal control problems involving optimal stopping is considered. Methods of Krylov are adapted to investigate the numerical solutions of the corresponding normalized Bellman equations and to estimate the rate of convergence of finite difference approximations for the optimal reward functions.Comment: 36 pages, ArXiv version updated to the version accepted in Appl. Math. Opti

Effect of time-correlation of input patterns on the convergence of on-line learning

We studied the effects of time correlation of subsequent patterns on the convergence of on-line learning by a feedforward neural network with backpropagation algorithm. By using chaotic time series as sequences of correlated patterns, we found that the unexpected scaling of converging time with learning parameter emerges when time-correlated patterns accelerate learning process.Comment: 8 pages(Revtex), 5 figure

Sequential design of computer experiments for the estimation of a probability of failure

This paper deals with the problem of estimating the volume of the excursion set of a function $f:\mathbb{R}^d \to \mathbb{R}$ above a given threshold, under a probability measure on $\mathbb{R}^d$ that is assumed to be known. In the industrial world, this corresponds to the problem of estimating a probability of failure of a system. When only an expensive-to-simulate model of the system is available, the budget for simulations is usually severely limited and therefore classical Monte Carlo methods ought to be avoided. One of the main contributions of this article is to derive SUR (stepwise uncertainty reduction) strategies from a Bayesian-theoretic formulation of the problem of estimating a probability of failure. These sequential strategies use a Gaussian process model of $f$ and aim at performing evaluations of $f$ as efficiently as possible to infer the value of the probability of failure. We compare these strategies to other strategies also based on a Gaussian process model for estimating a probability of failure.Comment: This is an author-generated postprint version. The published version is available at http://www.springerlink.co

Homogenized dynamics of stochastic partial differential equations with dynamical boundary conditions

A microscopic heterogeneous system under random influence is considered. The randomness enters the system at physical boundary of small scale obstacles as well as at the interior of the physical medium. This system is modeled by a stochastic partial differential equation defined on a domain perforated with small holes (obstacles or heterogeneities), together with random dynamical boundary conditions on the boundaries of these small holes. A homogenized macroscopic model for this microscopic heterogeneous stochastic system is derived. This homogenized effective model is a new stochastic partial differential equation defined on a unified domain without small holes, with static boundary condition only. In fact, the random dynamical boundary conditions are homogenized out, but the impact of random forces on the small holes' boundaries is quantified as an extra stochastic term in the homogenized stochastic partial differential equation. Moreover, the validity of the homogenized model is justified by showing that the solutions of the microscopic model converge to those of the effective macroscopic model in probability distribution, as the size of small holes diminishes to zero.Comment: Communications in Mathematical Physics, to appear, 200

Stochastic Approximation to Understand Simple Simulation Models

This paper illustrates how a deterministic approximation of a stochastic process can be usefully applied to analyse the dynamics of many simple simulation models. To demonstrate the type of results that can be obtained using this approximation, we present two illustrative examples which are meant to serve as methodological references for researchers exploring this area. Finally, we prove some convergence results for simulations of a family of evolutionary games, namely, intra-population imitation models in n-player games with arbitrary payoffs.Ministerio de Educación (JC2009- 00263), Ministerio de Ciencia e Innovación (CONSOLIDER-INGENIO 2010: CSD2010-00034, DPI2010-16920

Robust filtering for a class of nonlinear stochastic systems with probability constraints

This paper is concerned with the probability-constrained filtering problem for a class of time-varying nonlinear stochastic systems with estimation error variance constraint. The stochastic nonlinearity considered is quite general that is capable of describing several well-studied stochastic nonlinear systems. The second-order statistics of the noise sequence are unknown but belong to certain known convex set. The purpose of this paper is to design a filter guaranteeing a minimized upper-bound on the estimation error variance. The existence condition for the desired filter is established, in terms of the feasibility of a set of difference Riccati-like equations, which can be solved forward in time. Then, under the probability constraints, a minimax estimation problem is proposed for determining the suboptimal filter structure that minimizes the worst-case performance on the estimation error variance with respect to the uncertain second-order statistics. Finally, a numerical example is presented to show the effectiveness and applicability of the proposed method