74 research outputs found
Discretisation of Stochastic Control Problems for Continuous Time Dynamics with Delay
As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretizing time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established.Markov, Markov chain, time dynamics, stochastic control problem
Discretisation of stochastic control problems for continuous time dynamics with delay
As a main step in the numerical solution of control problems in continuous
time, the controlled process is approximated by sequences of controlled Markov
chains, thus discretising time and space. A new feature in this context is to
allow for delay in the dynamics. The existence of an optimal strategy with
respect to the cost functional can be guaranteed in the class of relaxed
controls. Weak convergence of the approximating extended Markov chains to the
original process together with convergence of the associated optimal strategies
is established.Comment: submitted to JCA
On the Continuity of Stochastic Exit Time Control Problems
We determine a weaker sufficient condition than that of Theorem 5.2.1 in
Fleming and Soner (2006) for the continuity of the value functions of
stochastic exit time control problems.Comment: The proof of Lemma 3.1 is slightly modified, and Remark 4.1 is
rephrased for better presentations. In addition, some typos are corrected
Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees
Asynchronous distributed algorithms are a popular way to reduce
synchronization costs in large-scale optimization, and in particular for neural
network training. However, for nonsmooth and nonconvex objectives, few
convergence guarantees exist beyond cases where closed-form proximal operator
solutions are available. As most popular contemporary deep neural networks lead
to nonsmooth and nonconvex objectives, there is now a pressing need for such
convergence guarantees. In this paper, we analyze for the first time the
convergence of stochastic asynchronous optimization for this general class of
objectives. In particular, we focus on stochastic subgradient methods allowing
for block variable partitioning, where the shared-memory-based model is
asynchronously updated by concurrent processes. To this end, we first introduce
a probabilistic model which captures key features of real asynchronous
scheduling between concurrent processes; under this model, we establish
convergence with probability one to an invariant set for stochastic subgradient
methods with momentum.
From the practical perspective, one issue with the family of methods we
consider is that it is not efficiently supported by machine learning
frameworks, as they mostly focus on distributed data-parallel strategies. To
address this, we propose a new implementation strategy for shared-memory based
training of deep neural networks, whereby concurrent parameter servers are
utilized to train a partitioned but shared model in single- and multi-GPU
settings. Based on this implementation, we achieve on average 1.2x speed-up in
comparison to state-of-the-art training methods for popular image
classification tasks without compromising accuracy
Multigrid methods for two-player zero-sum stochastic games
We present a fast numerical algorithm for large scale zero-sum stochastic
games with perfect information, which combines policy iteration and algebraic
multigrid methods. This algorithm can be applied either to a true finite state
space zero-sum two player game or to the discretization of an Isaacs equation.
We present numerical tests on discretizations of Isaacs equations or
variational inequalities. We also present a full multi-level policy iteration,
similar to FMG, which allows to improve substantially the computation time for
solving some variational inequalities.Comment: 31 page
An Irregular Grid Approach for Pricing High-Dimensional American Options
We propose and test a new method for pricing American options in a high-dimensional setting.The method is centred around the approximation of the associated complementarity problem on an irregular grid.We approximate the partial differential operator on this grid by appealing to the SDE representation of the underlying process and computing the root of the transition probability matrix of an approximating Markov chain.Experimental results in five dimensions are presented for four different payoff functions.option pricing;inequality;markov chains
Nonlinear Stochastic Systems And Controls: Lotka-Volterra Type Models, Permanence And Extinction, Optimal Harvesting Strategies, And Numerical Methods For Systems Under Partial Observations
This dissertation focuses on a class of stochastic models formulated using stochastic differential equations with regime switching represented by a continuous-time Markov chain, which also known as hybrid switching diffusion processes. Our motivations for studying such processes in this dissertation stem from emerging and existing applications in biological systems, ecosystems, financial engineering, modeling, analysis, and control and optimization of stochastic systems under the influence of random environments, with complete observations or partial observations.
The first part is concerned with Lotka-Volterra models with white noise and regime switching represented by a continuous-time Markov chain. Different from the existing literature, the Markov chain is hidden and canonly be observed in a Gaussian white noise in our work. We use a Wonham filter to estimate the Markov chain from the observable evolution of the given process, and convert the original system to a completely observable one. We then establish the regularity, positivity, stochastic boundedness, and sample path
continuity of the solution. Moreover, stochastic permanence and extinction using feedback controls are investigated.
The second part develops optimal harvest strategies for Lotka-Volterra systems so as to establish economically, ecologically, and environmentally reasonable strategies for populations subject to the risk of extinction. The underlying systems are controlled regime-switching diffusions that belong to the class of singular control problems. We construct upper bounds for the value functions, prove the finiteness of the harvesting value, and derive properties of the value functions. Then we construct explicit chattering harvesting strategies and the corresponding lower bounds for the value functions by using the idea of harvesting only one species at a time. We further show that this is a reasonable candidate for the best lower bound that one can expect.
In the last part, we study optimal harvesting problems for a general systems in the case that the Markov chain is hidden and can only be observed in a Gaussian white noise. The Wonham filter is employed to convert the original problem to a completely observable one. Then we treat the resulting optimal control problem. Because the problem is virtually impossible to solve in closed form, our main effort is devoted to developing numerical approximation algorithms. To approximate the value function and optimal strategies, Markov chain approximation methods are used to construct a discrete-time controlled Markov chain. Convergence of the algorithm is proved by weak convergence method and suitable scaling
Estimation of the parameters of a stochastic logistic growth model
We consider a stochastic logistic growth model involving both birth and death
rates in the drift and diffusion coefficients for which extinction eventually
occurs almost surely. The associated complete Fokker-Planck equation describing
the law of the process is established and studied. We then use its solution to
build a likelihood function for the unknown model parameters, when discretely
sampled data is available. The existing estimation methods need adaptation in
order to deal with the extinction problem. We propose such adaptations, based
on the particular form of the Fokker-Planck equation, and we evaluate their
performances with numerical simulations. In the same time, we explore the
identifiability of the parameters which is a crucial problem for the
corresponding deterministic (noise free) model
- …