Lyapunov stabilizability of controlled diffusions via a superoptimality principle for viscosity solutions

We prove optimality principles for semicontinuous bounded viscosity solutions of Hamilton-Jacobi-Bellman equations. In particular we provide a representation formula for viscosity supersolutions as value functions of suitable obstacle control problems. This result is applied to extend the Lyapunov direct method for stability to controlled Ito stochastic differential equations. We define the appropriate concept of Lyapunov function to study the stochastic open loop stabilizability in probability and the local and global asymptotic stabilizability (or asymptotic controllability). Finally we illustrate the theory with some examples.Comment: 22 page

Estimation of Solutions of Differential Systems with Delayed Argument of Neutral Type

Tato disertační práce pojednává o řešení diferenciálních rovnic a systémů diferenciálních rovnic. Hlavní pozornost je věnována asymptotickým vlastnostem rovnic se zpožděním a systémů rovnic se zpožděním. V první kapitole jsou uvedeny fyzikální a technické příklady popsané pomocí diferenciálních rovnic se zpožděním a jejich systémů. Je uvedena klasifikace rovnic se zpožděním a jsou zformulovány základní pojmy stability s důrazem na druhou metodu Ljapunova. Ve druhé kapitole jsou studovány odhady řešení rovnic neutrálního typu. Třetí kapitola se zabývá systémy diferenciálních rovnic neutrálního typu. Jsou odvozeny asymptotické odhady pro řešení i pro derivace řešení. V závěru kapitoly jsou uvedeny příklady a srovnání výsledků s pracemi jiných autorů. Výpočty byly prováděny pomocí programu MATLAB. Poslední, čtvrtá kapitola, se zabývá asymptotickými vlastnostmi systémů se speciálním typem nelinearity, tzv. sektorové nelinearity. Jsou odvozeny vlastnosti řešení a derivace řešení. Základní metodou pro důkazy je v celé práci druhá Ljapunovova metoda a použití funkcionálů Ljapunova-Krasovského.This dissertation discusses the solutions to the differential equation and to systems of differential equations. The main attention is paid to study of asymptotical properties of equations with delay and systems of equations with delay. In the first chapter are given physical and technical examples described by differential equations with delay and their systems. The classification of equations with delay is given and basic notions of theory of stability are formulated (mainly with the emphasis on the Lyapunov second method). In the second chapter estimates of solutions of equations of neutral type are studied. The third chapter deals with systems of differential equations of neutral type. Asymptotic estimates for solutions and their derivatives are proved. At the end of the chapter examples and comparisons of our results and of other authors are given. The calculation were performed with the MATLAB software. Last, the fourth chapter deals with asymptotical properties of systems having a special type of nonlinearities, so called ``sector nonlinearities''. Properties and estimations of solutions and derivatives are derived. The basic tools used in the dissertation are the Lyapunov second method and functionals of Lyapunov-Krasovskii type.

Linearly Solvable Stochastic Control Lyapunov Functions

This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear partial differential equation can then be relaxed to a linear differential inclusion, allowing for relaxed solutions to be generated using sum of squares programming. The resulting relaxed solutions are in fact viscosity super/subsolutions, and by the maximum principle are pointwise upper and lower bounds to the underlying value function, even for coarse polynomial approximations. Furthermore, the pointwise upper bound is shown to be a stochastic control Lyapunov function, yielding a method for generating nonlinear controllers with pointwise bounded distance from the optimal cost when using the optimal controller. These approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems. Finally, this paper develops a-priori bounds on trajectory suboptimality when using these approximate value functions, as well as demonstrates that these methods, and bounds, can be applied to a more general class of nonlinear systems not obeying the constraint on stochastic forcing. Simulated examples illustrate the methodology.Comment: Published in SIAM Journal of Control and Optimizatio

Non-Smooth Stochastic Lyapunov Functions With Weak Extension of Viscosity Solutions

This paper proposes a notion of viscosity weak supersolutions to build a bridge between stochastic Lyapunov stability theory and viscosity solution theory. Different from ordinary differential equations, stochastic differential equations can have the origins being stable despite having no smooth stochastic Lyapunov functions (SLFs). The feature naturally requires that the related Lyapunov equations are illustrated via viscosity solution theory, which deals with non-smooth solutions to partial differential equations. This paper claims that stochastic Lyapunov stability theory needs a weak extension of viscosity supersolutions, and the proposed viscosity weak supersolutions describe non-smooth SLFs ensuring a large class of the origins being noisily (asymptotically) stable and (asymptotically) stable in probability. The contribution of the non-smooth SLFs are confirmed by a few examples; especially, they ensure that all the linear-quadratic-Gaussian (LQG) controlled systems have the origins being noisily asymptotically stable for any additive noises

Local stability of Kolmogorov forward equations for finite state nonlinear Markov processes

The focus of this work is on local stability of a class of nonlinear ordinary differential equations (ODE) that describe limits of empirical measures associated with finite-state weakly interacting N-particle systems. Local Lyapunov functions are identified for several classes of such ODE, including those associated with systems with slow adaptation and Gibbs systems. Using results from [5] and large deviations heuristics, a partial differential equation (PDE) associated with the nonlinear ODE is introduced and it is shown that positive definite subsolutions of this PDE serve as local Lyapunov functions for the ODE. This PDE characterization is used to construct explicit Lyapunov functions for a broad class of models called locally Gibbs systems. This class of models is significantly larger than the family of Gibbs systems and several examples of such systems are presented, including models with nearest neighbor jumps and models with simultaneous jumps that arise in applications.Comment: Updated to include Acknowledgement