Hybrid Stochastic Systems: Numerical Methods, Limit Results, And Controls

This dissertation is concerned with the so-called stochastic hybrid systems, which are featured by the coexistence of continuous dynamics and discrete events and their interactions. Such systems have drawn much needed attentions in recent years. One of the main reasons is that such systems can be used to better reflect the reality for a wide range of applications in networked systems, communication systems, economic systems, cyber-physical systems, and biological and ecological systems, among others. Our main interest is centered around one class of such hybrid systems known as switching diffusions. In such a system, in addition to the driving force of a Brownian motion as in a stochastic system represented by a stochastic differential equation (SDE), there is an additional continuous-time switching process that models the environmental changes due to random events. In the first part, we develops numerical schemes for stochastic differential equations with Markovian switching (Markovian switching SDEs). By utilizing a special form of It^o\u27s formula for switching SDEs and special structural of the jumps of the switching component we derived a new scheme to simulate switching SDEs in the spirit of Milstein\u27s scheme for purely SDEs. We also develop a new approach to establish the convergence of the proposed algorithm that incorporates martingale methods, quadratic variations, and Markovian stopping times. Detailed and delicate analysis is carried out. Under suitable conditions which are natural extensions of the classical ones, the convergence of the algorithms is established. The rate of convergence is also ascertained. The second part is concerned with a limit theorem for general stochastic differential equations with Markovian regime switching. Given a sequence of stochastic regime switching systems where the discrete switching processes are independent of the state of the systems. The continuous-state component of these systems are governed by stochastic differential equations with driving processes that are continuous increasing processes and square integrable martingales. We establish the convergence of the sequence of systems to the one described by a state independent regime-switching diffusion process when the two driving processes converge to the usual time process and the Brownian motion in suitable sense. The third part is concerned with controlled hybrid systems that are good approximations to controlled switching diffusion processes. In lieu of a Brownian motion noise, we use a wide-band noise formulation, which facilitates the treatment of non-Markovian models. The wide-band noise is one whose spectrum has band width wide enough. We work with a basic stationary mixing type process. On top of this wide-band noise process, we allow the system to be subject to random discrete event influence. The discrete event process is a continuous time Markov chain with a finite state space. Although the state space is finite, we assume that the state space is rather large and the Markov chain is irreducible. Using a two-time-scale formulation and assuming the Markov chain also subjects to fast variations, using weak convergence and singular perturbation test function method we first proved that the when controlled by nearly optimal and equilibrium controls, the state and the corresponding costs of the original systems would converge to those of controlled diffusions systems. Using the limit controlled dynamic system as a guidance, we construct controls for the original problem and show that the controls so constructed are near optimal and nearly equilibrium

Optimal financing and dividend distribution in a general diffusion model with regime switching

We study the optimal financing and dividend distribution problem with restricted dividend rates in a diffusion type surplus model where the drift and volatility coefficients are general functions of the level of surplus and the external environment regime. The environment regime is modeled by a Markov process. Both capital injections and dividend payments incur expenses. The objective is to maximize the expectation of the total discounted dividends minus the total cost of capital injections. We prove that it is optimal to inject capitals only when the surplus tends to fall below zero and to pay out dividends at the maximal rate when the surplus is at or above the threshold dependent on the environment regime

Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

Switching Diffusions: Applications To Ecological Models, And Numerical Methods For Games In Insurance

Recently, a class of dynamic systems called ``hybrid systems containing both continuous dynamics and discrete events has been adapted to treat a wide variety of situations arising in many real-world situations. Motivated by such development, this dissertation is devoted to the study of dynamical systems involving a Markov chain as the randomly switching process. The systems studied include hybrid competitive Lotka-Volterra ecosystems and non-zero-sum stochastic differential games between two insurance companies with regime-switching. The first part is concerned with competitive Lotka-Volterra model with Markov switching. A novelty of the contribution is that the Markov chain has a countable state space. Our main objective is to reduce the computational complexity by using the two-time-scale formulation. Because the existence and uniqueness as well as continuity of solutions for Lotka-Volterra ecosystems with Markovian switching in which the switching takes place in a countable set are not available, such properties are studied first. The two-time scale feature is highlighted by introducing a small parameter into the generator of the Markov chain. When the small parameter goes to 0, there is a limit system or reduced system. It is established in this work that if the reduced system possesses certain properties such as permanence and extinction, etc., then the complex original system also has the same properties when the parameter is sufficiently small. These results are obtained by using the perturbed Lyapunov function methods. The second part develops an approximation procedure for a class of non-zero-sum stochastic differential games for investment and reinsurance between two insurance companies. Both proportional reinsurance and excess-of-loss reinsurance policies are considered. We develop numerical algorithms to obtain the approximation to the Nash equilibrium by adopting the Markov chain approximation methodology. We establish the convergence of the approximation sequences and the approximation to the value functions. Numerical examples are presented to illustrate the applicability of the algorithms

Numerical methods for optimal dividend payment and investment strategies of regime-switching jump diffusion models with capital injections

postprin