Numerical method for impulse control of Piecewise Deterministic Markov Processes

This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying Markov chain defined by the post jump location and inter-arrival time. Convergence results are obtained and more importantly we are able to give a convergence rate of the algorithm. The paper is illustrated by a numerical example.Comment: This work was supported by ARPEGE program of the French National Agency of Research (ANR), project "FAUTOCOES", number ANR-09-SEGI-00

Tail of a linear diffusion with Markov switching

Let Y be an Ornstein-Uhlenbeck diffusion governed by a stationary and ergodic Markov jump process X: dY_t=a(X_t)Y_t dt+\sigma(X_t) dW_t, Y_0=y_0. Ergodicity conditions for Y have been obtained. Here we investigate the tail propriety of the stationary distribution of this model. A characterization of either heavy or light tail case is established. The method is based on a renewal theorem for systems of equations with distributions on R.Comment: Published at http://dx.doi.org/10.1214/105051604000000828 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Approximate Kalman-Bucy filter for continuous-time semi-Markov jump linear systems

The aim of this paper is to propose a new numerical approximation of the Kalman-Bucy filter for semi-Markov jump linear systems. This approximation is based on the selection of typical trajectories of the driving semi-Markov chain of the process by using an optimal quantization technique. The main advantage of this approach is that it makes pre-computations possible. We derive a Lipschitz property for the solution of the Riccati equation and a general result on the convergence of perturbed solutions of semi-Markov switching Riccati equations when the perturbation comes from the driving semi-Markov chain. Based on these results, we prove the convergence of our approximation scheme in a general infinite countable state space framework and derive an error bound in terms of the quantization error and time discretization step. We employ the proposed filter in a magnetic levitation example with markovian failures and compare its performance with both the Kalman-Bucy filter and the Markovian linear minimum mean squares estimator

Optimal stopping for partially observed piecewise-deterministic Markov processes

This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an actual $\epsilon$-optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence

Numerical method for expectations of piecewise-determistic Markov processes

We present a numerical method to compute expectations of functionals of a piecewise-deterministic Markov process. We discuss time dependent functionals as well as deterministic time horizon problems. Our approach is based on the quantization of an underlying discrete-time Markov chain. We obtain bounds for the rate of convergence of the algorithm. The approximation we propose is easily computable and is flexible with respect to some of the parameters defining the problem. Two examples illustrate the paper.Comment: 41 page

Predictive maintenance for the heated hold-up tank

We present a numerical method to compute an optimal maintenance date for the test case of the heated hold-up tank. The system consists of a tank containing a fluid whose level is controlled by three components: two inlet pumps and one outlet valve. A thermal power source heats up the fluid. The failure rates of the components depends on the temperature, the position of the three components monitors the liquid level in the tank and the liquid level determines the temperature. Therefore, this system can be modeled by a hybrid process where the discrete (components) and continuous (level, temperature) parts interact in a closed loop. We model the system by a piecewise deterministic Markov process, propose and implement a numerical method to compute the optimal maintenance date to repair the components before the total failure of the system.Comment: arXiv admin note: text overlap with arXiv:1101.174

Asymmetry tests for Bifurcating Auto-Regressive Processes with missing data

We present symmetry tests for bifurcating autoregressive processes (BAR) when some data are missing. BAR processes typically model cell division data. Each cell can be of one of two types \emph{odd} or \emph{even}. The goal of this paper is to study the possible asymmetry between odd and even cells in a single observed lineage. We first derive asymmetry tests for the lineage itself, modeled by a two-type Galton-Watson process, and then derive tests for the observed BAR process. We present applications on both simulated and real data