Stopping Markov processes and first path on graphs

Given a strongly stationary Markov chain ( discrete or continuous) and a finite set of stopping rules, we show a noncombinatorial method to compute the law of stopping. Several examples are presented. The problem of embedding a graph into a larger but minimal graph under some constraints is studied. Given a connected graph, we show a noncombinatorial manner to compute the law of a first given path among a set of stopping paths. We prove the existence of a minimal Markov chain without oversized information

The set-indexed bandit problem

Motivated by spatial problems of allocations, we give a proof of the existence of an optimal solution to a set-indexed formulation of the bandit problem. The proof is based on a compactization of collections of fuzzy stopping sets and fuzzy optional increasing paths, and a construction of set-indexed integrals

Regularity and decomposition of two-parameter supermartingales

The well-known Doob-Meyer decomposition of a supermartingale as the difference of a martingale and an increasing process is extended in several ways for two-parameter stochastic processes. In particular, the notion of laplacian is introduced which gives more explicit decomposition for potentials. The optional sampling theorem is stated for a wide class of supermartingales justifying the study of local martingales. Conditions for regularity and continuity for two-parameter processes are given using approximate laplacians. By introducing the notion of optional increasing path, the relation between the regularity of certain quasimartingales and the continuity of the associated integrable variation process is proved.Stopping points optional increasing paths two-parameter processes supermartingales potential regularity