Nonzero-sum Stochastic Games

This paper treats of stochastic games. We focus on nonzero-sum games and provide a detailed survey of selected recent results. In Section 1, we consider stochastic Markov games. A correlation of strategies of the players, involving ``public signals'', is described, and a correlated equilibrium theorem proved recently by Nowak and Raghavan for discounted stochastic games with general state space is presented. We also report an extension of this result to a class of undiscounted stochastic games, satisfying some uniform ergodicity condition. Stopping games are related to stochastic Markov games. In Section 2, we describe a version of Dynkin's game related to observation of a Markov process with random assignment mechanism of states to the players. Some recent contributions of the second author in this area are reported. The paper also contains a brief overview of the theory of nonzero-sum stochastic games and stopping games which is very far from being complete.average payoff stochastic games, correlated stationary equilibria, nonzero-sum games, stopping time, stopping games

Continuous Level Monte Carlo and Sample-Adaptive Model Hierarchies

This is the final version. Available from SIAM via the DOI in this record.In this paper, we present a generalisation of the Multilevel Monte Carlo (MLMC) method to a setting where the level parameter is a continuous variable. This Continuous Level Monte Carlo (CLMC) estimator provides a natural framework in PDE applications to adapt the model hierarchy to each sample. In addition, it can be made unbiased with respect to the expected value of the true quantity of interest provided the quantity of interest converges sufficiently fast. The practical implementation of the CLMC estimator is based on interpolating actual evaluations of the quantity of interest at a finite number of resolutions. As our new level parameter, we use the logarithm of a goal-oriented finite element error estimator for the accuracy of the quantity of interest. We prove the unbiasedness, as well as a complexity theorem that shows the same rate of complexity for CLMC as for MLMC. Finally, we provide some numerical evidence to support our theoretical results, by successfully testing CLMC on a standard PDE test problem. The numerical experiments demonstrate clear gains for sample-wise adaptive refinement strategies over uniform refinements

Sequential correlated equilibrium in stopping games

In many situations, such as trade in stock exchanges, agents have many instances to act even though the duration of interactions take a relatively short time. The agents in such situations can often coordinate their actions in advance, but coordination during the game consumes too much time. An equilibrium in such situations has to be sequential in order to handle mistakes made by players. In this paper, we present a new solution concept for infinite-horizon dynamic games, which is appropriate for such situations: a sequential constant-expectation normal-form correlated approximate equilibrium. Under additional assumptions, we show that every such game admits this kind of equilibrium

Perfect correlated equilibria in stopping games

In many situations, such as trade in stock exchanges, agents have many instances to act even though the duration of interactions take a relatively short time. The agents in such situations can often coordinate their actions in advance, but coordination during the game consumes too much time. An equilibrium in such situations has to be sequential in order to handle mistakes made by players. In this paper, we present a new solution concept for infinite-horizon dynamic games, which is appropriate for such situations: a sequential constant-expectation normal-form correlated approximate equilibrium. Under additional assumptions, we show that every such game admits this kind of equilibrium

Sequential correlated equilibrium in stopping games

In many situations, such as trade in stock exchanges, agents have many instances to act even though the duration of interactions take a relatively short time. The agents in such situations can often coordinate their actions in advance, but coordination during the game consumes too much time. An equilibrium in such situations has to be sequential in order to handle mistakes made by players. In this paper, we present a new solution concept for infinite-horizon dynamic games, which is appropriate for such situations: a sequential constant-expectation normal-form correlated approximate equilibrium. Under additional assumptions, we show that every such game admits this kind of equilibrium

Optimal Dynamic Control of a Useful Class of Randomly Jumping Processes

The purpose of the paper is to present a complete theory of optimal control of piecewise linear and piecewise monotone processes. The theory consists of a description of the processes, necessary and sufficient optimality conditions and existence and uniqueness results, as well as extremal and regularity properties of the optimal strategy. Mathematical proofs are only outlined (they will appear elsewhere), but hints concerning efficient determination of the optimal strategy are included. Piecewise linear (monotone) processes are discontinuous Markov processes whose state components stay constant or change linearly (monotonically) between two consecutive jumps. All processes of inventory, storage, queuing, reliability and risk theory belong to these classes. The processes will be controlled by feedback (Markov) strategies based on complete state observations. The expected value of a performance functional of integral type with additional terminal costs is to be minimized. The semigroup theory of Markov processes will be used as the uniform mathematical tool for the whole theory, and the control problem will be reduced to the integration of a system of ordinary differential equations. Special emphasis will be given to the description of the processes by their infinitesimal characteristics which are available explicitly in applied models-no finite dimensional distributions are used

On some Stochastic Control Problems arising in Environmental Economics and Commodity Markets

Koch T. On some Stochastic Control Problems arising in Environmental Economics and Commodity Markets. Bielefeld: Universität Bielefeld; 2020

Stability of attitude control systems acted upon by random perturbations

Mathematical models on stability of attitude control systems acted upon by random perturbation processe

Ensembles poissoniens de boucles markoviennes

In this thesis I study an infinite measure on loops naturally associated to a wide range of Markovian processes and the Poisson point processes of intensity proportional to this measure (intensity parameter alpha>0). This Poissson point processes are called Poisson ensembles of Markov loops or loop soups. The measure on loops is covariant with some transformation on Markovian processes, for instance the change of time. In the setting of Brownian loop soups inside a proper open simply connected domain of C it was shown that the outer boundaries of outermost clusters of loops are, for alpha1/2, Conformal Loop Ensembles CLE(kappa), kappa in (8/3,4]. Besides, it was shown for a wide range of symmetric Markovian processes that for alpha=1/2 the occupation field of a loop soup (the sum of times spent by loops over points) is the square of the Gaussian free field. First I studied the loop soups associated to one-dimensional diffusions, and particularly the occupation field and its zeroes that delimit in this case the clusters of loops. Then I studied the loop soups on discrete graphs and metric graphs (edges replaced by continuous lines). On a metric graph on one hand the loops have a non-trivial geometry and on the other hand one has the same property as in the setting of one-dimensional diffusions that the zeroes of the occupation field delimit the clusters of loops. By combing metric graphs and the isomorphism with the Gaussian free field I have shown that alpha=1/2 is the critical parameter for random walk loop soup percolation on the discrete half-plane Z*N (existence or not of an infinite cluster of loops) and that for alpha0). Ces processus ponctuels de Poisson portent le nom d'ensembles poissoniens de boucles markoviennes ou de soupes de boucles. La mesure sur les boucles est covariante par un certain nombre de transformations sur les processus de Markov, par exemple le changement de temps.Dans le cadre de soupe de boucles brownienne à l'intérieur d'un sous-domaine ouvert propre simplement connexe de C, il a été montré que les contours extérieurs des amas extérieurs de boucles sont, pour alpha<=1/2, des Conformal Loop Ensembles CLE(kappa), kappa dans (8/3,4]. D'autre part il a été montré pour une large classe de processus de Markov symétriques que lorsque alpha=1/2, le champ d'occupation d'une soupe de boucle (somme des temps passés par les boucles aux dessus des points) est le carré du champ libre gaussien. J'ai étudié d'abord les soupes de boucles associés aux processus de diffusion unidimensionnels, notamment leur champ d'occupation dont les zéros délimitent dans ce cas les amas de boucles. Puis j'ai étudié les soupes de boucles sur graphe discret ainsi que sur graphe métrique (arêtes remplacés par des fils continus). Sur graphe métrique on a d'une part une géométrie non triviale pour les boucles et d'autre part on a comme dans le cas unidimensionnel continu la propriété que les zéros du champ d'occupation délimitent les amas des boucles. En combinant les graphes métriques et l'isomorphisme avec le champ libre gaussien j'ai montré que alpha=1/2 est le paramètre d'intensité critique pour la percolation par soupe de boucles de marche aléatoire sur le demi plan discret Z*N (existence ou non d'un amas infini) et que pour alpha<=1/2 la limite d'échelle des contours extérieurs des amas extérieurs sur Z*N est un CLE(kappa) dans le demi-plan continu

Collective excitations in low dimensional systems and stochastic control of population growth in a fluctuating environment

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Chemistry, 2007.Vita.Includes bibliographical references.In this thesis, I study several problems in the following areas: collective excitations in condensed matter physics, noise in gene network and stochastic control in biophysics. In the first area, I construct an effective field theory to describe Bose-Einstein Condensate (BEC) realized in an external potential. This theory explicitly explores the idea of spontaneous symmetry breaking and its application in the description of phase transitions of confined systems. Based on the effective lagrangian, I calculate the excitation spectrum and Matsubara Green's functions using the method of functional integrals. The theory also shows that in one dimension the collective excitation of a bosonic system can be unified with that of a fermionic system, which is described by Luttinger liquid theory. The unified theory of collective excitations of low dimensional quantum systems motivates my study of collective excitations of interacting classical particles confined in one dimension. It is shown in my paper that the structure of Hamiltonian or Lagrangian for one dimensional constrained systems is uniquely determined by conservation laws. Therefore the excitations of bosonic, fermionic and classical particles are strikingly similar in one dimension.(cont.) In the second area, i. e., noise in gene networks and phenotypic switching in a fluctuating environment, I study the noise propagation in a gene network cascade using the method of master equations which examines the validity of the more popular methods such as the Langevin equation. To further explore the applications of stochastic processes for complex systems, I study phenotypic switches in a fluctuating environment. By combining the techniques of stochastic differential equation and stochastic dynamical programming, I propose a simple framework which can be used to study phenotypic growth dynamics. Another work is to explore the influence of environment on the dynamical properties of small systems is directed to the unusual blinking statistics of semiconductor quantum dots. I show in a model system that a broad spectrum of decay rates is possible when disorder is present in the environment.by Xiang Xia.Ph.D