Constant payoff in zero-sum stochastic games

In a zero-sum stochastic game, at each stage, two adversary players take decisions and receive a stage payoff determined by them and by a random variable representing the state of nature. The total payoff is the discounted sum of the stage payoffs. Assume that the players are very patient and use optimal strategies. We then prove that, at any point in the game, players get essentially the same expected payoff: the payoff is constant. This solves a conjecture by Sorin, Venel and Vigeral (2010). The proof relies on the semi-algebraic approach for discounted stochastic games introduced by Bewley and Kohlberg (1976), on the theory of Markov chains with rare transitions, initiated by Friedlin and Wentzell (1984), and on some variational inequalities for value functions inspired by the recent work of Davini, Fathi, Iturriaga and Zavidovique (2016

Perturbed Markov Chains

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.Comment: 22 page

On the Robustness of the Twin-Peaked Ergodic Distribution of Income Across Countries

In the literature on convergence, the simple Markov chain model indicates evolution towards a twin-peaked world. Although cleansing the ergodic distribution of income across countries of short-run noise reinforces its twin-peaked shape, these twin peaks are not statistically significant. Moreover, the specific type high immobility reflected by the data on income renders the estimated transition matrix particularly prone to the generation of twin-peaked ergodic distributions.

Testing for Equilibrium Multiplicity in Dynamic Markov Games

This paper proposes several statistical tests for finite state Markov games to examine the null hypothesis that the data are generated from a single equilibrium. We formulate tests of (i) the conditional choice probabilities, (ii) the steady-state distribution of states and (iii) the conditional distribution of states conditional on an initial state. In a Monte Carlo study we find that the chi-squared test of the steady-state distribution performs well and has high power even with a small number of markets and time periods. We apply the chi-squared test to the empirical application of Ryan (2012) that analyzes dynamics of the U.S. Portland Cement industry and test if his assumption of single equilibrium is supported by the data

Optimal Monte Carlo Updating

Based on Peskun's theorem it is shown that optimal transition matrices in Markov chain Monte Carlo should have zero diagonal elements except for the diagonal element corresponding to the largest weight. We will compare the statistical efficiency of this sampler to existing algorithms, such as heat-bath updating and the Metropolis algorithm. We provide numerical results for the Potts model as an application in classical physics. As an application in quantum physics we consider the spin 3/2 XY model and the Bose-Hubbard model which have been simulated by the directed loop algorithm in the stochastic series expansion framework.Comment: 6 pages, 5 figures, replaced with published versio

Analysis of business demography using markov chains : an application to Belgian data

This paper applies the theory of finite Markov chains to analyse the demographic evolution of Belgian enterprises. While other methodologies concentrate on the entry and exit of firms, the Markov approach also analyses migrations between economic sectors. Besides helping to provide a fuller picture of the evolution of the population, Markov chains also enable forecasts of its future composition to be made, as well as the computation of average lifetimes of companies by branch of activity. The method is applied to Belgian data from the Crossroads Bank for Enterprises (CBE). To ensure compliance with Eurostat-OECD definitions, only 'active' enterprises, i.e. enterprises with a positive turnover and/or at least one employee, are considered. The forecasting method is applied to simulate the demographic evolution of the CBE population between 2000 and 2006. This simulation seems to match well the observed changes. Taking migrations into account yields better forecasts than if they are not considered. Moreover, several off-diagonal percentages in the transition matrix are sigificantly different from zero. A case study shows that these migrations are changes in main activity and not the consequence of corrections of wrongly classified firms. Next, the average remaining lifetime and the average age of enterprises in a particular branch of activity is computed and analysed. These lifetimes and ages differ considerably across branches. As expected the life-times of public services are longer than average. Shorter lifetimes combined with an increasing number of enterprises is an indication of renewal inside the branch. A low average age is a sign of relatively new branches. Comparing age to total expected lifetime yields an indicator of closeness to extinction. This might be an indicator of the maturity of the branch. The method is more generally applicable in the sense that it can be used to analyse other populations than those from the CBE and other partitions of the populationBusiness demography, Markov chains, Transition matrix