Average optimality for continuous-time Markov decision processes under weak continuity conditions

This article considers the average optimality for a continuous-time Markov decision process with Borel state and action spaces and an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is proved under a different and general set of conditions as compared to the previous literature; the controlled process can be explosive, the transition rates can be arbitrarily unbounded and are weakly continuous, the multifunction defining the admissible action spaces can be neither compact-valued nor upper semi-continuous, and the cost rate is not necessarily inf-compact

Growth-optimal portfolios under transaction costs

This paper studies a portfolio optimization problem in a discrete-time Markovian model of a financial market, in which asset price dynamics depend on an external process of economic factors. There are transaction costs with a structure that covers, in particular, the case of fixed plus proportional costs. We prove that there exists a self-financing trading strategy maximizing the average growth rate of the portfolio wealth. We show that this strategy has a Markovian form. Our result is obtained by large deviations estimates on empirical measures of the price process and by a generalization of the vanishing discount method to discontinuous transition operators.Comment: 32 page

Optimal Starting-Stopping Problems for Markov-Feller Processes

By means of nested inequalities in semigroup form we give a characterization of the value functions of the starting-stopping problem for general Markov-Feller processes. Next, we consider two versions of constrained problems on the nal state or on the final time. The plan is as follows: Introduction Nested variational inequalities Solution of optimal starting-stopping problem Problems with constraints References

Average cost optimal control under weak ergodicity hypotheses: Relative value iterations

We study Markov decision processes with Polish state and action spaces. The action space is state dependent and is not necessarily compact. We first establish the existence of an optimal ergodic occupation measure using only a near-monotone hypothesis on the running cost. Then we study the well-posedness of Bellman equation, or what is commonly known as the average cost optimality equation, under the additional hypothesis of the existence of a small set. We deviate from the usual approach which is based on the vanishing discount method and instead map the problem to an equivalent one for a controlled split chain. We employ a stochastic representation of the Poisson equation to derive the Bellman equation. Next, under suitable assumptions, we establish convergence results for the 'relative value iteration' algorithm which computes the solution of the Bellman equation recursively. In addition, we present some results concerning the stability and asymptotic optimality of the associated rolling horizon policies.Comment: 32 page

Impulse control maximising average cost per unit time: a non-uniformly ergodic case

This paper studies maximisation of an average-cost-per-unit-time ergodic functional over impulse strategies controlling a Feller-Markov process. The uncontrolled process is assumed to be ergodic but, unlike the extant literature, the convergence to invariant measure does not have to be uniformly geometric in total variation norm; in particular, we allow for non-uniform geometric or polynomial convergence. Cost of an impulse may be unbounded, e.g., proportional to the distance the process is shifted. We show that the optimal value does not depend on the initial state and provide optimal or \ve-optimal strategies.Comment: 25 pages; This is an updated version after spinning off two sections of the paper as a basis for arxiv:1607.0601

Optimal Transport and Skorokhod Embedding

The Skorokhod embedding problem is to represent a given probability as the distribution of Brownian motion at a chosen stopping time. Over the last 50 years this has become one of the important classical problems in probability theory and a number of authors have constructed solutions with particular optimality properties. These constructions employ a variety of techniques ranging from excursion theory to potential and PDE theory and have been used in many different branches of pure and applied probability. We develop a new approach to Skorokhod embedding based on ideas and concepts from optimal mass transport. In analogy to the celebrated article of Gangbo and McCann on the geometry of optimal transport, we establish a geometric characterization of Skorokhod embeddings with desired optimality properties. This leads to a systematic method to construct optimal embeddings. It allows us, for the first time, to derive all known optimal Skorokhod embeddings as special cases of one unified construction and leads to a variety of new embeddings. While previous constructions typically used particular properties of Brownian motion, our approach applies to all sufficiently regular Markov processes.Comment: Substantial revision to improve the readability of the pape