Average cost optimal control under weak ergodicity hypotheses: Relative value iterations

We study Markov decision processes with Polish state and action spaces. The action space is state dependent and is not necessarily compact. We first establish the existence of an optimal ergodic occupation measure using only a near-monotone hypothesis on the running cost. Then we study the well-posedness of Bellman equation, or what is commonly known as the average cost optimality equation, under the additional hypothesis of the existence of a small set. We deviate from the usual approach which is based on the vanishing discount method and instead map the problem to an equivalent one for a controlled split chain. We employ a stochastic representation of the Poisson equation to derive the Bellman equation. Next, under suitable assumptions, we establish convergence results for the 'relative value iteration' algorithm which computes the solution of the Bellman equation recursively. In addition, we present some results concerning the stability and asymptotic optimality of the associated rolling horizon policies.Comment: 32 page

A Relative Value Iteration Algorithm for Non-degenerate Controlled Diffusions

The ergodic control problem for a non-degenerate controlled diffusion controlled through its drift is considered under a uniform stability condition that ensures the well-posedness of the associated Hamilton-Jacobi-Bellman (HJB) equation. A nonlinear parabolic evolution equation is then proposed as a continuous time continuous state space analog of White's `relative value iteration' algorithm for solving the ergodic dynamic programming equation for the finite state finite action case. Its convergence to the solution of the HJB equation is established using the theory of monotone dynamical systems and also, alternatively, by using the theory of reverse martingales.Comment: 17 page

Harbours of crisis and consent: The technopolitics of coastal infrastructure in colonial Cyprus, 1895-1908

By the late 1800s British colonial rule in Cyprus was experiencing both a socioeconomic and a legitimacy crisis. Britain's development projects were intended to quell the crisis and consolidate colonial authority. Famagusta Harbour construction was an integral part of that programme, but it antagonised wealthy and influential Cypriots in Larnaca. They believed that such infrastructure would undermine the importance of Larnaca harbour and threaten their commercial and political interests. Their protests threatened the colonial administration with a new crisis that was averted by the integration of Larnaca's Harbour into British plans. The colonial regime had to negotiate and co-operate with local networks of power in order to realise its development programme: harbour development was no mere rational engineering exercise. © The Author(s) 2016

Discrete-time controlled markov processes with average cost criterion: a survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation