Nonzero-sum Stochastic Games

This paper treats of stochastic games. We focus on nonzero-sum games and provide a detailed survey of selected recent results. In Section 1, we consider stochastic Markov games. A correlation of strategies of the players, involving ``public signals'', is described, and a correlated equilibrium theorem proved recently by Nowak and Raghavan for discounted stochastic games with general state space is presented. We also report an extension of this result to a class of undiscounted stochastic games, satisfying some uniform ergodicity condition. Stopping games are related to stochastic Markov games. In Section 2, we describe a version of Dynkin's game related to observation of a Markov process with random assignment mechanism of states to the players. Some recent contributions of the second author in this area are reported. The paper also contains a brief overview of the theory of nonzero-sum stochastic games and stopping games which is very far from being complete

Conditions for the Solvability of the Linear Programming Formulation for Constrained Discounted Markov Decision Processes

We consider a discrete-time constrained discounted Markov decision process (MDP) with Borel state and action spaces, compact action sets, and lower semi-continuous cost functions. We introduce a set of hypotheses related to a positive weight function which allow us to consider cost functions that might not be bounded below by a constant, and which imply the solvability of the linear programming formulation of the constrained MDP. In particular, we establish the existence of a constrained optimal stationary policy. Our results are illustrated with an application to a fishery management problem

Plans as Situated Action: An Activity Theory Approach to Workflow Systems

: Within the community of CSCW the notion and nature of workflow systems as prescriptions of human work has been debated and criticised. Based on the work of Suchman (1987) the notion of situated action has often been viewed as opposed to planning work. Plans, however, do play an essential role in realising work. Based on experiences from designing a computer system that supports the collaboration within a hospital, this paper discusses how plans themselves are made out of situated action, and in return are realised in situ. Thus, work can be characterised as situated planning. This understanding is backed up by Activity Theory, which emphasises the connection between plans and the contextual conditions for realising these plans in actual work. Introduction The issue of workflow systems has been addressed by several authors as ways of routing information objects among users, and to specify automatic actions to be taken in that routing typically according to certain process models (Me..