M/M/∞\infty queues in semi-Markovian random environment

In this paper we investigate an M/M/$\infty$ queue whose parameters depend on an external random environment that we assume to be a semi-Markovian process with finite state space. For this model we show a recursive formula that allows to compute all the factorial moments for the number of customers in the system in steady state. The used technique is based on the calculation of the raw moments of the measure of a bidimensional random set. Finally the case when the random environment has only two states is deeper analyzed. We obtain an explicit formula to compute the above mentioned factorial moments when at least one of the two states has sojourn time exponentially distributed.Comment: 17 pages, 2 figure

A decomposition approach for undiscounted two-person zero-sum stochastic games

Two-person zero-sum stochastic games are considered under the long-run average expected payoff criterion. State and action spaces are assumed finite. By making use of the concept of maximal communicating classes, the following decomposition algorithm is introduced for solving two-person zero-sum stochastic games: First, the state space is decomposed into maximal communicating classes. Then, these classes are organized in an hierarchical order where each level may contain more than one maximal communicating class. Best stationary strategies for the states in a maximal communicating class at a level are determined by using the best stationary strategies of the states in the previous levels that are accessible from that class. At the initial level, a restricted game is defined for each closed maximal communicating class and these restricted games are solved independently. It is shown that the proposed decomposition algorithm is exact in the sense that the solution obtained from the decomposition procedure gives the best stationary strategies for the original stochastic game

An infinite-server queue influenced by a semi-Markovian environment

We consider an infinite-server queue, where the arrival and service rates are both governed by a semi-Markov process that is independent of all other aspects of the queue. In particular, we derive a system of equations that are satisfied by various "parts" of the generating function of the steady-state queue-length, while assuming that all arrivals bring an amount of work to the system that is either Erlang or hyperexponentially distributed. These equations are then used to show how to derive all moments of the steady-state queue-length. We then conclude by showing how these results can be slightly extended, and used, along with a transient version of Little’s law, to generate rigorous approximations of the steady-state queue-length in the case that the amount of work brought by a given arrival is of an arbitrary distribution