To Stay Or To Switch: Multiuser Dynamic Channel Access

In this paper we study opportunistic spectrum access (OSA) policies in a multiuser multichannel random access cognitive radio network, where users perform channel probing and switching in order to obtain better channel condition or higher instantaneous transmission quality. However, unlikely many prior works in this area, including those on channel probing and switching policies for a single user to exploit spectral diversity, and on probing and access policies for multiple users over a single channel to exploit temporal and multiuser diversity, in this study we consider the collective switching of multiple users over multiple channels. In addition, we consider finite arrivals, i.e., users are not assumed to always have data to send and demand for channel follow a certain arrival process. Under such a scenario, the users' ability to opportunistically exploit temporal diversity (the temporal variation in channel quality over a single channel) and spectral diversity (quality variation across multiple channels at a given time) is greatly affected by the level of congestion in the system. We investigate the optimal decision process in this case, and evaluate the extent to which congestion affects potential gains from opportunistic dynamic channel switching

Characterization and computation of restless bandit marginal productivity indices

The Whittle index [P. Whittle (1988). Restless bandits: Activity allocation in a changing world. J. Appl. Probab. 25A, 287-298] yields a practical scheduling rule for the versatile yet intractable multi-armed restless bandit problem, involving the optimal dynamic priority allocation to multiple stochastic projects, modeled as restless bandits, i.e., binary-action (active/passive) (semi-) Markov decision processes. A growing body of evidence shows that such a rule is nearly optimal in a wide variety of applications, which raises the need to efficiently compute the Whittle index and more general marginal productivity index (MPI) extensions in large-scale models. For such a purpose, this paper extends to restless bandits the parametric linear programming (LP) approach deployed in [J. Niño-Mora. A (2/3)$n^{3}$ fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain, INFORMS J. Comp., in press], which yielded a fast Gittins-index algorithm. Yet the extension is not straightforward, as the MPI is only defined for the limited range of socalled indexable bandits, which motivates the quest for methods to establish indexability. This paper furnishes algorithmic and analytical tools to realize the potential of MPI policies in largescale applications, presenting the following contributions: (i) a complete algorithmic characterization of indexability, for which two block implementations are given; and (ii) more importantly, new analytical conditions for indexability — termed LP-indexability — that leverage knowledge on the structure of optimal policies in particular models, under which the MPI is computed faster by the adaptive-greedy algorithm previously introduced by the author under the more stringent PCL-indexability conditions, for which a new fast-pivoting block implementation is given. The paper further reports on a computational study, measuring the runtime performance of the algorithms, and assessing by a simulation study the high prevalence of indexability and PCL-indexability.

The stack resource protocol based on real time transactions

Current hard real time (HRT) kernels have their timely behaviour guaranteed at the cost of a rather restrictive use of the available resources. This makes current HRT scheduling techniques inadequate for use in a multimedia environment where one can profit by a better and more flexible use of the resources. It is shown that one can improve the flexibility and efficiency of real time kernels and a method is proposed for precise quality of service schedulability analysis of the stack resource protocol. This protocol is generalised by introducing real time transactions, which makes its use straightforward and efficient. Transactions can be refined to nested critical sections if the smallest estimation of blocking is desired. The method can be used for hard real time systems in general and for multimedia systems in particular

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work