This paper surveys recent work by the author on the theoretical and
algorithmic aspects of restless bandit indexation as well as on its application
to a variety of problems involving the dynamic allocation of priority to
multiple stochastic projects. The main aim is to present ideas and methods in
an accessible form that can be of use to researchers addressing problems of
such a kind. Besides building on the rich literature on bandit problems, our
approach draws on ideas from linear programming, economics, and multi-objective
optimization. In particular, it was motivated to address issues raised in the
seminal work of Whittle (Restless bandits: activity allocation in a changing
world. In: Gani J. (ed.) A Celebration of Applied Probability, J. Appl.
Probab., vol. 25A, Applied Probability Trust, Sheffield, pp. 287-298, 1988)
where he introduced the index for restless bandits that is the starting point
of this work. Such an index, along with previously proposed indices and more
recent extensions, is shown to be unified through the intuitive concept of
``marginal productivity index'' (MPI), which measures the marginal productivity
of work on a project at each of its states. In a multi-project setting, MPI
policies are economically sound, as they dynamically allocate higher priority
to those projects where work appears to be currently more productive. Besides
being tractable and widely applicable, a growing body of computational evidence
indicates that such index policies typically achieve a near-optimal performance
and substantially outperform benchmark policies derived from conventional
approaches.Comment: 7 figure