Two-stage index computation for bandits with switching penalties I : switching costs

This paper addresses the multi-armed bandit problem with switching costs. Asawa and Teneketzis (1996) introduced an index that partly characterizes optimal policies, attaching to each bandit state a "continuation index" (its Gittins index) and a "switching index". They proposed to jointly compute both as the Gittins index of a bandit having 2n states — when the original bandit has n states — which results in an eight-fold increase in O($n^{3}$) arithmetic operations relative to those to compute the continuation index alone. This paper presents a more efficient, decoupled computation method, which in a first stage computes the continuation index and then, in a second stage, computes the switching index an order of magnitude faster in at most $n^{2}$+O(n) arithmetic operations. The paper exploits the fact that the Asawa and Teneketzis index is the Whittle, or marginal productivity, index of a classic bandit with switching costs in its restless reformulation, by deploying work-reward analysis and PCL-indexability methods introduced by the author. A computational study demonstrates the dramatic runtime savings achieved by the new algorithm, the near-optimality of the index policy, and its substantial gains against the benchmark Gittins index policy across a wide range of instances.

Two-stage index computation for bandits with switching penalties II : switching delays

This paper addresses the multi-armed bandit problem with switching penalties including both costs and delays, extending results of the companion paper [J. Niño-Mora. "Two-Stage Index Computation for Bandits with Switching Penalties I: Switching Costs". Conditionally accepted at INFORMS J. Comp.], which addressed the no switching delays case. Asawa and Teneketzis (1996) introduced an index for bandits with delays that partly characterizes optimal policies, attaching to each bandit state a "continuation index" (its Gittins index) and a "switching index", yet gave no algorithm for it. This paper presents an efficient, decoupled computation method, which in a first stage computes the continuation index and then, in a second stage, computes the switching index an order of magnitude faster in at most (5/2)$n^{3}$+O(n) arithmetic operations for an n -state bandit. The paper exploits the fact that the Asawa and Teneketzis index is the Whittle, or marginal productivity, index of a classic bandit with switching penalties in its semi- Markov restless reformulation, by deploying work-reward analysis and LP-indexability methods introduced by the author. A computational study demonstrates the dramatic runtime savings achieved by the new algorithm, the near-optimality of the index policy, and its substantial gains against a benchmark index policy across a wide instance range.

Supraregional agglomeration economies and regional clubs

Recently there have appeared considerable number of papers concerning the idea that externalities affect growth and its dynamics. This paper determines the importance of these external effects to the estimatiom and analysis of an output function. We have observed this fact for the Spanish case at the regional level. The relation of factors in a production function becomes altered by external facts. We estimate the relevance of externalities in this function and try to identify their sources. Convergence or divergence and its rate depend on the existence of regional externalities. Considering the idea that regional economies could be optimally grouped (groups showing the highest levels of homogeneity), there is evidence that these groups are not randomly located in the space. Then, this question naturally arises: Could it be due to the presence of regional externalities? If economic activity is conditioned by regional externalities, the location decision of a firm will be influence by those externalities. This point would be related to the existence of supraregional agglomeration economies. Furthermore, we evaluate the presence and intensity of the sources of such externalities in each group of regions and the possibility of difference sources in different groups.

MARGINAL PRODUCTIVITY INDEX POLICIES FOR SCHEDULING A MULTICLASS DELAY-/LOSS-SENSITIVE QUEUE

We address the problem of scheduling a multiclass M/M/1 queue with a finite dedicated buffer for each class. Some classes are delay-sensitive, modeling real-time traffic (e.g. voice, video), whereas others are loss-sensitive, modeling nonreal-time traffic (e.g. data). Different levels of tolerance to delay and loss are modeled by appropriate linear holding cost and rejection cost rates. The goal is to design well-grounded and tractable scheduling policies which nearly minimize the discounted or long-run average expected cost objective. We develop new dynamic index policies, prescribing to give higher service priority to classes with larger index values, where the priority index of a class measures the marginal productivity of work at its current state. To construct the indices, we deploy the theory of marginal productivity indices (MPIs) and PCLindexability we have introduced in recent work, and further introduce significant extensions to such theory motivated by phenomena observed in the model of concern. The MPI policies are shown to furnish new, insightful structural results, and to exhibit a nearly optimal performance in a computational study.

Characterization and computation of restless bandit marginal productivity indices

The Whittle index [P. Whittle (1988). Restless bandits: Activity allocation in a changing world. J. Appl. Probab. 25A, 287-298] yields a practical scheduling rule for the versatile yet intractable multi-armed restless bandit problem, involving the optimal dynamic priority allocation to multiple stochastic projects, modeled as restless bandits, i.e., binary-action (active/passive) (semi-) Markov decision processes. A growing body of evidence shows that such a rule is nearly optimal in a wide variety of applications, which raises the need to efficiently compute the Whittle index and more general marginal productivity index (MPI) extensions in large-scale models. For such a purpose, this paper extends to restless bandits the parametric linear programming (LP) approach deployed in [J. Niño-Mora. A (2/3)$n^{3}$ fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain, INFORMS J. Comp., in press], which yielded a fast Gittins-index algorithm. Yet the extension is not straightforward, as the MPI is only defined for the limited range of socalled indexable bandits, which motivates the quest for methods to establish indexability. This paper furnishes algorithmic and analytical tools to realize the potential of MPI policies in largescale applications, presenting the following contributions: (i) a complete algorithmic characterization of indexability, for which two block implementations are given; and (ii) more importantly, new analytical conditions for indexability — termed LP-indexability — that leverage knowledge on the structure of optimal policies in particular models, under which the MPI is computed faster by the adaptive-greedy algorithm previously introduced by the author under the more stringent PCL-indexability conditions, for which a new fast-pivoting block implementation is given. The paper further reports on a computational study, measuring the runtime performance of the algorithms, and assessing by a simulation study the high prevalence of indexability and PCL-indexability.

Marginal productivity index policies for problems of admission control and routing to parallel queues with delay

In this paper we consider the problem of admission control of Bernoulli arrivals to a buffer with geometric server, in which the controller’s actions take effect one period after the actual change in the queue length. An optimal policy in terms of marginal productivity indices (MPI) is derived for this problem under the following three performance objectives: (i) minimization of the expected total discounted sum of holding costs and rejection costs, (ii) minimization of the expected time-average sum of holding costs and rejection costs, and (iii) maximization of the expected time-average number of job completions. Our employment of existing theoretical and algorithmic results on restless bandit indexation together with some new results yields a fast algorithm that computes the MPI for a queue with a buffer size of I performing only O(I) arithmetic operations. Such MPI values can be used both to immediately obtain the optimal thresholds for the admission control problem, and to design an index policy for the routing problem (with possible admission control) in the multi-queue system. Thus, this paper further addresses the problem of designing and computing a tractable heuristic policy for dynamic job admission control and/or routing in a discrete time Markovian model of parallel loss queues with one-period delayed state observation and/or action implementation, which comes close to optimizing an infinite-horizon problem under the above three objectives. Our approach seems to be tractable also for the analogous problems with larger delays and, more generally, for arbitrary restless bandits with delays.Admission control, Routing, Parallel queues, Delayed information, Delayed action implementation, Index policy, Restless bandits, Marginal productivity index

An index for dynamic product promotion and the knapsack problem for perishable items

This paper introduces the knapsack problem for perishable items (KPPI), which concerns the optimal dynamic allocation of a limited promotion space to a collection of perishable items. Such a problem is motivated by applications in a variety of industries, where products have an associated lifetime after which they cannot be sold. The paper builds on recent developments on restless bandit indexation and gives an optimal marginal productivity index policy for the dynamic (single) product promotion problem with closed-form indices that yield estructural insights. The performance of the proposed policy for KPPI is investigated in a computational study.Dynamic promotion, Perishable items, Index policies, Knapsack problem, Festless bandits, Finite horizon, Marginal productivity index

Changes and challenges in Spanish higher education

Significant political and sociological changes have occurred in Spain in the last two decades. These changes have considerably affected the higher education system. While the results may be considered positive in general, many aspects still could and should he improved. Higher education in Spain currently has to cope with new challenges in order to place the system at the desirable level of quality. The main changes that have occurred in the Spanish higher education system, and its current challenges, are discussed in this paper.peer-reviewe