Restless bandit marginal productivity indices I: singleproject case and optimal control of a make-to-stock M/G/1 queue

This paper develops a framework based on convex optimization and economic ideas to formulate and solve by an index policy the problem of optimal dynamic effort allocation to a generic discrete-state restless bandit (i.e. binary-action: work/rest) project, elucidating a host of issues raised by Whittle (1988)Žs seminal work on the topic. Our contributions include: (i) a unifying definition of a projectŽs marginal productivity index (MPI), characterizing optimal policies; (ii) a complete characterization of indexability (existence of the MPI) as satisfaction by the project of the law of diminishing returns (to effort); (iii) sufficient indexability conditions based on partial conservation laws (PCLs), extending previous results of the author from the finite to the countable state case; (iv) application to a semi-Markov project, including a new MPI for a mixed longrun-average (LRA)/ bias criterion, which exists in relevant queueing control models where the index proposed by Whittle (1988) does not; and (v) optimal MPI policies for service-controlled make-to-order (MTO) and make-to-stock (MTS) M/G/1 queues with convex back order and stock holding cost rates, under discounted and LRA criteria

Device-Centric Cooperation in Mobile Networks

The increasing popularity of applications such as video streaming in today's mobile devices introduces higher demand for throughput, and puts a strain especially on cellular links. Cooperation among mobile devices by exploiting both cellular and local area connections is a promising approach to meet the increasing demand. In this paper, we consider that a group of cooperative mobile devices, exploiting both cellular and local area links and within proximity of each other, are interested in the same video content. Traditional network control algorithms introduce high overhead and delay in this setup as the network control and cooperation decisions are made in a source-centric manner. Instead, we develop a device-centric stochastic cooperation scheme. Our device-centric scheme; DcC allows mobile devices to make control decisions such as flow control, scheduling, and cooperation without loss of optimality. Thanks to being device-centric, DcC reduces; (i) overhead; i.e., the number of control packets that should be transmitted over cellular links, so cellular links are used more efficiently, and (ii) the amount of delay that each packet experiences, which improves quality of service. The simulation results demonstrate the benefits of DcC

Self-Evaluation Applied Mathematics 2003-2008 University of Twente

This report contains the self-study for the research assessment of the Department of Applied Mathematics (AM) of the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) at the University of Twente (UT). The report provides the information for the Research Assessment Committee for Applied Mathematics, dealing with mathematical sciences at the three universities of technology in the Netherlands. It describes the state of affairs pertaining to the period 1 January 2003 to 31 December 2008

Queue Length Asymptotics for Generalized Max-Weight Scheduling in the presence of Heavy-Tailed Traffic

We investigate the asymptotic behavior of the steady-state queue length distribution under generalized max-weight scheduling in the presence of heavy-tailed traffic. We consider a system consisting of two parallel queues, served by a single server. One of the queues receives heavy-tailed traffic, and the other receives light-tailed traffic. We study the class of throughput optimal max-weight-alpha scheduling policies, and derive an exact asymptotic characterization of the steady-state queue length distributions. In particular, we show that the tail of the light queue distribution is heavier than a power-law curve, whose tail coefficient we obtain explicitly. Our asymptotic characterization also contains an intuitively surprising result - the celebrated max-weight scheduling policy leads to the worst possible tail of the light queue distribution, among all non-idling policies. Motivated by the above negative result regarding the max-weight-alpha policy, we analyze a log-max-weight (LMW) scheduling policy. We show that the LMW policy guarantees an exponentially decaying light queue tail, while still being throughput optimal