A state action frequency approach to throughput maximization over uncertain wireless channels

We consider scheduling over a wireless system, where the channel state information is not available a priori to the scheduler, but can be inferred from the past. Specifically, the wireless system is modeled as a network of parallel queues. We assume that the channel state of each queue evolves stochastically as an ON/OFF Markov chain. The scheduler, which is aware of the queue lengths but is oblivious of the channel states, has to choose one queue at a time for transmission. The scheduler has no information regarding the current channel states, but can estimate them by using the acknowledgment history. We first characterize the capacity region of the system using tools from Markov Decision Processes (MDP) theory. Specifically, we prove that the capacity region boundary is the uniform limit of a sequence of Linear Programming (LP) solutions. Next, we combine the LP solution with a queue length based scheduling mechanism that operates over long `frames,' to obtain a throughput optimal policy for the system. By incorporating results from MDP theory within the Lyapunov-stability framework, we show that our frame-based policy stabilizes the system for all arrival rates that lie in the interior of the capacity region.National Science Foundation (U.S.) (NSF grants CNS-0626781)National Science Foundation (U.S.) (NSF grant CNS-0915988)United States. Army Research Office (ARO Muri Grant W911NF-08-1-0238)Israel Science Foundation (contract 890015

Exploiting Channel Memory for Multi-User Wireless Scheduling without Channel Measurement: Capacity Regions and Algorithms

We study the fundamental network capacity of a multi-user wireless downlink under two assumptions: (1) Channels are not explicitly measured and thus instantaneous states are unknown, (2) Channels are modeled as ON/OFF Markov chains. This is an important network model to explore because channel probing may be costly or infeasible in some contexts. In this case, we can use channel memory with ACK/NACK feedback from previous transmissions to improve network throughput. Computing in closed form the capacity region of this network is difficult because it involves solving a high dimension partially observed Markov decision problem. Instead, in this paper we construct an inner and outer bound on the capacity region, showing that the bound is tight when the number of users is large and the traffic is symmetric. For the case of heterogeneous traffic and any number of users, we propose a simple queue-dependent policy that can stabilize the network with any data rates strictly within the inner capacity bound. The stability analysis uses a novel frame-based Lyapunov drift argument. The outer-bound analysis uses stochastic coupling and state aggregation to bound the performance of a restless bandit problem using a related multi-armed bandit system. Our results are useful in cognitive radio networks, opportunistic scheduling with delayed/uncertain channel state information, and restless bandit problems.Comment: 17 pages, 8 figures. Submitted to IEEE Transactions on Information Theory. The whole paper is revised and the title is changed for better clarification

Dynamic Server Allocation over Time Varying Channels with Switchover Delay

We consider a dynamic server allocation problem over parallel queues with randomly varying connectivity and server switchover delay between the queues. At each time slot the server decides either to stay with the current queue or switch to another queue based on the current connectivity and the queue length information. Switchover delay occurs in many telecommunications applications and is a new modeling component of this problem that has not been previously addressed. We show that the simultaneous presence of randomly varying connectivity and switchover delay changes the system stability region and the structure of optimal policies. In the first part of the paper, we consider a system of two parallel queues, and develop a novel approach to explicitly characterize the stability region of the system using state-action frequencies which are stationary solutions to a Markov Decision Process (MDP) formulation. We then develop a frame-based dynamic control (FBDC) policy, based on the state-action frequencies, and show that it is throughput-optimal asymptotically in the frame length. The FBDC policy is applicable to a broad class of network control systems and provides a new framework for developing throughput-optimal network control policies using state-action frequencies. Furthermore, we develop simple Myopic policies that provably achieve more than 90% of the stability region. In the second part of the paper, we extend our results to systems with an arbitrary but finite number of queues.Comment: 38 Pages, 18 figures. arXiv admin note: substantial text overlap with arXiv:1008.234

Asymptotic performance of queue length based network control policies

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 199-204).In a communication network, asymptotic quality of service metrics specify the probability that the delay or buffer occupancy becomes large. An understanding of these metrics is essential for providing worst-case delay guarantees, provisioning buffer sizes in networks, and to estimate the frequency of packet-drops due to buffer overflow. Second, many network control tasks utilize queue length information to perform effectively, which inevitably adds to the control overheads in a network. Therefore, it is important to understand the role played by queue length information in network control, and its impact on various performance metrics. In this thesis, we study the interplay between the asymptotic behavior of buffer occupancy, queue length information, and traffic statistics in the context of scheduling, flow control, and resource allocation. First, we consider a single-server queue and deal with the question of how often control messages need to be sent in order to effectively control congestion in the queue. Our results show that arbitrarily infrequent queue length information is sufficient to ensure optimal asymptotic decay for the congestion probability, as long as the control information is accurately received. However, if the control messages are subject to errors, the congestion probability can increase drastically, even if the control messages are transmitted often. Next, we consider a system of parallel queues sharing a server, and fed by a statistically homogeneous traffic pattern. We obtain the large deviation exponent of the buffer overflow probability under the well known max-weight scheduling policy. We also show that the queue length based max-weight scheduling outperforms some well known queue-blind policies in terms of the buffer overflow probability. Finally, we study the asymptotic behavior of the queue length distributions when a mix of heavy-tailed and light-tailed traffic flows feeds a system of parallel queues. We obtain an exact asymptotic queue length characterization under generalized max-weight scheduling. In contrast to the statistically homogeneous traffic scenario, we show that max-weight scheduling leads to poor asymptotic behavior for the light-tailed traffic, whereas a queue-blind priority policy gives good asymptotic behavior.by Krishna Prasanna Jagannathan.Ph.D

Scheduling algorithms for throughput maximization in time-varying networks with reconfiguration delays

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 247-258).We consider the control of possibly time-varying wireless networks under reconfiguration delays. Reconfiguration delay is the time it takes to switch network resources from one subset of nodes to another and it is a widespread phenomenon observed in many practical systems. Optimal control of networks has been studied to a great extent in the literature, however, the significant effects of reconfiguration delays received limited attention. Moreover, simultaneous presence of time-varying channels and reconfiguration delays has never been considered and we show that it impacts the system fundamentally. We first consider a Delay Tolerant Network model where data messages arriving randomly in time and space are collected by mobile collectors. In this setting reconfiguration delays correspond to travel times of collectors. We utilize a combination of wireless transmission and controlled mobility to improve the system delay scaling with load [rho] from [theta](1/(1-[rho])²) to [theta](1/1-[rho]), where the former is the delay for the corresponding system without wireless transmission. We propose control algorithms that stabilize the system whenever possible and have optimal delay scaling. Next, we consider a general queuing network model under reconfiguration delays and interference constraints which includes wireless, satellite and optical networks as special cases. We characterize the impacts of reconfiguration delays on system stability and delay, and propose scheduling algorithms that persist with service schedules for durations of time based on queue lengths to minimize negative impacts of reconfiguration delays. These algorithms provide throughput-optimality without requiring knowledge of arrival rates since they dynamically adapt inter-switching durations to stochastic arrivals. Finally, we present optimal scheduling under time-varying channels and reconfiguration delays, which is the main contribution of this thesis. We show that under the simultaneous presence of these two phenomenon network stability region shrinks, previously suggested policies are unstable, and new algorithmic approaches are necessary. We propose techniques based on state-action frequencies of Markov Decision Process theory to characterize the network stability region and propose throughput-optimal algorithms. The state-action frequency technique is applicable to a broad class of systems with or without reconfiguration delays, and provides a new framework for characterizing network stability region and developing throughput-optimal scheduling policies.by Güner Dinc̦er C̦elik.Ph.D