Ergodic Control and Polyhedral approaches to PageRank Optimization

We study a general class of PageRank optimization problems which consist in finding an optimal outlink strategy for a web site subject to design constraints. We consider both a continuous problem, in which one can choose the intensity of a link, and a discrete one, in which in each page, there are obligatory links, facultative links and forbidden links. We show that the continuous problem, as well as its discrete variant when there are no constraints coupling different pages, can both be modeled by constrained Markov decision processes with ergodic reward, in which the webmaster determines the transition probabilities of websurfers. Although the number of actions turns out to be exponential, we show that an associated polytope of transition measures has a concise representation, from which we deduce that the continuous problem is solvable in polynomial time, and that the same is true for the discrete problem when there are no coupling constraints. We also provide efficient algorithms, adapted to very large networks. Then, we investigate the qualitative features of optimal outlink strategies, and identify in particular assumptions under which there exists a "master" page to which all controlled pages should point. We report numerical results on fragments of the real web graph.Comment: 39 page

Dynamic Server Allocation over Time Varying Channels with Switchover Delay

We consider a dynamic server allocation problem over parallel queues with randomly varying connectivity and server switchover delay between the queues. At each time slot the server decides either to stay with the current queue or switch to another queue based on the current connectivity and the queue length information. Switchover delay occurs in many telecommunications applications and is a new modeling component of this problem that has not been previously addressed. We show that the simultaneous presence of randomly varying connectivity and switchover delay changes the system stability region and the structure of optimal policies. In the first part of the paper, we consider a system of two parallel queues, and develop a novel approach to explicitly characterize the stability region of the system using state-action frequencies which are stationary solutions to a Markov Decision Process (MDP) formulation. We then develop a frame-based dynamic control (FBDC) policy, based on the state-action frequencies, and show that it is throughput-optimal asymptotically in the frame length. The FBDC policy is applicable to a broad class of network control systems and provides a new framework for developing throughput-optimal network control policies using state-action frequencies. Furthermore, we develop simple Myopic policies that provably achieve more than 90% of the stability region. In the second part of the paper, we extend our results to systems with an arbitrary but finite number of queues.Comment: 38 Pages, 18 figures. arXiv admin note: substantial text overlap with arXiv:1008.234

Scheduling algorithms for throughput maximization in time-varying networks with reconfiguration delays

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 247-258).We consider the control of possibly time-varying wireless networks under reconfiguration delays. Reconfiguration delay is the time it takes to switch network resources from one subset of nodes to another and it is a widespread phenomenon observed in many practical systems. Optimal control of networks has been studied to a great extent in the literature, however, the significant effects of reconfiguration delays received limited attention. Moreover, simultaneous presence of time-varying channels and reconfiguration delays has never been considered and we show that it impacts the system fundamentally. We first consider a Delay Tolerant Network model where data messages arriving randomly in time and space are collected by mobile collectors. In this setting reconfiguration delays correspond to travel times of collectors. We utilize a combination of wireless transmission and controlled mobility to improve the system delay scaling with load [rho] from [theta](1/(1-[rho])²) to [theta](1/1-[rho]), where the former is the delay for the corresponding system without wireless transmission. We propose control algorithms that stabilize the system whenever possible and have optimal delay scaling. Next, we consider a general queuing network model under reconfiguration delays and interference constraints which includes wireless, satellite and optical networks as special cases. We characterize the impacts of reconfiguration delays on system stability and delay, and propose scheduling algorithms that persist with service schedules for durations of time based on queue lengths to minimize negative impacts of reconfiguration delays. These algorithms provide throughput-optimality without requiring knowledge of arrival rates since they dynamically adapt inter-switching durations to stochastic arrivals. Finally, we present optimal scheduling under time-varying channels and reconfiguration delays, which is the main contribution of this thesis. We show that under the simultaneous presence of these two phenomenon network stability region shrinks, previously suggested policies are unstable, and new algorithmic approaches are necessary. We propose techniques based on state-action frequencies of Markov Decision Process theory to characterize the network stability region and propose throughput-optimal algorithms. The state-action frequency technique is applicable to a broad class of systems with or without reconfiguration delays, and provides a new framework for characterizing network stability region and developing throughput-optimal scheduling policies.by Güner Dinc̦er C̦elik.Ph.D