Delay-Optimal User Scheduling and Inter-Cell Interference Management in Cellular Network via Distributive Stochastic Learning

In this paper, we propose a distributive queueaware intra-cell user scheduling and inter-cell interference (ICI) management control design for a delay-optimal celluar downlink system with M base stations (BSs), and K users in each cell. Each BS has K downlink queues for K users respectively with heterogeneous arrivals and delay requirements. The ICI management control is adaptive to joint queue state information (QSI) over a slow time scale, while the user scheduling control is adaptive to both the joint QSI and the joint channel state information (CSI) over a faster time scale. We show that the problem can be modeled as an infinite horizon average cost Partially Observed Markov Decision Problem (POMDP), which is NP-hard in general. By exploiting the special structure of the problem, we shall derive an equivalent Bellman equation to solve the POMDP problem. To address the distributive requirement and the issue of dimensionality and computation complexity, we derive a distributive online stochastic learning algorithm, which only requires local QSI and local CSI at each of the M BSs. We show that the proposed learning algorithm converges almost surely (with probability 1) and has significant gain compared with various baselines. The proposed solution only has linear complexity order O(MK)

Optimal provision of distributed reserves under dynamic energy service preferences

We propose and solve a stochastic dynamic programming (DP) problem addressing the optimal provision of regulation service reserves (RSR) by controlling dynamic demand preferences in smart buildings. A major contribution over past dynamic pricing work is that we pioneer the relaxation of static, uniformly distributed utility of demand. In this paper we model explicitly the dynamics of energy service preferences leading to a non-uniform and time varying probability distribution of demand utility. More explicitly, we model active and idle duty cycle appliances in a smart building as a closed queuing system with price-controlled arrival rates into the active appliance queue. Focusing on cooling appliances, we model the utility associated with the transition from idle to active as a non-uniform time varying function. We (i) derive an analytic characterization of the optimal policy and the differential cost function, and (ii) prove optimal policy monotonicity and value function convexity. These properties enable us to propose and implement a smart assisted value iteration (AVI) algorithm and an approximate DP (ADP) that exploits related functional approximations. Numerical results demonstrate the validity of the solution techniques and the computational advantage of the proposed ADP on realistic, large-state-space problems

Traffic-Driven Spectrum Allocation in Heterogeneous Networks

Next generation cellular networks will be heterogeneous with dense deployment of small cells in order to deliver high data rate per unit area. Traffic variations are more pronounced in a small cell, which in turn lead to more dynamic interference to other cells. It is crucial to adapt radio resource management to traffic conditions in such a heterogeneous network (HetNet). This paper studies the optimization of spectrum allocation in HetNets on a relatively slow timescale based on average traffic and channel conditions (typically over seconds or minutes). Specifically, in a cluster with $n$ base transceiver stations (BTSs), the optimal partition of the spectrum into $2^n$ segments is determined, corresponding to all possible spectrum reuse patterns in the downlink. Each BTS's traffic is modeled using a queue with Poisson arrivals, the service rate of which is a linear function of the combined bandwidth of all assigned spectrum segments. With the system average packet sojourn time as the objective, a convex optimization problem is first formulated, where it is shown that the optimal allocation divides the spectrum into at most $n$ segments. A second, refined model is then proposed to address queue interactions due to interference, where the corresponding optimal allocation problem admits an efficient suboptimal solution. Both allocation schemes attain the entire throughput region of a given network. Simulation results show the two schemes perform similarly in the heavy-traffic regime, in which case they significantly outperform both the orthogonal allocation and the full-frequency-reuse allocation. The refined allocation shows the best performance under all traffic conditions.Comment: 13 pages, 11 figures, accepted for publication by JSAC-HC

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms

Continuous-time Markov chains with alarms (ACTMCs) allow for alarm events that can be non-exponentially distributed. Within parametric ACTMCs, the parameters of alarm-event distributions are not given explicitly and can be subject of parameter synthesis. An algorithm solving the $\varepsilon$-optimal parameter synthesis problem for parametric ACTMCs with long-run average optimization objectives is presented. Our approach is based on reduction of the problem to finding long-run average optimal strategies in semi-Markov decision processes (semi-MDPs) and sufficient discretization of parameter (i.e., action) space. Since the set of actions in the discretized semi-MDP can be very large, a straightforward approach based on explicit action-space construction fails to solve even simple instances of the problem. The presented algorithm uses an enhanced policy iteration on symbolic representations of the action space. The soundness of the algorithm is established for parametric ACTMCs with alarm-event distributions satisfying four mild assumptions that are shown to hold for uniform, Dirac and Weibull distributions in particular, but are satisfied for many other distributions as well. An experimental implementation shows that the symbolic technique substantially improves the efficiency of the synthesis algorithm and allows to solve instances of realistic size.Comment: This article is a full version of a paper accepted to the Conference on Quantitative Evaluation of SysTems (QEST) 201