Learning for Cross-layer Resource Allocation in the Framework of Cognitive Wireless Networks

The framework of cognitive wireless networks is expected to endow wireless devices with a cognition-intelligence ability with which they can efficiently learn and respond to the dynamic wireless environment. In this dissertation, we focus on the problem of developing cognitive network control mechanisms without knowing in advance an accurate network model. We study a series of cross-layer resource allocation problems in cognitive wireless networks. Based on model-free learning, optimization and game theory, we propose a framework of self-organized, adaptive strategy learning for wireless devices to (implicitly) build the understanding of the network dynamics through trial-and-error. The work of this dissertation is divided into three parts. In the first part, we investigate a distributed, single-agent decision-making problem for real-time video streaming over a time-varying wireless channel between a single pair of transmitter and receiver. By modeling the joint source-channel resource allocation process for video streaming as a constrained Markov decision process, we propose a reinforcement learning scheme to search for the optimal transmission policy without the need to know in advance the details of network dynamics. In the second part of this work, we extend our study from the single-agent to a multi-agent decision-making scenario, and study the energy-efficient power allocation problems in a two-tier, underlay heterogeneous network and in a self-sustainable green network. For the heterogeneous network, we propose a stochastic learning algorithm based on repeated games to allow individual macro- or femto-users to find a Stackelberg equilibrium without flooding the network with local action information. For the self-sustainable green network, we propose a combinatorial auction mechanism that allows mobile stations to adaptively choose the optimal base station and sub-carrier group for transmission only from local payoff and transmission strategy information. In the third part of this work, we study a cross-layer routing problem in an interweaved Cognitive Radio Network (CRN), where an accurate network model is not available and the secondary users that are distributed within the CRN only have access to local action/utility information. In order to develop a spectrum-aware routing mechanism that is robust against potential insider attackers, we model the uncoordinated interaction between CRN nodes in the dynamic wireless environment as a stochastic game. Through decomposition of the stochastic routing game, we propose two stochastic learning algorithm based on a group of repeated stage games for the secondary users to learn the best-response strategies without the need of information flooding

Scalable Multiuser Immersive Communications with Multi-numerology and Mini-slot

This paper studies multiuser immersive communications networks in which different user equipment may demand various extended reality (XR) services. In such heterogeneous networks, time-frequency resource allocation needs to be more adaptive since XR services are usually multi-modal and latency-sensitive. To this end, we develop a scalable time-frequency resource allocation method based on multi-numerology and mini-slot. To appropriately determining the discrete parameters of multi-numerology and mini-slot for multiuser immersive communications, the proposed method first presents a novel flexible time-frequency resource block configuration, then it leverages the deep reinforcement learning to maximize the total quality-of-experience (QoE) under different users' QoE constraints. The results confirm the efficiency and scalability of the proposed time-frequency resource allocation method

Resource Allocation for Delay Differentiated Traffic in Multiuser OFDM Systems

Most existing work on adaptive allocation of subcarriers and power in multiuser orthogonal frequency division multiplexing (OFDM) systems has focused on homogeneous traffic consisting solely of either delay-constrained data (guaranteed service) or non-delay-constrained data (best-effort service). In this paper, we investigate the resource allocation problem in a heterogeneous multiuser OFDM system with both delay-constrained (DC) and non-delay-constrained (NDC) traffic. The objective is to maximize the sum-rate of all the users with NDC traffic while maintaining guaranteed rates for the users with DC traffic under a total transmit power constraint. Through our analysis we show that the optimal power allocation over subcarriers follows a multi-level water-filling principle; moreover, the valid candidates competing for each subcarrier include only one NDC user but all DC users. By converting this combinatorial problem with exponential complexity into a convex problem or showing that it can be solved in the dual domain, efficient iterative algorithms are proposed to find the optimal solutions. To further reduce the computational cost, a low-complexity suboptimal algorithm is also developed. Numerical studies are conducted to evaluate the performance the proposed algorithms in terms of service outage probability, achievable transmission rate pairs for DC and NDC traffic, and multiuser diversity.Comment: 29 pages, 8 figures, submitted to IEEE Transactions on Wireless Communication