Learning for Cross-layer Resource Allocation in the Framework of Cognitive Wireless Networks

The framework of cognitive wireless networks is expected to endow wireless devices with a cognition-intelligence ability with which they can efficiently learn and respond to the dynamic wireless environment. In this dissertation, we focus on the problem of developing cognitive network control mechanisms without knowing in advance an accurate network model. We study a series of cross-layer resource allocation problems in cognitive wireless networks. Based on model-free learning, optimization and game theory, we propose a framework of self-organized, adaptive strategy learning for wireless devices to (implicitly) build the understanding of the network dynamics through trial-and-error. The work of this dissertation is divided into three parts. In the first part, we investigate a distributed, single-agent decision-making problem for real-time video streaming over a time-varying wireless channel between a single pair of transmitter and receiver. By modeling the joint source-channel resource allocation process for video streaming as a constrained Markov decision process, we propose a reinforcement learning scheme to search for the optimal transmission policy without the need to know in advance the details of network dynamics. In the second part of this work, we extend our study from the single-agent to a multi-agent decision-making scenario, and study the energy-efficient power allocation problems in a two-tier, underlay heterogeneous network and in a self-sustainable green network. For the heterogeneous network, we propose a stochastic learning algorithm based on repeated games to allow individual macro- or femto-users to find a Stackelberg equilibrium without flooding the network with local action information. For the self-sustainable green network, we propose a combinatorial auction mechanism that allows mobile stations to adaptively choose the optimal base station and sub-carrier group for transmission only from local payoff and transmission strategy information. In the third part of this work, we study a cross-layer routing problem in an interweaved Cognitive Radio Network (CRN), where an accurate network model is not available and the secondary users that are distributed within the CRN only have access to local action/utility information. In order to develop a spectrum-aware routing mechanism that is robust against potential insider attackers, we model the uncoordinated interaction between CRN nodes in the dynamic wireless environment as a stochastic game. Through decomposition of the stochastic routing game, we propose two stochastic learning algorithm based on a group of repeated stage games for the secondary users to learn the best-response strategies without the need of information flooding

Energy Optimization for Hybrid ARQ

Hybrid automatic repeat request (HARQ) \cite{costello1983error} plays an important role in providing reliable and efficient data transmission. In wireless communications, the wireless channel may vary fast, due to the mobility of the transmitter/receiver and the channel. Forward error correction (FEC) and automatic repeat request (ARQ) are two basic techniques to control errors. FEC employs error correction coding, by adding parity bits to the information bits, to combat channel errors. ARQ allows the receiver to request a retransmission of the packet when an error is detected in the received packet. HARQ gives protection to the wireless transmission by combining FEC and ARQ. In typical HARQ systems, redundancy is added to the information bits, and a retransmission is performed until either the packet is successfully decoded, or a maximum number of transmissions is reached.The motivation to optimize the energy consumption of HARQ is the high energy consumption of wireless communications on mobile devices. Wireless devices usually have a limited battery life, and wireless communications consume the majority of the battery energy of mobile devices. One example is that 3G and Wifi units consume more than 50\% of the energy for some smart phones \cite{tawalbeh2016studying}. Another example is that battery depletion has been identified as one of the primary factors that limit the lifetime of wireless sensor networks \cite{verdone2010wireless}.Previous works on HARQ mainly use information-theoretic approach, which assumes that the number of bits in each transmission round is sufficiently large. This assumption does not necessarily hold for actual codes with finite length. Therefore, in this dissertation, we consider HARQ with actual codes. We use turbo-coded HARQ, since turbo codes are well-known capacity-approaching codes \cite{berrou1993near} and widely used in standards such as 3GPP Long-Term Evolution (LTE) \cite{3gpp2007mulltiplexing}. We study the energy optimization for HARQ in two scenarios: the energy optimization for incremental redundancy (IR) HARQ, and the energy optimization for HARQ in wireless video transmission. For IR HARQ, each retransmission contains additional parity bits beyond those of the previous transmissions. For the first scenario, we consider different cases of channel state information (CSI) at the transmitter: the transmitter has no knowledge of any CSI, or knows the CSI in previous transmission rounds through a perfect feedback channel, or knows both current and previous CSI. The transmitter decides the forward error correction code rate based on the CSI it has. We minimize the energy consumption of turbo-coded HARQ, subject to a packet loss rate constraint. Numerical results show that the energy consumption of HARQ decreases when more CSI information is available at the transmitter. We also compare IR combining with both Chase combining and the system without combining, and IR combining yields the least energy consumption.For the second scenario, we formulate the problem as maximizing the video quality, subject to a constraint on the wireless transmission energy consumption. We consider multiple parameters in multiple layers in a wireless video transmission system: transmit power, alphabet size, FEC code rate, maximum number of transmissions and unequal video data importance. An analytical framework is proposed to include these parameters, which allows us to divide this problem into two sub-problems: data transmission and unequal error protection (UEP) for video content. The problem is tackled by solving the two sub-problems, which are done by exhaustive search and convex optimization, respectively. Simulations of different videos show that the proposed scheme outperforms methods using conventional data transmission and/or unequal error protection. For example, in the low SNR region, there is a total gain of 4.8 to 5.6dB on the peak signal-to-noise ratio of the received video compared to video transmission using conventional HARQ without any video UEP

DeepSHARQ: hybrid error coding using deep learning

Cyber-physical systems operate under changing environments and on resource-constrained devices. Communication in these environments must use hybrid error coding, as pure pro- or reactive schemes cannot always fulfill application demands or have suboptimal performance. However, finding optimal coding configurations that fulfill application constraints—e.g., tolerate loss and delay—under changing channel conditions is a computationally challenging task. Recently, the systems community has started addressing these sorts of problems using hybrid decomposed solutions, i.e., algorithmic approaches for wellunderstood formalized parts of the problem and learning-based approaches for parts that must be estimated (either for reasons of uncertainty or computational intractability). For DeepSHARQ, we revisit our own recent work and limit the learning problem to block length prediction, the major contributor to inference time (and its variation) when searching for hybrid error coding configurations. The remaining parameters are found algorithmically, and hence we make individual contributions with respect to finding close-to-optimal coding configurations in both of these areas—combining them into a hybrid solution. DeepSHARQ applies block length regularization in order to reduce the neural networks in comparison to purely learningbased solutions. The hybrid solution is nearly optimal concerning the channel efficiency of coding configurations it generates, as it is trained so deviations from the optimum are upper bound by a configurable percentage. In addition, DeepSHARQ is capable of reacting to channel changes in real time, thereby enabling cyber-physical systems even on resource-constrained platforms. Tightly integrating algorithmic and learning-based approaches allows DeepSHARQ to react to channel changes faster and with a more predictable time than solutions that rely only on either of the two approaches

Online Learning for QoE-based Video Streaming to Mobile Receivers

International audienceThis paper proposes a cross-layer control mechanism to stream efficiently scalable videos to mobile receivers. Its goal is to maximize the quality of the received video while accounting for the variations of the characteristics of the transmitted content and of the channel. The control problem is cast in the framework of Markov Decision Processes. The optimal actions to apply to the system are learned using reinforcement learning. For that purpose, the quality of the decoded frames at receiver is inferred by an observation (i) of the quality of the various scalability layers and (ii) of the level of queues at the Application and Medium Access Control layers of the transmitter only. Delayed as well as absence of information on the channel state are considered. Experiments show that the performance of the proposed solution is only slightly degraded with delayed or missing channel state information. The performance degradation is larger when considering a basic bitstream extractor, which serves as reference

Lightweight mobile and wireless systems: technologies, architectures, and services

1Department of Information and Communication Systems Engineering (ICSE), University of the Aegean, 81100 Mytilene, Greece 2Department of Information Engineering and Computer Science (DISI), University of Trento, 38123 Trento, Italy 3Department of Informatics, Alexander Technological Educational Institute of Thessaloniki, Thessaloniki, 574 00 Macedonia, Greece 4Centre Tecnologic de Telecomunicacions de Catalunya (CTTC), 08860 Barcelona, Spain 5North Carolina State University (NCSU), Raleigh, NC 27695, US