Reinforcement Learning-based Access Schemes in Cognitive Radio Networks

In this thesis, we propose different MAC protocols based on three Reinforcement Learning (RL) approaches, namely Q-Learning, Deep Q-Network (DQN), and Deep Deterministic Policy Gradient (DDPG). We exploit the primary user (PU) feedback, in the form of ARQ and CQI bits, to enhance the performance of the secondary user (SU) MAC protocols. Exploiting the PU feedback information can be applied on the top of any SU sensing-based MAC protocol. Our proposed model relies on two main pillars, namely, an infinite-state Partially Observable Markov Decision Process (POMDP) to model the system dynamics besides a queuing-theoretic model for the PU queue; the states represent whether a packet is delivered or not from the PU’s queue and the PU channel state. The proposed RL access schemes are meant to design the best SU’s access probabilities in the absence of prior knowledge of the environment, by exploring and exploiting discrete and continuous action spaces, based on the last observed PU’s feedback. The performance of the proposed schemes show better results compared to conventional methods under more realistic assumptions, which is one major advantage of our proposed MAC protocols

Interference Alignment for Cognitive Radio Communications and Networks: A Survey

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).Interference alignment (IA) is an innovative wireless transmission strategy that has shown to be a promising technique for achieving optimal capacity scaling of a multiuser interference channel at asymptotically high-signal-to-noise ratio (SNR). Transmitters exploit the availability of multiple signaling dimensions in order to align their mutual interference at the receivers. Most of the research has focused on developing algorithms for determining alignment solutions as well as proving interference alignment’s theoretical ability to achieve the maximum degrees of freedom in a wireless network. Cognitive radio, on the other hand, is a technique used to improve the utilization of the radio spectrum by opportunistically sensing and accessing unused licensed frequency spectrum, without causing harmful interference to the licensed users. With the increased deployment of wireless services, the possibility of detecting unused frequency spectrum becomes diminished. Thus, the concept of introducing interference alignment in cognitive radio has become a very attractive proposition. This paper provides a survey of the implementation of IA in cognitive radio under the main research paradigms, along with a summary and analysis of results under each system model.Peer reviewe

Cognitive Radio Networks: Realistic or Not?

A large volume of research has been conducted in the cognitive radio (CR) area the last decade. However, the deployment of a commercial CR network is yet to emerge. A large portion of the existing literature does not build on real world scenarios, hence, neglecting various important interactions of the research with commercial telecommunication networks. For instance, a lot of attention has been paid to spectrum sensing as the front line functionality that needs to be completed in an efficient and accurate manner to enable an opportunistic CR network architecture. This is necessary to detect the existence of spectrum holes without which no other procedure can be fulfilled. However, simply sensing (cooperatively or not) the energy received from a primary transmitter cannot enable correct dynamic spectrum access. For example, the low strength of a primary transmitter's signal does not assure that there will be no interference to a nearby primary receiver. In addition, the presence of a primary transmitter's signal does not mean that CR network users cannot access the spectrum since there might not be any primary receiver in the vicinity. Despite the existing elegant and clever solutions to the DSA problem no robust, implementable scheme has emerged. In this paper, we challenge the basic premises of the proposed schemes. We further argue that addressing the technical challenges we face in deploying robust CR networks can only be achieved if we radically change the way we design their basic functionalities. In support of our argument, we present a set of real-world scenarios, inspired by realistic settings in commercial telecommunications networks, focusing on spectrum sensing as a basic and critical functionality in the deployment of CRs. We use these scenarios to show why existing DSA paradigms are not amenable to realistic deployment in complex wireless environments.Comment: Work in progres

A Message Passing Approach for Decision Fusion in Adversarial Multi-Sensor Networks

We consider a simple, yet widely studied, set-up in which a Fusion Center (FC) is asked to make a binary decision about a sequence of system states by relying on the possibly corrupted decisions provided by byzantine nodes, i.e. nodes which deliberately alter the result of the local decision to induce an error at the fusion center. When independent states are considered, the optimum fusion rule over a batch of observations has already been derived, however its complexity prevents its use in conjunction with large observation windows. In this paper, we propose a near-optimal algorithm based on message passing that greatly reduces the computational burden of the optimum fusion rule. In addition, the proposed algorithm retains very good performance also in the case of dependent system states. By first focusing on the case of small observation windows, we use numerical simulations to show that the proposed scheme introduces a negligible increase of the decision error probability compared to the optimum fusion rule. We then analyse the performance of the new scheme when the FC make its decision by relying on long observation windows. We do so by considering both the case of independent and Markovian system states and show that the obtained performance are superior to those obtained with prior suboptimal schemes. As an additional result, we confirm the previous finding that, in some cases, it is preferable for the byzantine nodes to minimise the mutual information between the sequence system states and the reports submitted to the FC, rather than always flipping the local decision