Machine Learning-Enabled Resource Allocation for Underlay Cognitive Radio Networks

Due to the rapid growth of new wireless communication services and applications, much attention has been directed to frequency spectrum resources and the way they are regulated. Considering that the radio spectrum is a natural limited resource, supporting the ever increasing demands for higher capacity and higher data rates for diverse sets of users, services and applications is a challenging task which requires innovative technologies capable of providing new ways of efficiently exploiting the available radio spectrum. Consequently, dynamic spectrum access (DSA) has been proposed as a replacement for static spectrum allocation policies. The DSA is implemented in three modes including interweave, overlay and underlay mode [1]. The key enabling technology for DSA is cognitive radio (CR), which is among the core prominent technologies for the next generation of wireless communication systems. Unlike conventional radio which is restricted to only operate in designated spectrum bands, a CR has the capability to operate in different spectrum bands owing to its ability in sensing, understanding its wireless environment, learning from past experiences and proactively changing the transmission parameters as needed. These features for CR are provided by an intelligent software package called the cognitive engine (CE). In general, the CE manages radio resources to accomplish cognitive functionalities and allocates and adapts the radio resources to optimize the performance of the network. Cognitive functionality of the CE can be achieved by leveraging machine learning techniques. Therefore, this thesis explores the application of two machine learning techniques in enabling the cognition capability of CE. The two considered machine learning techniques are neural network-based supervised learning and reinforcement learning. Specifically, this thesis develops resource allocation algorithms that leverage the use of machine learning techniques to find the solution to the resource allocation problem for heterogeneous underlay cognitive radio networks (CRNs). The proposed algorithms are evaluated under extensive simulation runs. The first resource allocation algorithm uses a neural network-based learning paradigm to present a fully autonomous and distributed underlay DSA scheme where each CR operates based on predicting its transmission effect on a primary network (PN). The scheme is based on a CE with an artificial neural network that predicts the adaptive modulation and coding configuration for the primary link nearest to a transmitting CR, without exchanging information between primary and secondary networks. By managing the effect of the secondary network (SN) on the primary network, the presented technique maintains the relative average throughput change in the primary network within a prescribed maximum value, while also finding transmit settings for the CRs that result in throughput as large as allowed by the primary network interference limit. The second resource allocation algorithm uses reinforcement learning and aims at distributively maximizing the average quality of experience (QoE) across transmission of CRs with different types of traffic while satisfying a primary network interference constraint. To best satisfy the QoE requirements of the delay-sensitive type of traffics, a cross-layer resource allocation algorithm is derived and its performance is compared against a physical-layer algorithm in terms of meeting end-to-end traffic delay constraints. Moreover, to accelerate the learning performance of the presented algorithms, the idea of transfer learning is integrated. The philosophy behind transfer learning is to allow well-established and expert cognitive agents (i.e. base stations or mobile stations in the context of wireless communications) to teach newly activated and naive agents. Exchange of learned information is used to improve the learning performance of a distributed CR network. This thesis further identifies the best practices to transfer knowledge between CRs so as to reduce the communication overhead. The investigations in this thesis propose a novel technique which is able to accurately predict the modulation scheme and channel coding rate used in a primary link without the need to exchange information between the two networks (e.g. access to feedback channels), while succeeding in the main goal of determining the transmit power of the CRs such that the interference they create remains below the maximum threshold that the primary network can sustain with minimal effect on the average throughput. The investigations in this thesis also provide a physical-layer as well as a cross-layer machine learning-based algorithms to address the challenge of resource allocation in underlay cognitive radio networks, resulting in better learning performance and reduced communication overhead

Skedulering i kognitiva månghopps- och mångbandsradionätverk utnyttjandes potentialfält

The problem of scheduling transmissions in multi-hop multi-band cognitive radio (CR) networks is studied. The scheduling is subdivided into long- and short-term problems. The long-term problem concerns the routing of data packets. It is solved by means of a novel potential field based modeling technique that abstracts routes away from paths defined by sequences of specific network links and onto an underlying potential field. The short-term problem is formulated as a mixed integer nonlinear program (MINLP) that describes the instantaneous interference constraints that secondary users (SUs) have to satisfy if they wish to utilize the spectrum licensed to primary users (PUs). These interference constraints are defined in terms of required signal-to-interference-plus-noise ratios (SINRs) at receivers. A centralized greedy algorithm, as well as a distributed version of it, is proposed for solving the short-term problem. Simulations show that the proposed methods work well in both static and dynamic networks. Moreover, the methods contain adjustable parameters that allow one to e.g. improve the fairness of the scheduling at the cost of overall throughput, or to significantly improve the overall throughput at the cost of a non-zero but arbitrarily small probability of violating the interference constraints.Skedulering av sändningar i kognitiva månghopps- och mångbandsradionätverk undersöks. Skeduleringen delas upp i ett långsiktigt respektive ett kortsiktigt problem. Det långsiktiga problemet behandlar dirigeringen av datapaket. Problemet löses genom en ny potentialfältsbaserad modelleringsteknik där datarutterna beskrivs av ett underliggande potentialfält istället för sekvenser av specifika nätverkslänkar. Det kortsiktiga problemet utgörs av ett icke-linjärt optimeringsproblem som beskriver de interferensvillkor vilka de sekundära användarna måste uppfylla för att få tillgång till det spektrum som licensierats till primära användare. Interferensvillkoren är definierade i form av nödvändiga signal-till-interferens-plus-brusförhållanden hos mottagare. En centraliserad girig algoritm, samt en distribuerad version av den, föreslås som lösning på det kortsiktiga problemet. Simulationer visar att de föreslagna metoderna fungerar väl i både statiska och dynamiska nätverk. Dessutom innehåller metoderna justerbara parametrar med vilka man t.ex. kan förbättra rättvisheten i skeduleringen på bekostnad av den totala datahastigheten, eller märkbart förbättra den totala datahastigheten på bekostnad av en positiv men godtyckligt liten sannolikhet att interferensvillkoren inte alltid uppfylls

Studies on efficient spectrum sharing in coexisting wireless networks.

Wireless communication is facing serious challenges worldwide: the severe spectrum shortage along with the explosive increase of the wireless communication demands. Moreover, different communication networks may coexist in the same geographical area. By allowing multiple communication networks cooperatively or opportunistically sharing the same frequency will potentially enhance the spectrum efficiency. This dissertation aims to investigate important spectrum sharing schemes for coexisting networks. For coexisting networks operating in interweave cognitive radio mode, most existing works focus on the secondary network’s spectrum sensing and accessing schemes. However, the primary network can be selfish and tends to use up all the frequency resource. In this dissertation, a novel optimization scheme is proposed to let primary network maximally release unnecessary frequency resource for secondary networks. The optimization problems are formulated for both uplink and downlink orthogonal frequency-division multiple access (OFDMA)-based primary networks, and near optimal algorithms are proposed as well. For coexisting networks in the underlay cognitive radio mode, this work focuses on the resource allocation in distributed secondary networks as long as the primary network’s rate constraint can be met. Global optimal multicarrier discrete distributed (MCDD) algorithm and suboptimal Gibbs sampler based Lagrangian algorithm (GSLA) are proposed to solve the problem distributively. Regarding to the dirty paper coding (DPC)-based system where multiple networks share the common transmitter, this dissertation focuses on its fundamental performance analysis from information theoretic point of view. Time division multiple access (TDMA) as an orthogonal frequency sharing scheme is also investigated for comparison purpose. Specifically, the delay sensitive quality of service (QoS) requirements are incorporated by considering effective capacity in fast fading and outage capacity in slow fading. The performance metrics in low signal to noise ratio (SNR) regime and high SNR regime are obtained in closed forms followed by the detailed performance analysis

COOPERATIVE NETWORKING AND RELATED ISSUES: STABILITY, ENERGY HARVESTING, AND NEIGHBOR DISCOVERY

This dissertation deals with various newly emerging topics in the context of cooperative networking. The first part is about the cognitive radio. To guarantee the performance of high priority users, it is important to know the activity of the high priority communication system but the knowledge is usually imperfect due to randomness in the observed signal. In such a context, the stability property of cognitive radio systems in the presence of sensing errors is studied. General guidelines on controlling the operating point of the sensing device over its receiver operating characteristics are also given. We then consider the hybrid of different modes of operation for cognitive radio systems with time-varying connectivity. The random connectivity gives additional chances that can be utilized by the low priority communication system. The second part of this dissertation is about the random access. We are specifically interested in the scenario when the nodes are harvesting energy from the environment. For such a system, we accurately assess the effect of limited, but renewable, energy availability on the stability region. The effect of finite capacity batteries is also studied. We next consider the exploitation of diversity amongst users under random access framework. That is, each user adapts its transmission probability based on the local channel state information in a decentralized manner. The impact of imperfect channel state information on the stability region is investigated. Furthermore, it is compared to the class of stationary scheduling policies that make centralized decisions based on the channel state feedback. The backpressure policy for cross-layer control of wireless multi-hop networks is known to be throughput-optimal for i.i.d. arrivals. The third part of this dissertation is about the backpressure-based control for networks with time-correlated arrivals that may exhibit long-range dependency. It is shown that the original backpressure policy is still throughput-optimal but with increased average network delay. The case when the arrival rate vector is possibly outside the stability region is also studied by augmenting the backpressure policy with the flow control mechanism. Lastly, the problem of neighbor discovery in a wireless sensor network is dealt. We first introduce the realistic effect of physical layer considerations in the evaluation of the performance of logical discovery algorithms by incorporating physical layer parameters. Secondly, given the lack of knowledge of the number of neighbors along with the lack of knowledge of the individual signal parameters, we adopt the viewpoint of random set theory to the problem of detecting the transmitting neighbors. Random set theory is a generalization of standard probability theory by assigning sets, rather than values, to random outcomes and it has been applied to multi-user detection problem when the set of transmitters are unknown and dynamically changing

Optimal Pricing Effect on Equilibrium Behaviors of Delay-Sensitive Users in Cognitive Radio Networks

This paper studies price-based spectrum access control in cognitive radio networks, which characterizes network operators' service provisions to delay-sensitive secondary users (SUs) via pricing strategies. Based on the two paradigms of shared-use and exclusive-use dynamic spectrum access (DSA), we examine three network scenarios corresponding to three types of secondary markets. In the first monopoly market with one operator using opportunistic shared-use DSA, we study the operator's pricing effect on the equilibrium behaviors of self-optimizing SUs in a queueing system. %This queue represents the congestion of the multiple SUs sharing the operator's single \ON-\OFF channel that models the primary users (PUs) traffic. We provide a queueing delay analysis with the general distributions of the SU service time and PU traffic using the renewal theory. In terms of SUs, we show that there exists a unique Nash equilibrium in a non-cooperative game where SUs are players employing individual optimal strategies. We also provide a sufficient condition and iterative algorithms for equilibrium convergence. In terms of operators, two pricing mechanisms are proposed with different goals: revenue maximization and social welfare maximization. In the second monopoly market, an operator exploiting exclusive-use DSA has many channels that will be allocated separately to each entering SU. We also analyze the pricing effect on the equilibrium behaviors of the SUs and the revenue-optimal and socially-optimal pricing strategies of the operator in this market. In the third duopoly market, we study a price competition between two operators employing shared-use and exclusive-use DSA, respectively, as a two-stage Stackelberg game. Using a backward induction method, we show that there exists a unique equilibrium for this game and investigate the equilibrium convergence.Comment: 30 pages, one column, double spac

Resource Allocation and Pricing in Secondary Dynamic Spectrum Access Networks

The paradigm shift from static spectrum allocation to a dynamic one has opened many challenges that need to be addressed for the true vision of Dynamic Spectrum Access (DSA) to materialize. This dissertation proposes novel solutions that include: spectrum allocation, routing, and scheduling in DSA networks. First, we propose an auction-based spectrum allocation scheme in a multi-channel environment where secondary users (SUs) bid to buy channels from primary users (PUs) based on the signal to interference and noise ratio (SINR). The channels are allocated such that i) the SUs get their preferred channels, ii) channels are re-used, and iii) there is no interference. Then, we propose a double auction-based spectrum allocation technique by considering multiple bids from SUs and heterogeneity of channels. We use virtual grouping of conflict-free buyers to transform multi-unit bids to single-unit bids. For routing, we propose a market-based model where the PUs determine the optimal price based on the demand for bandwidth by the SUs. Routes are determined through a series of price evaluations between message senders and forwarders. Also, we consider auction-based routing for two cases where buyers can bid for only one channel or they could bid for a combination of non-substitutable channels. For a centralized DSA, we propose two scheduling algorithms-- the first one focuses on maximizing the throughput and the second one focuses on fairness. We extend the scheduling algorithms to multi-channel environment. Expected throughput for every channel is computed by modelling channel state transitions using a discrete-time Markov chain. The state transition probabilities are calculated which occur at the frame/slot boundaries. All proposed algorithms are validated using simulation experiments with different network settings and their performance are studied

Cognitive Radio Systems

Cognitive radio is a hot research area for future wireless communications in the recent years. In order to increase the spectrum utilization, cognitive radio makes it possible for unlicensed users to access the spectrum unoccupied by licensed users. Cognitive radio let the equipments more intelligent to communicate with each other in a spectrum-aware manner and provide a new approach for the co-existence of multiple wireless systems. The goal of this book is to provide highlights of the current research topics in the field of cognitive radio systems. The book consists of 17 chapters, addressing various problems in cognitive radio systems