Machine Learning-Enabled Resource Allocation for Underlay Cognitive Radio Networks

Due to the rapid growth of new wireless communication services and applications, much attention has been directed to frequency spectrum resources and the way they are regulated. Considering that the radio spectrum is a natural limited resource, supporting the ever increasing demands for higher capacity and higher data rates for diverse sets of users, services and applications is a challenging task which requires innovative technologies capable of providing new ways of efficiently exploiting the available radio spectrum. Consequently, dynamic spectrum access (DSA) has been proposed as a replacement for static spectrum allocation policies. The DSA is implemented in three modes including interweave, overlay and underlay mode [1]. The key enabling technology for DSA is cognitive radio (CR), which is among the core prominent technologies for the next generation of wireless communication systems. Unlike conventional radio which is restricted to only operate in designated spectrum bands, a CR has the capability to operate in different spectrum bands owing to its ability in sensing, understanding its wireless environment, learning from past experiences and proactively changing the transmission parameters as needed. These features for CR are provided by an intelligent software package called the cognitive engine (CE). In general, the CE manages radio resources to accomplish cognitive functionalities and allocates and adapts the radio resources to optimize the performance of the network. Cognitive functionality of the CE can be achieved by leveraging machine learning techniques. Therefore, this thesis explores the application of two machine learning techniques in enabling the cognition capability of CE. The two considered machine learning techniques are neural network-based supervised learning and reinforcement learning. Specifically, this thesis develops resource allocation algorithms that leverage the use of machine learning techniques to find the solution to the resource allocation problem for heterogeneous underlay cognitive radio networks (CRNs). The proposed algorithms are evaluated under extensive simulation runs. The first resource allocation algorithm uses a neural network-based learning paradigm to present a fully autonomous and distributed underlay DSA scheme where each CR operates based on predicting its transmission effect on a primary network (PN). The scheme is based on a CE with an artificial neural network that predicts the adaptive modulation and coding configuration for the primary link nearest to a transmitting CR, without exchanging information between primary and secondary networks. By managing the effect of the secondary network (SN) on the primary network, the presented technique maintains the relative average throughput change in the primary network within a prescribed maximum value, while also finding transmit settings for the CRs that result in throughput as large as allowed by the primary network interference limit. The second resource allocation algorithm uses reinforcement learning and aims at distributively maximizing the average quality of experience (QoE) across transmission of CRs with different types of traffic while satisfying a primary network interference constraint. To best satisfy the QoE requirements of the delay-sensitive type of traffics, a cross-layer resource allocation algorithm is derived and its performance is compared against a physical-layer algorithm in terms of meeting end-to-end traffic delay constraints. Moreover, to accelerate the learning performance of the presented algorithms, the idea of transfer learning is integrated. The philosophy behind transfer learning is to allow well-established and expert cognitive agents (i.e. base stations or mobile stations in the context of wireless communications) to teach newly activated and naive agents. Exchange of learned information is used to improve the learning performance of a distributed CR network. This thesis further identifies the best practices to transfer knowledge between CRs so as to reduce the communication overhead. The investigations in this thesis propose a novel technique which is able to accurately predict the modulation scheme and channel coding rate used in a primary link without the need to exchange information between the two networks (e.g. access to feedback channels), while succeeding in the main goal of determining the transmit power of the CRs such that the interference they create remains below the maximum threshold that the primary network can sustain with minimal effect on the average throughput. The investigations in this thesis also provide a physical-layer as well as a cross-layer machine learning-based algorithms to address the challenge of resource allocation in underlay cognitive radio networks, resulting in better learning performance and reduced communication overhead

Spectrum Sensing Security in Cognitive Radio Networks

This thesis explores the use of unsupervised machine learning for spectrum sensing in cognitive radio (CR) networks from a security perspective. CR is an enabling technology for dynamic spectrum access (DSA) because of a CR's ability to reconfigure itself in a smart way. CR can adapt and use unoccupied spectrum with the help of spectrum sensing and DSA. DSA is an efficient way to dynamically allocate white spaces (unutilized spectrum) to other CR users in order to tackle the spectrum scarcity problem and improve spectral efficiency. So far various techniques have been developed to efficiently detect and classify signals in a DSA environment. Neural network techniques, especially those using unsupervised learning have some key advantages over other methods mainly because of the fact that minimal preconfiguration is required to sense the spectrum. However, recent results have shown some possible security vulnerabilities, which can be exploited by adversarial users to gain unrestricted access to spectrum by fooling signal classifiers. It is very important to address these new classes of security threats and challenges in order to make CR a long-term commercially viable concept. This thesis identifies some key security vulnerabilities when unsupervised machine learning is used for spectrum sensing and also proposes mitigation techniques to counter the security threats. The simulation work demonstrates the ability of malicious user to manipulate signals in such a way to confuse signal classifier. The signal classifier is forced by the malicious user to draw incorrect decision boundaries by presenting signal features which are akin to a primary user. Hence, a malicious user is able to classify itself as a primary user and thus gains unrivaled access to the spectrum. First, performance of various classification algorithms are evaluated. K-means and weighted classification algorithms are selected because of their robustness against proposed attacks as compared to other classification algorithm. Second, connection attack, point cluster attack, and random noise attack are shown to have an adverse effect on classification algorithms. In the end, some mitigation techniques are proposed to counter the effect of these attacks

Cognitive Radio Communications for Vehicular Technology – Wavelet Applications

Wireless communications are nowadays a dominant part of our lives: from domotics, through industrial applications and up to infomobility services. The key to the co-existence of wireless systems operating in closely located or even overlapping areas, is sharing of the spectral resource. The optimization of this resource is the main driving force behind the emerging changes in the policies for radio resources allocation. The current approach in spectrum usage specifies fixed frequency bands and transmission power limits for each radio transmitting system. This approach leads to a very low medium utilization factor for some frequency bands, caused by inefficient service allocation over vast geographical areas (radiomobile, radio and TV broadcasting, WiMAX) and also by the usage of large guard bands, obsolete now due to technological progress. A more flexible use of the spectral resource implies that the radio transceivers have the ability to monitor their radio environment and to adapt at specific transmission conditions. If this concept is supplemented with learning and decision capabilities, we refer to the Cognitive Radio (CR) paradigm. Some of the characteristics of a CR include localization, monitoring of the spectrum usage, frequency changing, transmission power control and, finally, the capacity of dynamically altering all these parameters (Haykin, 2005). This new cognitive approach is expected to have an important impact on the future regulations and spectrum policies. The dynamic access at the spectral resource is of extreme interest both for the scientific community as, considering the continuous request for wideband services, for the development of wireless technologies. From this point of view, a fundamental role is played by the Institute of Electrical and Electronic Engineers (IEEE) which in 2007 formed the Standards Coordinating Committee (SCC) 41 on Dynamic Spectrum Access Networks (DySPAN) having as main objective a standard for dynamic access wireless networks. Still within the IEEE frame, the 802.22 initiative defines a new WRAN (Wireless Regional Area Network) interface for wideband access based on cognitive radio techniques in the TV guard bands (the so-called “white spaces”). Coupled with the advantages and flexibility of CR systems and technologies, there is an ever-growing interest around the world in exploiting CR-enabled communications in vehicular and transportation environments. The integration of CR devices and cognitive radio networks into vehicles and associated infrastructures can lead to intelligent interactions with the transportation system, among vehicles, and even among radios within vehicles. Thus, improvements can be achieved in radio resource management and energy efficiency, road traffic management, network management, vehicular diagnostics, road traffic awareness for applications such as route planning, mobile commerce, and much more. Still open within the framework of dynamic and distributed access to the radio resource are the methods for monitoring the radio environment (the so-called “spectrum sensing”) and the transceiver technology to be used on the radio channels. A CR system works on a opportunistic basis searching for unused frequency bands called “white spaces” within the radio frequency spectrum with the intent to operate invisibly and without disturbing the primary users (PU) holding a license for one or more frequency bands. Spectrum sensing, that is, the fast and reliable detection of the PU’s even in the presence of in-band noise, is still a very complex problem with a decisive impact on the functionalities and capabilities of the CRs. The spectrum sensing techniques can be classified in two types: local and cooperative (distributed). The local techniques are performed by single devices exploiting the spectrum occupancy information in their spatial neighbourhood and can be divided into three categories (Budiarjo et al., 2008): "matched filter" (detection of pilot signals, preambles, etc.), "energy detection” (signal strength analysis) and “feature detection" (classification of signals according to their characteristics). Also, a combination of local techniques in a multi-stage design can be used to improve the sensing accuracy (Maleki et al., 2010). Nevertheless, the above-mentioned techniques are mostly inefficient for signals with reduced power or affected by phenomena typical for vehicular technology applications, such as shadowing and multi-path fading. To overcome such problems, cooperatives techniques can be used. Cooperative sensing is based on the aggregation of the spectrum data detected by multiple nodes using cognitive convergence algorithms in order to avoid the channel impairment problems that can lead to false detections. (Sanna et al., 2009). Within the energy detection method, a particular attention needs to be paid to the properties of the packets wavelet transformation for subband analysis, which, according to the literature, seems to be a feasible alternative to the classical FFT-based energy detection. Vehicular applications are in most cases characterized by the need of coping with fast changes in the radio environment, which lead, in this specific case of cognitive communication, to constrains in terms of short execution time of the spectrum sensing operations. From this point of view, the computational complexity of the wavelet packets method is of the same order of the state-of-the-art FFT algorithms, but the number of mathematical operations is lower using IIR polyphase filters (Murroni et al., 2010). In our work we are investigating the use of the wavelet packets for energy detection spectrum sensing operations based on the consideration that they have a finite duration and are self- and mutually-orthogonal at integer multiples of dyadic intervals. Hence, they are suitable for subband division and analysis: a generic signal can be then decomposed on the wavelet packet basis and represented as a collection of coefficients belonging to orthogonal subbands. Therefore, the total power of the signal can be evaluated as sum of the contributions of each subband, which can be separately computed in the wavelet domain. Furthermore, the wavelet packets can be used also for the feature detection spectrum sensing, using statistical parameters such as moments and medians. We concentrate in our research on both applications of the wavelet packets to the spectrum sensing operations, investigating their efficiency in terms of reliability and execution time, applied specifically to the needs of vehicular technology and transportation environments. The other key issue for the development of the previously mentioned standard is the choice of an adaptive/multicarrier modulation as basic candidate for data transmission, having as the most known representative the Orthogonal Frequency Division Multiplexing (OFDM) modulation. OFDM-like schemes are mature enough to be chosen as a core technology for dynamic access wireless networks. At the same time, the potentialities in terms of optimization for this specific purpose are not yet thoroughly investigated. Particularly, the Wavelet Packet Division Multiplexing (WPDM) modulation method, already known for about ten years to the scientific community, is a suitable candidate to satisfy the requirements on physical level for a dynamic access network (Wong et al., 1997): WPDM has already proven to be able to overcome some of the OFDM limits (limited spectral efficiency, problems with temporal synchronization especially in channels affected by fading) and is at the same time based on use of the same wavelet packets employed for subband analysis used for spectrum sensing operations . Our research investigates the use of the WPDM for cognitive radio purposes, combined with the wavelet approach for spectrum sensing, for offering a complete, wavelet-based solution for cognitive application focused on the problematic of vehicular communication (channel impairments, high relative velocity of the communication peers etc.)

Learning-Based Approaches for Intelligent Cognitive Radio

Today with the growing demand for more data transmissions and increased network capacities, cognitive radio technology is ever more relevant. Traditional static spectrum allocation is no longer a feasible option. Through dynamic spectrum access, cognitive devices are able to tap into unused licensed spectrum bands. Thus, improving the spectrum utilization efficiency and fueling spectrum scarce applications. Cognitive Radio (CR) networks consist of smart radio devices that have the ability to sense and adapt to the rapidly changing radio environment. A cognitive device goes through a process of intelligent decision-making, which intrinsically shapes them into smart devices. Motivated by the superior performance of machine learning in various research paradigms, a cooperative Secondary Network (SN) is proposed that operates under a hybrid underlay-interweave access model. By taking advantage of both access models, the SN maximizes its throughput. A detection problem is formulated for each access model and Machine Learning (ML) techniques are applied to the SN, namely Gaussian Mixture Model (GMM), Support Vector Machine (SVM), and Naive Bayes' (NB) to classify the state of the channel. The multi-class SVM (MSVM) algorithm is reformulated and used to further classify the state of each primary user in the network. The performance of the hybrid network is evaluated based on the Receiver Operating Characteristics (ROC) and classification accuracy. In addition, we show that the accuracy of the MSVM is improved through the cooperation of the secondary users. Our results show that the proposed ML-based hybrid model is robust to low Signal-to-Noise Ratio (SNR) environments, and yields an improved performance compared with traditional cooperative sensing techniques. Moreover, we show that the Gaussian SVM surpasses other proposed learning algorithms achieving as high as an 80% detection rate with as low as 10% false alarm. Energy detection-based spectrum sensing, relies on measuring the energy level in the spectrum, and accordingly deciding the current occupancy state of the channel. Therefore, CR devices are required to determine the corresponding channel state given a measured energy level. CR networks that use supervised learning techniques to perform the sensing task require data examples of energy levels and the corresponding channel state for training purposes , i.e., labeled data. Having readily available labeled data is a complex task for CR networks, since it requires cooperation from both primary and secondary users. Such cooperation violates the ground rules for the interweave and underlay CR access models. Tackling the problem of labeled data scarcity in practical CR applications, we propose a two-stage learning framework for cooperative spectrum sensing. The algorithm combines the superior performance of the SVM algorithm and low cost training data of the GMM. Thus, rendering the two-stage learning framework suitable for practical CR applications. Finally, a system model is proposed and the performance of the system is evaluated based on the ROC for its upper and lower performance bounds. Additionally, our results show that the two-stage learning attains a higher detection performance compared with using the GMM algorithm

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig