An energy-efficient spectrum-aware reinforcement learning-based clustering algorithm for cognitive radio sensor networks

It is well-known that clustering partitions network into logical groups of nodes in order to achieve energy efficiency and to enhance dynamic channel access in cognitive radio through cooperative sensing. While the topic of energy efficiency has been well investigated in conventional wireless sensor networks, the latter has not been extensively explored. In this paper, we propose a reinforcement learning-based spectrum-aware clustering algorithm that allows a member node to learn the energy and cooperative sensing costs for neighboring clusters to achieve an optimal solution. Each member node selects an optimal cluster that satisfies pairwise constraints, minimizes network energy consumption and enhances channel sensing performance through an exploration technique. We first model the network energy consumption and then determine the optimal number of clusters for the network. The problem of selecting an optimal cluster is formulated as a Markov Decision Process (MDP) in the algorithm and the obtained simulation results show convergence, learning and adaptability of the algorithm to dynamic environment towards achieving an optimal solution. Performance comparisons of our algorithm with the Groupwise Spectrum Aware (GWSA)-based algorithm in terms of Sum of Square Error (SSE), complexity, network energy consumption and probability of detection indicate improved performance from the proposed approach. The results further reveal that an energy savings of 9% and a significant Primary User (PU) detection improvement can be achieved with the proposed approach

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Packet Scheduling Algorithms in LTE/LTE-A cellular Networks: Multi-agent Q-learning Approach

Spectrum utilization is vital for mobile operators. It ensures an efficient use of spectrum bands, especially when obtaining their license is highly expensive. Long Term Evolution (LTE), and LTE-Advanced (LTE-A) spectrum bands license were auctioned by the Federal Communication Commission (FCC) to mobile operators with hundreds of millions of dollars. In the first part of this dissertation, we study, analyze, and compare the QoS performance of QoS-aware/Channel-aware packet scheduling algorithms while using CA over LTE, and LTE-A heterogeneous cellular networks. This included a detailed study of the LTE/LTE-A cellular network and its features, and the modification of an open source LTE simulator in order to perform these QoS performance tests. In the second part of this dissertation, we aim to solve spectrum underutilization by proposing, implementing, and testing two novel multi-agent Q-learning-based packet scheduling algorithms for LTE cellular network. The Collaborative Competitive scheduling algorithm, and the Competitive Competitive scheduling algorithm. These algorithms schedule licensed users over the available radio resources and un-licensed users over spectrum holes. In conclusion, our results show that the spectrum band could be utilized by deploying efficient packet scheduling algorithms for licensed users, and can be further utilized by allowing unlicensed users to be scheduled on spectrum holes whenever they occur

Review on Radio Resource Allocation Optimization in LTE/LTE-Advanced using Game Theory

Recently, there has been a growing trend toward ap-plying game theory (GT) to various engineering fields in order to solve optimization problems with different competing entities/con-tributors/players. Researches in the fourth generation (4G) wireless network field also exploited this advanced theory to overcome long term evolution (LTE) challenges such as resource allocation, which is one of the most important research topics. In fact, an efficient de-sign of resource allocation schemes is the key to higher performance. However, the standard does not specify the optimization approach to execute the radio resource management and therefore it was left open for studies. This paper presents a survey of the existing game theory based solution for 4G-LTE radio resource allocation problem and its optimization