Quantum Reinforcement Learning for Dynamic Spectrum Access in Cognitive Radio Networks

Abstract This thesis proposes Quantum Reinforcement Learning (QRL) as an improvement to conventional reinforcement learning-based dynamic spectrum access used within cognitive radio networks. The aim is to overcome the slow convergence problem associated with exploration within reinforcement learning schemes. A literature review for the background of the carried out research work is illustrated. Review of research works on learning-based assignment techniques as well as quantum search techniques is provided. Modelling of three traditional dynamic channel assignment techniques is illustrated and the advantage characteristic of each technique is discussed. These techniques have been simulated to provide a comparison with learning based techniques, including QRL. Reinforcement learning techniques are used as a direct comparison with the Quantum Reinforcement Learning approaches. The elements of Quantum computation are then presented as an introduction to quantum search techniques. The Grover search algorithm is introduced. The algorithm is discussed from a theoretical perspective. The Grover algorithm is then used for the first time as a spectrum allocation scheme and compared to conventional schemes. Quantum Reinforcement Learning (QRL) is introduced as a natural evolution of the quantum search. The Grover search algorithm is combined as a decision making mechanism with conventional Reinforcement Learning (RL) algorithms resulting in a more efficient learning engine. Simulation results are provided and discussed. The convergence speed has been significantly increased. The beneficial effects of Quantum Reinforcement Learning (QRL) become more pronounced as the traffic load increases. The thesis shows that both system performance and capacity can be improved. Depending on the traffic load, the system capacity has improved by 9-84% from a number of users supported perspective. It also demonstrated file delay reduction for up to an average of 26% and 2.8% throughput improvement

Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research

Emerging applications such as Internet of Everything, Holographic Telepresence, collaborative robots, and space and deep-sea tourism are already highlighting the limitations of existing fifth-generation (5G) mobile networks. These limitations are in terms of data-rate, latency, reliability, availability, processing, connection density and global coverage, spanning over ground, underwater and space. The sixth-generation (6G) of mobile networks are expected to burgeon in the coming decade to address these limitations. The development of 6G vision, applications, technologies and standards has already become a popular research theme in academia and the industry. In this paper, we provide a comprehensive survey of the current developments towards 6G. We highlight the societal and technological trends that initiate the drive towards 6G. Emerging applications to realize the demands raised by 6G driving trends are discussed subsequently. We also elaborate the requirements that are necessary to realize the 6G applications. Then we present the key enabling technologies in detail. We also outline current research projects and activities including standardization efforts towards the development of 6G. Finally, we summarize lessons learned from state-of-the-art research and discuss technical challenges that would shed a new light on future research directions towards 6G

6G Enabled Smart Infrastructure for Sustainable Society: Opportunities, Challenges, and Research Roadmap

The 5G wireless communication network is currently faced with the challenge of limited data speed exacerbated by the proliferation of billions of data-intensive applications. To address this problem, researchers are developing cutting-edge technologies for the envisioned 6G wireless communication standards to satisfy the escalating wireless services demands. Though some of the candidate technologies in the 5G standards will apply to 6G wireless networks, key disruptive technologies that will guarantee the desired quality of physical experience to achieve ubiquitous wireless connectivity are expected in 6G. This article first provides a foundational background on the evolution of different wireless communication standards to have a proper insight into the vision and requirements of 6G. Second, we provide a panoramic view of the enabling technologies proposed to facilitate 6G and introduce emerging 6G applications such as multi-sensory–extended reality, digital replica, and more. Next, the technology-driven challenges, social, psychological, health and commercialization issues posed to actualizing 6G, and the probable solutions to tackle these challenges are discussed extensively. Additionally, we present new use cases of the 6G technology in agriculture, education, media and entertainment, logistics and transportation, and tourism. Furthermore, we discuss the multi-faceted communication capabilities of 6G that will contribute significantly to global sustainability and how 6G will bring about a dramatic change in the business arena. Finally, we highlight the research trends, open research issues, and key take-away lessons for future research exploration in 6G wireless communicatio

A Decade of Research in Fog computing: Relevance, Challenges, and Future Directions

Recent developments in the Internet of Things (IoT) and real-time applications, have led to the unprecedented growth in the connected devices and their generated data. Traditionally, this sensor data is transferred and processed at the cloud, and the control signals are sent back to the relevant actuators, as part of the IoT applications. This cloud-centric IoT model, resulted in increased latencies and network load, and compromised privacy. To address these problems, Fog Computing was coined by Cisco in 2012, a decade ago, which utilizes proximal computational resources for processing the sensor data. Ever since its proposal, fog computing has attracted significant attention and the research fraternity focused at addressing different challenges such as fog frameworks, simulators, resource management, placement strategies, quality of service aspects, fog economics etc. However, after a decade of research, we still do not see large-scale deployments of public/private fog networks, which can be utilized in realizing interesting IoT applications. In the literature, we only see pilot case studies and small-scale testbeds, and utilization of simulators for demonstrating scale of the specified models addressing the respective technical challenges. There are several reasons for this, and most importantly, fog computing did not present a clear business case for the companies and participating individuals yet. This paper summarizes the technical, non-functional and economic challenges, which have been posing hurdles in adopting fog computing, by consolidating them across different clusters. The paper also summarizes the relevant academic and industrial contributions in addressing these challenges and provides future research directions in realizing real-time fog computing applications, also considering the emerging trends such as federated learning and quantum computing.Comment: Accepted for publication at Wiley Software: Practice and Experience journa

Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future

The upcoming 5th Generation (5G) of wireless networks is expected to lay a foundation of intelligent networks with the provision of some isolated Artificial Intelligence (AI) operations. However, fully-intelligent network orchestration and management for providing innovative services will only be realized in Beyond 5G (B5G) networks. To this end, we envisage that the 6th Generation (6G) of wireless networks will be driven by on-demand self-reconfiguration to ensure a many-fold increase in the network performanceandservicetypes.Theincreasinglystringentperformancerequirementsofemergingnetworks may finally trigger the deployment of some interesting new technologies such as large intelligent surfaces, electromagnetic-orbital angular momentum, visible light communications and cell-free communications – tonameafew.Ourvisionfor6Gis–amassivelyconnectedcomplexnetworkcapableofrapidlyresponding to the users’ service calls through real-time learning of the network state as described by the network-edge (e.g., base-station locations, cache contents, etc.), air interface (e.g., radio spectrum, propagation channel, etc.), and the user-side (e.g., battery-life, locations, etc.). The multi-state, multi-dimensional nature of the network state, requiring real-time knowledge, can be viewed as a quantum uncertainty problem. In this regard, the emerging paradigms of Machine Learning (ML), Quantum Computing (QC), and Quantum ML (QML) and their synergies with communication networks can be considered as core 6G enablers. Considering these potentials, starting with the 5G target services and enabling technologies, we provide a comprehensivereviewoftherelatedstate-of-the-artinthedomainsofML(includingdeeplearning),QCand QML, and identify their potential benefits, issues and use cases for their applications in the B5G networks. Subsequently,weproposeanovelQC-assistedandQML-basedframeworkfor6Gcommunicationnetworks whilearticulatingitschallengesandpotentialenablingtechnologiesatthenetwork-infrastructure,networkedge, air interface and user-end. Finally, some promising future research directions for the quantum- and QML-assisted B5G networks are identified and discussed

Intelligent Reflective Surface Deployment in 6G: A Comprehensive Survey

Intelligent reflecting surfaces (IRSs) are considered a promising technology that can smartly reconfigure the wireless environment to enhance the performance of future wireless networks. However, the deployment of IRSs still faces challenges due to highly dynamic and mobile unmanned aerial vehicle (UAV) enabled wireless environments to achieve higher capacity. This paper sheds light on the different deployment strategies for IRSs in future terrestrial and non-terrestrial networks. Specifically, in this paper, we introduce key theoretical concepts underlying the IRS paradigm and discuss the design aspects related to the deployment of IRSs in 6G networks. We also explore optimization-based IRS deployment techniques to improve system performance in terrestrial and aerial IRSs. Furthermore, we survey model-free reinforcement learning (RL) techniques from the deployment aspect to address the challenges of achieving higher capacity in complex and mobile IRS-assisted UAV wireless systems. Finally, we highlight challenges and future research directions from the deployment aspect of IRSs for improving system performance for the future 6G network.Comment: 16 pages, 3 Figures, 7 table