UAV-Assisted Space-Air-Ground Integrated Networks: A Technical Review of Recent Learning Algorithms

Recent technological advancements in space, air and ground components have made possible a new network paradigm called "space-air-ground integrated network" (SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs. However, due to UAVs' high dynamics and complexity, the real-world deployment of a SAGIN becomes a major barrier for realizing such SAGINs. Compared to the space and terrestrial components, UAVs are expected to meet performance requirements with high flexibility and dynamics using limited resources. Therefore, employing UAVs in various usage scenarios requires well-designed planning in algorithmic approaches. In this paper, we provide a comprehensive review of recent learning-based algorithmic approaches. We consider possible reward functions and discuss the state-of-the-art algorithms for optimizing the reward functions, including Q-learning, deep Q-learning, multi-armed bandit (MAB), particle swarm optimization (PSO) and satisfaction-based learning algorithms. Unlike other survey papers, we focus on the methodological perspective of the optimization problem, which can be applicable to various UAV-assisted missions on a SAGIN using these algorithms. We simulate users and environments according to real-world scenarios and compare the learning-based and PSO-based methods in terms of throughput, load, fairness, computation time, etc. We also implement and evaluate the 2-dimensional (2D) and 3-dimensional (3D) variations of these algorithms to reflect different deployment cases. Our simulation suggests that the $3$D satisfaction-based learning algorithm outperforms the other approaches for various metrics in most cases. We discuss some open challenges at the end and our findings aim to provide design guidelines for algorithm selections while optimizing the deployment of UAV-assisted SAGINs.Comment: Submitted to the IEEE Internet of Things Journal in June 202

Maximizing the latency fairness in UAV-assisted MEC system

Unmanned aerial vehicles (UAV) assisted edge computing has risen as an assuring technique to accommodate ubiquitous edge computation for resource-limited devices. Thus, this paper proposes an approach to maximize the latency fairness in a UAV-assisted multi-access edge computing (MEC) system. To maximize latency fairness, the authors focus on minimizing the maximum latency experienced among the users. In here, multiple ground users (GUs) offload their tasks to MEC UAV in the absence or unavailability of ground servers due to a disaster or heavy traffic where an iterative algorithm is proposed to minimize the maximum latency among the users subject to minimum control link rate and total power constraints. Sequentially, the UAVs' 3D location, offloading ratio, GUs' transmit power and GUs' computational capacity are optimized. The location of the UAV is optimized by using the novel approach, guided pattern search algorithm while the altitude of the UAV is optimized by analyzing the elevation angle dependant behaviour of the channel gain. A simple approach is utilized for optimizing the offloading ratio of the users by considering the problem as minimizing the point-wise maximum of two convex functions while the bisection method is used to optimize the power allocation. Numerical simulation results illustrate that the proposed approach outperforms other baseline approaches in convergence, minimizing the maximum latency and maximizing and maintaining the fairness among the GUs. Furthermore, it is proved that the guided pattern search algorithm converges at least 3.5 times better while the proposed combined optimization gives 400% fairness gain, in comparison with the baseline approach

Architecture design for disaster resilient management network using D2D technology

Huge damages from natural disasters, such as earthquakes, floods, landslide, tsunamis, have been reported in recent years, claiming many lives, rendering millions homeless and causing huge financial losses worldwide. The lack of effective communication between the public rescue/safety agencies, rescue teams, first responders and trapped survivors/victims makes the situation even worse. Factors like dysfunctional communication networks, limited communications capacity, limited resources/services, data transformation and effective evaluation, energy, and power deficiency cause unnecessary hindrance in rescue and recovery services during a disaster. The new wireless communication technologies are needed to enhance life-saving capabilities and rescue services. In general, in order to improve societal resilience towards natural catastrophes and develop effective communication infrastructure, innovative approaches need to be initiated to provide improved quality, better connectivity in the events of natural and human disasters. In this thesis, a disaster resilient network architecture is proposed and analysed using multi-hop communications, clustering, energy harvesting, throughput optimization, reliability enhancement, adaptive selection, and low latency communications. It also examines the importance of mode selection, power management, frequency and time resource allocation to realize the promises of Long-term Evolution (LTE) Device to Device (D2D) communication. In particular, to support resilient and energy efficient communication in disaster-affected areas. This research is examined by thorough and vigorous simulations and validated through mathematical modelling. Overall, the impact of this research is twofold: i) it provides new technologies for effective inter- and intra-agency coordination system during a disaster event by establishing a stronger and resilient communication; and ii) It offers a potential solution for stakeholders such as governments, rescue teams, and general public with new informed information on how to establish effective policies to cope with challenges before, during and after the disaster events

Performance Analysis and Learning Algorithms in Advanced Wireless Networks

Over the past decade, wireless data traffic has experienced an exponential growth, especially with multimedia traffic becoming the dominant traffic, and such growth is expected to continue in the near future. This unprecedented growth has led to an increasing demand for high-rate wireless communications.Key solutions for addressing such demand include extreme network densification with more small-cells, the utilization of high frequency bands, such as the millimeter wave (mmWave) bands and terahertz (THz) bands, where more bandwidth is available, and unmanned aerial vehicle (UAV)-enabled cellular networks. With this motivation, different types of advanced wireless networks are considered in this thesis. In particular, mmWave cellular networks, networks with hybrid THz, mmWave and microwave transmissions, and UAV-enabled networks are studied, and performance metrics such as the signal-to-interference-plus-noise ratio (SINR) coverage, energy coverage, and area spectral efficiency are analyzed. In addition, UAV path planning in cellular networks are investigated, and deep reinforcement learning (DRL) based algorithms are proposed to find collision-free UAV trajectory to accomplish different missions. In the first part of this thesis, mmWave cellular networks are considered. First, K-tier heterogeneous mmWave cellular networks with user-centric small-cell deployments are studied. Particularly, a heterogeneous network model with user equipments (UEs) being distributed according to Poisson cluster processes (PCPs) is considered. Distinguishing features of mmWave communications including directional beamforming and a detailed path loss model are taken into account. General expressions for the association probabilities of different tier base stations (BSs) are determined. Using tools from stochastic geometry, the Laplace transform of the interference is characterized and general expressions for the SINR coverage probability and area spectral efficiency are derived. Second, a distributed multi-agent learning-based algorithm for beamforming in mmWave multiple input multiple output (MIMO) networks is proposed to maximize the sum-rate of all UEs. Following the analysis of mmWave cellular networks, a three-tier heterogeneous network is considered, where access points (APs), small-cell BSs (SBSs) and macrocell BSs (MBSs) transmit in THz, mmWave, microwave frequency bands, respectively. By using tools from stochastic geometry, the complementary cumulative distribution function (CCDF) of the received signal power, the Laplace transform of the aggregate interference, and the SINR coverage probability are determined. Next, system-level performance of UAV-enabled cellular networks is studied. More specifically, in the first part, UAV-assisted mmWave cellular networks are addressed, in which the UE locations are modeled using PCPs. In the downlink phase, simultaneous wireless information and power transfer (SWIPT) technique is considered. The association probability, energy coverages and a successful transmission probability to jointly determine the energy and SINR coverages are derived. In the uplink phase, a scenario that each UAV receives information from its own cluster member UEs is taken into account. The Laplace transform of the interference components and the uplink SINR coverage are characterized. In the second part, cellular-connected UAV networks is investigated, in which the UAVs are aerial UEs served by the ground base stations (GBSs). 3D antenna radiation combing the vertical and horizontal patterns is taken into account. In the final part of this thesis, deep reinforcement learning based algorithms are proposed for UAV path planning in cellular networks. Particularly, in the first part, multi-UAV non-cooperative scenarios is considered, where multiple UAVs need to fly from initial locations to destinations, while satisfying collision avoidance, wireless connectivity and kinematic constraints. The goal is to find trajectories for the cellular-connected UAVs to minimize their mission completion time. The multi-UAV trajectory optimization problem is formulated as a sequential decision making problem, and a decentralized DRL approach is proposed to solve the problem. Moreover, multiple UAV trajectory design in cellular networks with a dynamic jammer is studied, and a learning-based algorithm is proposed. Subsequently, a UAV trajectory optimization problem is considered to maximize the collected data from multiple Internet of things (IoT) nodes under realistic constraints. The problem is translated into a Markov decision process (MDP) and dueling double deep Q-network (D3QN) is proposed to learn the decision making policy

Space-Air-Ground Integrated 6G Wireless Communication Networks: A Review of Antenna Technologies and Application Scenarios

A review of technological solutions and advances in the framework of a Vertical Heterogeneous Network (VHetNet) integrating satellite, airborne and terrestrial networks is presented. The disruptive features and challenges offered by a fruitful cooperation among these segments within a ubiquitous and seamless wireless connectivity are described. The available technologies and the key research directions for achieving global wireless coverage by considering all these layers are thoroughly discussed. Emphasis is placed on the available antenna systems in satellite, airborne and ground layers by highlighting strengths and weakness and by providing some interesting trends in research. A summary of the most suitable applicative scenarios for future 6G wireless communications are finally illustrated

Machine Learning for Unmanned Aerial System (UAS) Networking

Fueled by the advancement of 5G new radio (5G NR), rapid development has occurred in many fields. Compared with the conventional approaches, beamforming and network slicing enable 5G NR to have ten times decrease in latency, connection density, and experienced throughput than 4G long term evolution (4G LTE). These advantages pave the way for the evolution of Cyber-physical Systems (CPS) on a large scale. The reduction of consumption, the advancement of control engineering, and the simplification of Unmanned Aircraft System (UAS) enable the UAS networking deployment on a large scale to become feasible. The UAS networking can finish multiple complex missions simultaneously. However, the limitations of the conventional approaches are still a big challenge to make a trade-off between the massive management and efficient networking on a large scale. With 5G NR and machine learning, in this dissertation, my contributions can be summarized as the following: I proposed a novel Optimized Ad-hoc On-demand Distance Vector (OAODV) routing protocol to improve the throughput of Intra UAS networking. The novel routing protocol can reduce the system overhead and be efficient. To improve the security, I proposed a blockchain scheme to mitigate the malicious basestations for cellular connected UAS networking and a proof-of-traffic (PoT) to improve the efficiency of blockchain for UAS networking on a large scale. Inspired by the biological cell paradigm, I proposed the cell wall routing protocols for heterogeneous UAS networking. With 5G NR, the inter connections between UAS networking can strengthen the throughput and elasticity of UAS networking. With machine learning, the routing schedulings for intra- and inter- UAS networking can enhance the throughput of UAS networking on a large scale. The inter UAS networking can achieve the max-min throughput globally edge coloring. I leveraged the upper and lower bound to accelerate the optimization of edge coloring. This dissertation paves a way regarding UAS networking in the integration of CPS and machine learning. The UAS networking can achieve outstanding performance in a decentralized architecture. Concurrently, this dissertation gives insights into UAS networking on a large scale. These are fundamental to integrating UAS and National Aerial System (NAS), critical to aviation in the operated and unmanned fields. The dissertation provides novel approaches for the promotion of UAS networking on a large scale. The proposed approaches extend the state-of-the-art of UAS networking in a decentralized architecture. All the alterations can contribute to the establishment of UAS networking with CPS