UAV-Assisted Space-Air-Ground Integrated Networks: A Technical Review of Recent Learning Algorithms

Recent technological advancements in space, air and ground components have made possible a new network paradigm called "space-air-ground integrated network" (SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs. However, due to UAVs' high dynamics and complexity, the real-world deployment of a SAGIN becomes a major barrier for realizing such SAGINs. Compared to the space and terrestrial components, UAVs are expected to meet performance requirements with high flexibility and dynamics using limited resources. Therefore, employing UAVs in various usage scenarios requires well-designed planning in algorithmic approaches. In this paper, we provide a comprehensive review of recent learning-based algorithmic approaches. We consider possible reward functions and discuss the state-of-the-art algorithms for optimizing the reward functions, including Q-learning, deep Q-learning, multi-armed bandit (MAB), particle swarm optimization (PSO) and satisfaction-based learning algorithms. Unlike other survey papers, we focus on the methodological perspective of the optimization problem, which can be applicable to various UAV-assisted missions on a SAGIN using these algorithms. We simulate users and environments according to real-world scenarios and compare the learning-based and PSO-based methods in terms of throughput, load, fairness, computation time, etc. We also implement and evaluate the 2-dimensional (2D) and 3-dimensional (3D) variations of these algorithms to reflect different deployment cases. Our simulation suggests that the $3$D satisfaction-based learning algorithm outperforms the other approaches for various metrics in most cases. We discuss some open challenges at the end and our findings aim to provide design guidelines for algorithm selections while optimizing the deployment of UAV-assisted SAGINs.Comment: Submitted to the IEEE Internet of Things Journal in June 202

Aerial base station placement in temporary-event scenarios

Die Anforderungen an den Netzdatenverkehr sind in den letzten Jahren dramatisch gestiegen, was ein großes Interesse an der Entwicklung neuartiger Lösungen zur Erhöhung der Netzkapazität in Mobilfunknetzen erzeugt hat. Besonderes Augenmerk wurde auf das Problem der Kapazitätsverbesserung bei temporären Veranstaltungen gelegt, bei denen das Umfeld im Wesentlichen dynamisch ist. Um der Dynamik der sich verändernden Umgebung gerecht zu werden und die Bodeninfrastruktur durch zusätzliche Kapazität zu unterstützen, wurde der Einsatz von Luftbasisstationen vorgeschlagen. Die Luftbasisstationen können in der Nähe des Nutzers platziert werden und aufgrund der im Vergleich zur Bodeninfrastruktur höheren Lage die Vorteile der Sichtlinienkommunikation nutzen. Dies reduziert den Pfadverlust und ermöglicht eine höhere Kanalkapazität. Das Optimierungsproblem der Maximierung der Netzkapazität durch die richtige Platzierung von Luftbasisstationen bildet einen Schwerpunkt der Arbeit. Es ist notwendig, das Optimierungsproblem rechtzeitig zu lösen, um auf Veränderungen in der dynamischen Funkumgebung zu reagieren. Die optimale Platzierung von Luftbasisstationen stellt jedoch ein NP-schweres Problem dar, wodurch die Lösung nicht trivial ist. Daher besteht ein Bedarf an schnellen und skalierbaren Optimierungsalgorithmen. Als Erstes wird ein neuartiger Hybrid-Algorithmus (Projected Clustering) vorgeschlagen, der mehrere Lösungen auf der Grundlage der schnellen entfernungsbasierten Kapazitätsapproximierung berechnet und sie auf dem genauen SINR-basierten Kapazitätsmodell bewertet. Dabei werden suboptimale Lösungen vermieden. Als Zweites wird ein neuartiges verteiltes, selbstorganisiertes Framework (AIDA) vorgeschlagen, welches nur lokales Wissen verwendet, den Netzwerkmehraufwand verringert und die Anforderungen an die Kommunikation zwischen Luftbasisstationen lockert. Bei der Formulierung des Platzierungsproblems konnte festgestellt werden, dass Unsicherheiten in Bezug auf die Modellierung der Luft-Bodensignalausbreitung bestehen. Da dieser Aspekt im Rahmen der Analyse eine wichtige Rolle spielt, erfolgte eine Validierung moderner Luft-Bodensignalausbreitungsmodelle, indem reale Messungen gesammelt und das genaueste Modell für die Simulationen ausgewählt wurden.As the traffic demands have grown dramatically in recent years, so has the interest in developing novel solutions that increase the network capacity in cellular networks. The problem of capacity improvement is even more complex when applied to a dynamic environment during a disaster or temporary event. The use of aerial base stations has received much attention in the last ten years as the solution to cope with the dynamics of the changing environment and to supplement the ground infrastructure with extra capacity. Due to higher elevations and possibility to place aerial base stations in close proximity to the user, path loss is significantly smaller in comparison to the ground infrastructure, which in turn enables high data capacity. We are studying the optimization problem of maximizing network capacity by proper placement of aerial base stations. To handle the changes in the dynamic radio environment, it is necessary to promptly solve the optimization problem. However, we show that the optimal placement of aerial base stations is the NP-hard problem and its solution is non-trivial, and thus, there is a need for fast and scalable optimization algorithms. This dissertation investigates how to solve the placement problem efficiently and to support the dynamics of temporary events. First, we propose a novel hybrid algorithm (Projected Clustering), which calculates multiple solutions based on the fast distance-based capacity approximation and evaluates them on the accurate SINR-based capacity model, avoiding sub-optimal solutions. Second, we propose a novel distributed, self-organized framework (AIDA), which conducts a decision-making process using only local knowledge, decreasing the network overhead and relaxing the requirements for communication between aerial base stations. During the formulation of the placement problem, we found that there is still considerable uncertainty with regard to air-to-ground propagation modeling. Since this aspect plays an important role in our analysis, we validated state-of-the-art air-to-ground propagation models by collecting real measurements and chose the most accurate model for the simulations

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Wireless communication networks have been witnessing an unprecedented demand due to the increasing number of connected devices and emerging bandwidth-hungry applications. Albeit many competent technologies for capacity enhancement purposes, such as millimeter wave communications and network densification, there is still room and need for further capacity enhancement in wireless communication networks, especially for the cases of unusual people gatherings, such as sport competitions, musical concerts, etc. Unmanned aerial vehicles (UAVs) have been identified as one of the promising options to enhance the capacity due to their easy implementation, pop up fashion operation, and cost-effective nature. The main idea is to deploy base stations on UAVs and operate them as flying base stations, thereby bringing additional capacity to where it is needed. However, because the UAVs mostly have limited energy storage, their energy consumption must be optimized to increase flight time. In this survey, we investigate different energy optimization techniques with a top-level classification in terms of the optimization algorithm employed; conventional and machine learning (ML). Such classification helps understand the state of the art and the current trend in terms of methodology. In this regard, various optimization techniques are identified from the related literature, and they are presented under the above mentioned classes of employed optimization methods. In addition, for the purpose of completeness, we include a brief tutorial on the optimization methods and power supply and charging mechanisms of UAVs. Moreover, novel concepts, such as reflective intelligent surfaces and landing spot optimization, are also covered to capture the latest trend in the literature.Comment: 41 pages, 5 Figures, 6 Tables. Submitted to Open Journal of Communications Society (OJ-COMS

Unmanned Aerial Vehicle (UAV)-Enabled Wireless Communications and Networking

The emerging massive density of human-held and machine-type nodes implies larger traffic deviatiolns in the future than we are facing today. In the future, the network will be characterized by a high degree of flexibility, allowing it to adapt smoothly, autonomously, and efficiently to the quickly changing traffic demands both in time and space. This flexibility cannot be achieved when the network’s infrastructure remains static. To this end, the topic of UAVs (unmanned aerial vehicles) have enabled wireless communications, and networking has received increased attention. As mentioned above, the network must serve a massive density of nodes that can be either human-held (user devices) or machine-type nodes (sensors). If we wish to properly serve these nodes and optimize their data, a proper wireless connection is fundamental. This can be achieved by using UAV-enabled communication and networks. This Special Issue addresses the many existing issues that still exist to allow UAV-enabled wireless communications and networking to be properly rolled out

Multiple Access in Aerial Networks: From Orthogonal and Non-Orthogonal to Rate-Splitting

Recently, interest on the utilization of unmanned aerial vehicles (UAVs) has aroused. Specifically, UAVs can be used in cellular networks as aerial users for delivery, surveillance, rescue search, or as an aerial base station (aBS) for communication with ground users in remote uncovered areas or in dense environments requiring prompt high capacity. Aiming to satisfy the high requirements of wireless aerial networks, several multiple access techniques have been investigated. In particular, space-division multiple access(SDMA) and power-domain non-orthogonal multiple access (NOMA) present promising multiplexing gains for aerial downlink and uplink. Nevertheless, these gains are limited as they depend on the conditions of the environment. Hence, a generalized scheme has been recently proposed, called rate-splitting multiple access (RSMA), which is capable of achieving better spectral efficiency gains compared to SDMA and NOMA. In this paper, we present a comprehensive survey of key multiple access technologies adopted for aerial networks, where aBSs are deployed to serve ground users. Since there have been only sporadic results reported on the use of RSMA in aerial systems, we aim to extend the discussion on this topic by modelling and analyzing the weighted sum-rate performance of a two-user downlink network served by an RSMA-based aBS. Finally, related open issues and future research directions are exposed.Comment: 16 pages, 6 figures, submitted to IEEE Journa

Intelligent Reflective Surface Deployment in 6G: A Comprehensive Survey

Intelligent reflecting surfaces (IRSs) are considered a promising technology that can smartly reconfigure the wireless environment to enhance the performance of future wireless networks. However, the deployment of IRSs still faces challenges due to highly dynamic and mobile unmanned aerial vehicle (UAV) enabled wireless environments to achieve higher capacity. This paper sheds light on the different deployment strategies for IRSs in future terrestrial and non-terrestrial networks. Specifically, in this paper, we introduce key theoretical concepts underlying the IRS paradigm and discuss the design aspects related to the deployment of IRSs in 6G networks. We also explore optimization-based IRS deployment techniques to improve system performance in terrestrial and aerial IRSs. Furthermore, we survey model-free reinforcement learning (RL) techniques from the deployment aspect to address the challenges of achieving higher capacity in complex and mobile IRS-assisted UAV wireless systems. Finally, we highlight challenges and future research directions from the deployment aspect of IRSs for improving system performance for the future 6G network.Comment: 16 pages, 3 Figures, 7 table

A Survey on Applications of Cache-Aided NOMA

Contrary to orthogonal multiple-access (OMA), non-orthogonal multiple-access (NOMA) schemes can serve a pool of users without exploiting the scarce frequency or time domain resources. This is useful in meeting the future network requirements (5G and beyond systems), such as, low latency, massive connectivity, users' fairness, and high spectral efficiency. On the other hand, content caching restricts duplicate data transmission by storing popular contents in advance at the network edge which reduces data traffic. In this survey, we focus on cache-aided NOMA-based wireless networks which can reap the benefits of both cache and NOMA; switching to NOMA from OMA enables cache-aided networks to push additional files to content servers in parallel and improve the cache hit probability. Beginning with fundamentals of the cache-aided NOMA technology, we summarize the performance goals of cache-aided NOMA systems, present the associated design challenges, and categorize the recent related literature based on their application verticals. Concomitant standardization activities and open research challenges are highlighted as well

Multi-Drone-Cell 3D Trajectory Planning and Resource Allocation for Drone-Assisted Radio Access Networks

Equipped with communication modules, drones can perform as drone-cells (DCs) that provide on-demand communication services to users in various scenarios, such as traffic monitoring, Internet of things (IoT) data collections, and temporal communication provisioning. As the aerial relay nodes between terrestrial users and base stations (BSs), DCs are leveraged to extend wireless connections for uncovered users of radio access networks (RAN), which forms the drone-assisted RAN (DA-RAN). In DA-RAN, the communication coverage, quality-of-service (QoS) performance and deployment flexibility can be improved due to the line-of-sight DC-to-ground (D2G) wireless links and the dynamic deployment capabilities of DCs. Considering the special mobility pattern, channel model, energy consumption, and other features of DCs, it is essential yet challenging to design the flying trajectories and resource allocation schemes for DA-RAN. In specific, given the emerging D2G communication models and dynamic deployment capability of DCs, new DC deployment strategies are required by DA-RAN. Moreover, to exploit the fully controlled mobility of DCs and promote the user fairness, the flying trajectories of DCs and the D2G communications must be jointly optimized. Further, to serve the high-mobility users (e.g. vehicular users) whose mobility patterns are hard to be modeled, both the trajectory planning and resource allocation schemes for DA-RAN should be re-designed to adapt to the variations of terrestrial traffic. To address the above challenges, in this thesis, we propose a DA-RAN architecture in which multiple DCs are leveraged to relay data between BSs and terrestrial users. Based on the theoretical analyses of the D2G communication, DC energy consumption, and DC mobility features, the deployment, trajectory planning and communication resource allocation of multiple DCs are jointly investigated for both quasi-static and high-mobility users. We first analyze the communication coverage, drone-to-BS (D2B) backhaul link quality, and optimal flying height of the DC according to the state-of-the-art drone-to-user (D2U) and D2B channel models. We then formulate the multi-DC three-dimensional (3D) deployment problem with the objective of maximizing the ratio of effectively covered users while guaranteeing D2B link qualities. To solve the problem, a per-drone iterated particle swarm optimization (DI-PSO) algorithm is proposed, which prevents the large particle searching space and the high violating probability of constraints existing in the pure PSO based algorithm. Simulations show that the DI-PSO algorithm can achieve higher coverage ratio with less complexity comparing to the pure PSO based algorithm. Secondly, to improve overall network performance and the fairness among edge and central users, we design 3D trajectories for multiple DCs in DA-RAN. The multi-DC 3D trajectory planning and scheduling is formulated as a mixed integer non-linear programming (MINLP) problem with the objective of maximizing the average D2U throughput. To address the non-convexity and NP-hardness of the MINLP problem due to the 3D trajectory, we first decouple the MINLP problem into multiple integer linear programming and quasi-convex sub-problems in which user association, D2U communication scheduling, horizontal trajectories and flying heights of DBSs are respectively optimized. Then, we design a multi-DC 3D trajectory planning and scheduling algorithm to solve the sub-problems iteratively based on the block coordinate descent (BCD) method. A k-means-based initial trajectory generation scheme and a search-based start slot scheduling scheme are also designed to improve network performance and control mutual interference between DCs, respectively. Compared with the static DBS deployment, the proposed trajectory planning scheme can achieve much lower average value and standard deviation of D2U pathloss, which indicate the improvements of network throughput and user fairness. Thirdly, considering the highly dynamic and uncertain environment composed by high-mobility users, we propose a hierarchical deep reinforcement learning (DRL) based multi-DC trajectory planning and resource allocation (HDRLTPRA) scheme for high-mobility users. The objective is to maximize the accumulative network throughput while satisfying user fairness, DC power consumption, and DC-to-ground link quality constraints. To address the high uncertainties of environment, we decouple the multi-DC TPRA problem into two hierarchical sub-problems, i.e., the higher-level global trajectory planning sub-problem and the lower-level local TPRA sub-problem. First, the global trajectory planning sub-problem is to address trajectory planning for multiple DCs in the RAN over a long time period. To solve the sub-problem, we propose a multi-agent DRL based global trajectory planning (MARL-GTP) algorithm in which the non-stationary state space caused by multi-DC environment is addressed by the multi-agent fingerprint technique. Second, based on the global trajectory planning results, the local TPRA (LTPRA) sub-problem is investigated independently for each DC to control the movement and transmit power allocation based on the real-time user traffic variations. A deep deterministic policy gradient based LTPRA (DDPG-LTPRA) algorithm is then proposed to solve the LTPRA sub-problem. With the two algorithms addressing both sub-problems at different decision granularities, the multi-DC TPRA problem can be resolved by the HDRLTPRA scheme. Simulation results show that 40% network throughput improvement can be achieved by the proposed HDRLTPRA scheme over the non-learning-based TPRA scheme. In summary, we have investigated the multi-DC 3D deployment, trajectory planning and communication resource allocation in DA-RAN considering different user mobility patterns in this thesis. The proposed schemes and theoretical results should provide useful guidelines for future research in DC trajectory planning, resource allocation, as well as the real deployment of DCs in complex environments with diversified users

Fairness-Driven Optimization of RIS-Augmented 5G Networks for Seamless 3D UAV Connectivity Using DRL Algorithms

In this paper, we study the problem of joint active and passive beamforming for reconfigurable intelligent surface (RIS)-assisted massive multiple-input multiple-output systems towards the extension of the wireless cellular coverage in 3D, where multiple RISs, each equipped with an array of passive elements, are deployed to assist a base station (BS) to simultaneously serve multiple unmanned aerial vehicles (UAVs) in the same time-frequency resource of 5G wireless communications. With a focus on ensuring fairness among UAVs, our objective is to maximize the minimum signal-to-interference-plus-noise ratio (SINR) at UAVs by jointly optimizing the transmit beamforming parameters at the BS and phase shift parameters at RISs. We propose two novel algorithms to address this problem. The first algorithm aims to mitigate interference by calculating the BS beamforming matrix through matrix inverse operations once the phase shift parameters are determined. The second one is based on the principle that one RIS element only serves one UAV and the phase shift parameter of this RIS element is optimally designed to compensate the phase offset caused by the propagation and fading. To obtain the optimal parameters, we utilize one state-of-the-art reinforcement learning algorithm, deep deterministic policy gradient, to solve these two optimization problems. Simulation results are provided to illustrate the effectiveness of our proposed solution and some insightful remarks are observed