UAV-Assisted Space-Air-Ground Integrated Networks: A Technical Review of Recent Learning Algorithms

Recent technological advancements in space, air and ground components have made possible a new network paradigm called "space-air-ground integrated network" (SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs. However, due to UAVs' high dynamics and complexity, the real-world deployment of a SAGIN becomes a major barrier for realizing such SAGINs. Compared to the space and terrestrial components, UAVs are expected to meet performance requirements with high flexibility and dynamics using limited resources. Therefore, employing UAVs in various usage scenarios requires well-designed planning in algorithmic approaches. In this paper, we provide a comprehensive review of recent learning-based algorithmic approaches. We consider possible reward functions and discuss the state-of-the-art algorithms for optimizing the reward functions, including Q-learning, deep Q-learning, multi-armed bandit (MAB), particle swarm optimization (PSO) and satisfaction-based learning algorithms. Unlike other survey papers, we focus on the methodological perspective of the optimization problem, which can be applicable to various UAV-assisted missions on a SAGIN using these algorithms. We simulate users and environments according to real-world scenarios and compare the learning-based and PSO-based methods in terms of throughput, load, fairness, computation time, etc. We also implement and evaluate the 2-dimensional (2D) and 3-dimensional (3D) variations of these algorithms to reflect different deployment cases. Our simulation suggests that the $3$D satisfaction-based learning algorithm outperforms the other approaches for various metrics in most cases. We discuss some open challenges at the end and our findings aim to provide design guidelines for algorithm selections while optimizing the deployment of UAV-assisted SAGINs.Comment: Submitted to the IEEE Internet of Things Journal in June 202

Distributed task offloading optimization with queueing dynamics in multi-agent mobile-edge computing networks

Task offloading decision-making plays a key role in enabling mobile-edge computing (MEC) technologies in Internet-of-Things (IoT). However, it meets the significant challenges arising from the stochastic dynamics of task queueing in the application layer and coupled wireless interference in the physical layer in a distributed multi-agent network without any centralized communication and computing coordination. In this paper, we investigate the distributed task offloading optimization problem with consideration of the upper-layer queueing dynamics and the lower-layer coupled wireless interference. We first propose a new optimization model that aims at maximizing the expected offloading rate of multiple agents by optimizing their offloading thresholds. Then, we transform the problem into a game-theoretic formulation, which further leads to the design of a distributed best-response (DBR) iterative optimization framework. The existence of Nash equilibrium strategies in the game-theoretic model has been analyzed. For the individual optimization of each agent’s threshold policy, we further propose a programming scheme by transforming a constrained threshold optimization into an unconstrained Lagrangian optimization (ULO). The individual ULO is integrated into the DBR framework to enable agents to cooperate and converge to a global optimum in a distributed manner. Finally, simulation results are provided to validate the proposed method and demonstrate its significant advantage over other existing distributed methods. The numerical results also show that the proposed method can achieve comparable performance to a centralized optimization method

Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey

Non-terrestrial networks (NTNs) traditionally have certain limited applications. However, the recent technological advancements and manufacturing cost reduction opened up myriad applications of NTNs for 5G and beyond networks, especially when integrated into terrestrial networks (TNs). This article comprehensively surveys the evolution of NTNs highlighting their relevance to 5G networks and essentially, how it will play a pivotal role in the development of 6G ecosystem. We discuss important features of NTNs integration into TNs and the synergies by delving into the new range of services and use cases, various architectures, technological enablers, and higher layer aspects pertinent to NTNs integration. Moreover, we review the corresponding challenges arising from the technical peculiarities and the new approaches being adopted to develop efficient integrated ground-air-space (GAS) networks. Our survey further includes the major progress and outcomes from academic research as well as industrial efforts representing the main industrial trends, field trials, and prototyping towards the 6G networks

Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey

Non-terrestrial networks (NTNs) traditionally have certain limited applications. However, the recent technological advancements and manufacturing cost reduction opened up myriad applications of NTNs for 5G and beyond networks, especially when integrated into terrestrial networks (TNs). This article comprehensively surveys the evolution of NTNs highlighting their relevance to 5G networks and essentially, how it will play a pivotal role in the development of 6G ecosystem. We discuss important features of NTNs integration into TNs and the synergies by delving into the new range of services and use cases, various architectures, technological enablers, and higher layer aspects pertinent to NTNs integration. Moreover, we review the corresponding challenges arising from the technical peculiarities and the new approaches being adopted to develop efficient integrated ground-air-space (GAS) networks. Our survey further includes the major progress and outcomes from academic research as well as industrial efforts representing the main industrial trends, field trials, and prototyping towards the 6G networks