3 research outputs found
Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks
In this paper, the problem of the trajectory design for a group of
energy-constrained drones operating in dynamic wireless network environments is
studied. In the considered model, a team of drone base stations (DBSs) is
dispatched to cooperatively serve clusters of ground users that have dynamic
and unpredictable uplink access demands. In this scenario, the DBSs must
cooperatively navigate in the considered area to maximize coverage of the
dynamic requests of the ground users. This trajectory design problem is posed
as an optimization framework whose goal is to find optimal trajectories that
maximize the fraction of users served by all DBSs. To find an optimal solution
for this non-convex optimization problem under unpredictable environments, a
value decomposition based reinforcement learning (VDRL) solution coupled with a
meta-training mechanism is proposed. This algorithm allows the DBSs to
dynamically learn their trajectories while generalizing their learning to
unseen environments. Analytical results show that, the proposed VD-RL algorithm
is guaranteed to converge to a local optimal solution of the non-convex
optimization problem. Simulation results show that, even without meta-training,
the proposed VD-RL algorithm can achieve a 53.2% improvement of the service
coverage and a 30.6% improvement in terms of the convergence speed, compared to
baseline multi-agent algorithms. Meanwhile, the use of meta-learning improves
the convergence speed of the VD-RL algorithm by up to 53.8% when the DBSs must
deal with a previously unseen task
Self-Evolving Integrated Vertical Heterogeneous Networks
6G and beyond networks tend towards fully intelligent and adaptive design in
order to provide better operational agility in maintaining universal wireless
access and supporting a wide range of services and use cases while dealing with
network complexity efficiently. Such enhanced network agility will require
developing a self-evolving capability in designing both the network
architecture and resource management to intelligently utilize resources, reduce
operational costs, and achieve the coveted quality of service (QoS). To enable
this capability, the necessity of considering an integrated vertical
heterogeneous network (VHetNet) architecture appears to be inevitable due to
its high inherent agility. Moreover, employing an intelligent framework is
another crucial requirement for self-evolving networks to deal with real-time
network optimization problems. Hence, in this work, to provide a better insight
on network architecture design in support of self-evolving networks, we
highlight the merits of integrated VHetNet architecture while proposing an
intelligent framework for self-evolving integrated vertical heterogeneous
networks (SEI-VHetNets). The impact of the challenges associated with
SEI-VHetNet architecture, on network management is also studied considering a
generalized network model. Furthermore, the current literature on network
management of integrated VHetNets along with the recent advancements in
artificial intelligence (AI)/machine learning (ML) solutions are discussed.
Accordingly, the core challenges of integrating AI/ML in SEI-VHetNets are
identified. Finally, the potential future research directions for advancing the
autonomous and self-evolving capabilities of SEI-VHetNets are discussed.Comment: 25 pages, 5 figures, 2 table
Meta-Reinforcement Learning for Trajectory Design in Wireless UAV Networks
In this paper, the design of an optimal trajectory for an energy-constrained drone operating in dynamic network environments is studied. In the considered model, a drone base station (DBS) is dispatched to provide uplink connectivity to ground users whose demand is dynamic and unpredictable. In this case, the DBS's trajectory must be adaptively adjusted to satisfy the dynamic user access requests. To this end, a meta-learning algorithm is proposed in order to adapt the DBS's trajectory when it encounters novel environments, by tuning a reinforcement learning (RL) solution. The meta-learning algorithm provides a solution that adapts the DBS in novel environments quickly based on limited former experiences. The meta-tuned RL is shown to yield a faster convergence to the optimal coverage in unseen environments with a considerably low computation complexity, compared to the baseline policy gradient algorithm. Simulation results show that, the proposed meta-learning solution yields a 25% improvement in the convergence speed, and about 10% improvement in the DBS' communication performance, compared to a baseline policy gradient algorithm. Meanwhile, the probability that the DBS serves over 50% of user requests increases about 27%, compared to the baseline policy gradient algorithm