Federated Multi-Agent Deep Reinforcement Learning for Dynamic and Flexible 3D Operation of 5G Multi-MAP Networks

This paper addresses the efficient management of Mobile Access Points (MAPs), which are Unmanned Aerial Vehicles (UAV), in 5G networks. We propose a two-level hierarchical architecture, which dynamically reconfigures the network while considering Integrated Access-Backhaul (IAB) constraints. The high-layer decision process determines the number of MAPs through consensus, and we develop a joint optimization process to account for co-dependence in network self-management. In the low-layer, MAPs manage their placement using a double-attention based Deep Reinforcement Learning (DRL) model that encourages cooperation without retraining. To improve generalization and reduce complexity, we propose a federated mechanism for training and sharing one placement model for every MAP in the low-layer. Additionally, we jointly optimize the placement and backhaul connectivity of MAPs using a multi-objective reward function, considering the impact of varying MAP placement on wireless backhaul connectivity.Comment: 2023 IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC

Dual-attention deep reinforcement learning for multi-map 3D trajectory optimization in dynamic 5G networks

International audience5G and beyond networks need to provide dynamic and efficient infrastructure management to better adapt to time-varying user behaviors (e.g., user mobility, interference, user traffic and evolution of the network topology). In this paper, we propose to manage the trajectory of Mobile Access Points (MAPs) under all these dynamic constraints with reduced complexity. We first formulate the placement problem to manage MAPs over time. Our solution addresses time-varying user traffic and user mobility through a Multi-Agent Deep Reinforcement Learning (MADRL). To achieve real-time behavior, the proposed solution learns to perform distributed assignment of MAP-user positions and schedules the MAP path among all users without centralized user's clustering feedback. Our solution exploits a dual-attention MADRL model via proximal policy optimization to dynamically move MAPs in 3D. The dual-attention takes into account information from both users and MAPs. The cooperation mechanism of our solution allows to manage different scenarios, without a priory information and without re-training, which significantly reduces complexity

Cost-Efficient and QoS-Aware User Association and 3D Placement of 6G Aerial Mobile Access Points

6G networks require a flexible infrastructure to dynamically provide ubiquitous network coverage. Mobile Access Points (MAP) deployment is a promising solution. In this paper, we formulate the joint 3D MAP deployment and user association problem over a dynamic network under interference and mobility constraints. First, we propose an iterative algorithm to optimize the deployment of MAPs. Our solution efficiently and quickly determines the number, position and configuration of MAPs for highly dynamic scenarios. MAPs provide appropriate Quality of Service (QoS) connectivity to mobile ground user in mmwave or sub-6GHz bands and find their optimal positions in a 3D grid. Each MAP also implies an energy cost (e.g. for travel) to be minimized. Once all MAPs deployed, a deep multiagent reinforcement learning algorithm is proposed to associate multiple users to multiple MAPs under interference constraint. Each user acts as an independent agent that operates in a fully distributed architecture and maximizes the network sum-rate.Comment: To be published to 2022 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit

Dynamic resource scheduling optimization for ultra-reliable low latency communications: from simulation to experimentation

International audienceIn this paper, we propose a dynamic and efficient resource scheduling based on Lyapunov's optimization for Ultra-Reliable Low Latency Communication, taking into account the traffic arrival at the network layer, the queues behavior at the data link layer and the risk that the applied decision will result in a loss. The trade-off between the resource efficiency, latency and reliability is achieved by the number, timing and intensity of decisions and is adapted to dynamic scenarios (e.g., random bursty traffic, time-varying channel). Our queue-aware and channel-aware solution is evaluated in terms of latency, reliability outage and resource efficiency in a system-level simulator and validated by an experimental testbed using OpenAirInterface

Spreading Factor Allocation for LoRa Nodes Progressively Joining a Multi-Gateway Adaptive Network

International audienc

Proactive resource scheduling for 5G and beyond ultra-reliable low latency communications

International audienceEffective resource use in Ultra-Reliable and Low-Latency Communications (URLLC) is one of the main challenges for 5G and beyond systems. In this paper, we propose a novel scheduling methodology (combining reactive and proactive resource allocation strategies) specifically devised for URLLC services. Our ultimate objective is to characterize the level of proactivity required to cope with various scenarios. Specifically, we propose to operate at the scheduling level, addressing the trade-off between reliability, latency and resource efficiency. We offer an evaluation of the proposed methodology in the case of the well-known Hybrid Automatic Repeat reQuest (HARQ) protocol in which the proactive strategy allows a number of parallel retransmissions instead of the ‘'send-wait-react’' mode. To this end, we propose some deviations from the HARQ procedure and benchmark the performance in terms of latency, reliability outage and resource efficiency as a function of the level of proactivity. Afterwards, we highlight the critical importance of proactive adaptation in dynamic scenarios (i.e. with changing traffic rates and channel conditions)

Hybrid radio resource management based on multi-agent reinforcement learning

International audienceIn this paper, we propose a novel hybrid grant-based and grant-free radio access scheme. We provide two multi-agentreinforcement learning algorithms to optimize a global network objective in terms of latency, reliability and network throughput: Multi-Agent Deep Q-Learning (MADQL) and Multi-Agent Deep Deterministic Policy Gradient (MADDPG). In MADQL, each user (agent) learns its optimal action-value function, which is based only on its local observation, and performs an optimal opportunistic action using the shared spectrum. MADDPG involves the attached gNB function as a global observer (critic), which criticizes the action of each associated agent (actor) in the network. By leveraging centralised training and decentralised execution, we achieve a shared goal better than the first algorithm. Then,through a system level simulation where the full protocol stack is considered, we show the gain of our approach to efficiently manage radio resources and guarantee latency

DataJoin: An Energy-Efficient Joining Scheme for 802.15.4e TSCH Networks

International audienc

Singer Cyclic Difference Sets for an Energy-Efficient Joining Scheme

International audience—This paper considers the problem of joining a network of constrained IoT devices. We propose to take advantage of Singer Cyclic Difference Sets (S-CDS) for a joining scheme that results in low duty cycles and apply the proposed scheme to the joining problem in 802.15.4e TSCH networks. S-CDS distributes the active periods of nodes over time so that a joining node does not suffer from long scanning periods in contrast to other schemes. We compare S-CDS through simulation with other state-of-the-art schemes adapted to TSCH networks. The comparisons show that S-CDS achieves a better trade-off between energy consumption and joining delay than the best joining schemes while being particularly suitable for devices with strong energy constraints such as energy harvesting nodes

Geolocalizador para Perros Y Gatos de La ciudad de Bogotá implementando la tecnologia Sigfox

Trabajo de InvestigaciónSe presenta la implementación de una red de amplia cobertura y de baja potencia usando Sigfox, para poder solucionar la pérdida de mascotas en la ciudad de Bogotá; en las siguientes páginas se explica la utilidad de este tipo de redes y cómo implementarlas, además se explica por qué Sigfox es la tecnología adecuada para resolver esta problemática y por qué no otra tecnología (LPWAM). Otros científicos e ingenieros han mostrado desarrollos con el tópico elegido, sin embargo, se ha creado un dispositivo aterrizando las nuevas necesidades. El documento expone el proceso desde la implementación, configuración, programación y puesta en marcha; incluye, las pruebas de campo realizadas en diferentes parques de la ciudad de Bogotá. Los datos obtenidos por el dispositivo se muestran en un mapa, usando esta red en la ciudad con cobertura tendrá adaptación en un dispositivo (collar) diseñado para el uso de mascotas.1 INTRODUCCIÓN 2 ANTECEDENTES, ESTADO DEL ARTE Y JUSTIFICACIÓN 3 PLANTEAMIENTO Y FORMULACIÓN DEL PROBLEMA 4 MARCO DE REFERENCIA 5 OBJETIVOS 6 ALCANCES Y LIMITACIONES 7 METODOLOGÍA 8 MANUAL DE USUARIO 9 ANÁLISIS DE RESULTADOS 10 CONCLUSIONES 11 RECOMENDACIONES 12 ANEXOS 13 BIBLIOGRAFÍAPregradoIngeniero Electrónic