Reinforcement Learning Scheduler for Vehicle-to-Vehicle Communications Outside Coverage

Radio resources in vehicle-to-vehicle (V2V) communication can be scheduled either by a centralized scheduler residing in the network (e.g., a base station in case of cellular systems) or a distributed scheduler, where the resources are autonomously selected by the vehicles. The former approach yields a considerably higher resource utilization in case the network coverage is uninterrupted. However, in case of intermittent or out-of-coverage, due to not having input from centralized scheduler, vehicles need to revert to distributed scheduling. Motivated by recent advances in reinforcement learning (RL), we investigate whether a centralized learning scheduler can be taught to efficiently pre-assign the resources to vehicles for out-of-coverage V2V communication. Specifically, we use the actor-critic RL algorithm to train the centralized scheduler to provide non-interfering resources to vehicles before they enter the out-of-coverage area. Our initial results show that a RL-based scheduler can achieve performance as good as or better than the state-of-art distributed scheduler, often outperforming it. Furthermore, the learning process completes within a reasonable time (ranging from a few hundred to a few thousand epochs), thus making the RL-based scheduler a promising solution for V2V communications with intermittent network coverage.Comment: Article published in IEEE VNC 201

Security of 5G-V2X: Technologies, Standardization and Research Directions

Cellular-Vehicle to Everything (C-V2X) aims at resolving issues pertaining to the traditional usability of Vehicle to Infrastructure (V2I) and Vehicle to Vehicle (V2V) networking. Specifically, C-V2X lowers the number of entities involved in vehicular communications and allows the inclusion of cellular-security solutions to be applied to V2X. For this, the evolvement of LTE-V2X is revolutionary, but it fails to handle the demands of high throughput, ultra-high reliability, and ultra-low latency alongside its security mechanisms. To counter this, 5G-V2X is considered as an integral solution, which not only resolves the issues related to LTE-V2X but also provides a function-based network setup. Several reports have been given for the security of 5G, but none of them primarily focuses on the security of 5G-V2X. This article provides a detailed overview of 5G-V2X with a security-based comparison to LTE-V2X. A novel Security Reflex Function (SRF)-based architecture is proposed and several research challenges are presented related to the security of 5G-V2X. Furthermore, the article lays out requirements of Ultra-Dense and Ultra-Secure (UD-US) transmissions necessary for 5G-V2X.Comment: 9 pages, 6 figures, Preprin

Radio resource management for V2X in cellular systems

The thesis focuses on the provision of cellular vehicle-to-everything (V2X) communications, which have attracted great interest for 5G due to the potential of improving traffic safety and enabling new services related to intelligent transportation systems. These types of services have strict requirements on reliability, access availability, and end-to-end (E2E) latency. V2X requires advanced network management techniques that must be developed based on the characteristics of the networks and traffic requirements. The integration of the Sidelink (SL), which enables the direct communication between vehicles (i.e., vehicle-to-vehicle (V2V)) without passing through the base station into cellular networks is a promising solution for enhancing the performance of V2X in cellular systems. In this thesis, we addressed some of the challenges arising from the integration of V2V communication in cellular systems and validated the potential of this technology by providing appropriate resource management solutions. Our main contributions have been in the context of radio access network slicing, mode selection, and radio resource allocation mechanisms. With regard to the first research direction that focuses on the RAN slicing management, a novel strategy based on offline Q-learning and softmax decision-making has been proposed as an enhanced solution to determine the adequate split of resources between a slice for eMBB communications and a slice for V2X. Then, starting from the outcome of the off-line Q-learning algorithm, a low-complexity heuristic strategy has been proposed to achieve further improvements in the use of resources. The proposed solution has been compared against proportional and fixed reference schemes. The extensive performance assessment have revealed the ability of the proposed algorithms to improve network performance compared to the reference schemes, especially in terms of resource utilization, throughput, latency and outage probability. Regarding the second research direction that focuses on the mode selection, two different mode selection solutions referred to as MSSB and MS-RBRS strategies have been proposed for V2V communication over a cellular network. The MSSB strategy decides when it is appropriate to use one or the other mode, i.e. sidelink or cellular, for the involved vehicles, taking into account the quality of the links between V2V users, the available resources, and the network traffic load situation. Moreover, the MS-RBRS strategy not only selects the appropriate mode of operation but also decides efficiently the amount of resources needed by V2V links in each mode and allows reusing RBs between different SL users while guaranteeing the minimum signal to interference requirements. The conducted simulations have revealed that the MS-RBRS and MSSB strategies are beneficial in terms of throughput, radio resource utilization, outage probability and latency under different offered loads comparing to the reference scheme. Last, we have focused on the resource allocation problem including jointly mode selection and radio resource scheduling. For the mode selection, a novel mode selection has been presented to decide when it is appropriate to select sidelink mode and use a distributed approach for radio resource allocation or cellular mode and use a centralized radio resource allocation. It takes into account three aspects: the quality of the links between V2V users, the available resources, and the latency. As for the radio resource allocation, the proposed approach includes a distributed radio resource allocation for sidelink mode and a centralized radio resource allocation for cellular mode. The proposed strategy supports dynamic assignments by allowing transmission over mini-slots. A simulation-based analysis has shown that the proposed strategies improved the network performance in terms of latency of V2V services, packet success rate and resource utilization under different network loads.La tesis se centra en la provisión de comunicaciones para vehículos sistemas celulares (V2X: Vehicle to Everything), que han atraído un gran interés en el contexto de 5G debido a su potencial de mejorar la seguridad del tráfico y habilitar nuevos servicios relacionados con los sistemas inteligentes de transporte. Estos tipos de servicios tienen requisitos estrictos en términos fiabilidad, disponibilidad de acceso y latencia de extremo a extremo (E2E). Para ello, V2X requiere técnicas avanzadas de gestión de red que deben desarrollarse en función de las características de las redes y los requisitos de tráfico. La integración del Sidelink (SL), que permite la comunicación directa entre vehículos (es decir, vehículo a vehículo (V2V)) sin pasar por la estación base de las redes celulares, es una solución prometedora para mejorar el rendimiento de V2X en el sistema celular. En esta tesis, abordamos algunos de los desafíos derivados de la integración de la comunicación V2V en los sistemas celulares y validamos el potencial de esta tecnología al proporcionar soluciones de gestión de recursos adecuadas. Nuestras principales contribuciones han sido en el contexto del denominado "slicing" de redes de acceso radio, la selección de modo y los mecanismos de asignación de recursos radio. Respecto a la primera dirección de investigación que se centra en la gestión del RAN slicing, se ha propuesto una estrategia novedosa basada en Q-learning y toma de decisiones softmax como una solución para determinar la división adecuada de recursos entre un slice para comunicaciones eMBB y un slice para V2X. Luego, a partir del resultado del algoritmo de Q-learning, se ha propuesto una estrategia heurística de baja complejidad para lograr mejoras adicionales en el uso de los recursos. La solución propuesta se ha comparado con esquemas de referencia proporcionales y fijos. La evaluación ha revelado la capacidad de los algoritmos propuestos para mejorar el rendimiento de la red en comparación con los esquemas de referencia, especialmente en términos de utilización de recursos, rendimiento, y latencia . Con respecto a la segunda dirección de investigación que se centra en la selección de modo, se han propuesto dos soluciones de diferentes llamadas estrategias MSSB y MS-RBRS para la comunicación V2V a través de una red celular. La estrategia MSSB decide cuándo es apropiado usar el modo SL o el modo celular, para los vehículos involucrados, teniendo en cuenta la calidad de los enlaces entre los usuarios de V2V, los recursos disponibles y la situación de carga de tráfico de la red. Además, la estrategia MS-RBRS no solo selecciona el modo de operación apropiado, sino que también decide eficientemente la cantidad de recursos que los enlaces V2V necesitan en cada modo, y permite que los RB se reutilicen entre diferentes usuarios de SL al tiempo que garantiza requisitos mínimos de señal a interferencia. Se ha presentado un análisis basado en simulación para evaluar el desempeño de las estrategias propuestas. Finalmente, nos hemos centrado en el problema conjunto de la selección de modo y la asignación de recursos de radio. Para la selección de modo, se ha presentado una nueva estrategia para decidir cuándo es apropiado seleccionar el modo SL y usar un enfoque distribuido para la asignación de recursos de radio o el modo celular y usar la asignación de recursos de radio centralizada. Tiene en cuenta tres aspectos: la calidad de los enlaces entre los usuarios de V2V, los recursos disponibles y la latencia. En términos de asignación de recursos de radio, el enfoque propuesto incluye una asignación de recursos de radio distribuida para el modo SL y una asignación de recursos de radio centralizada para el modo celular. La estrategia propuesta admite asignaciones dinámicas al permitir la transmisión a través de mini-slots. Los resultados muestran las mejoras en términos de latencia, tasa de recepción y la utilización de recursos bajo diferentes cargas de red.Postprint (published version

Heterogeneous V2V Communications in Multi-Link and Multi-RAT Vehicular Networks

Connected and automated vehicles will enable advanced traffic safety and efficiency applications thanks to the dynamic exchange of information between vehicles, and between vehicles and infrastructure nodes. Connected vehicles can utilize IEEE 802.11p for vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications. However, a widespread deployment of connected vehicles and the introduction of connected automated driving applications will notably increase the bandwidth and scalability requirements of vehicular networks. This paper proposes to address these challenges through the adoption of heterogeneous V2V communications in multi-link and multi-RAT vehicular networks. In particular, the paper proposes the first distributed (and decentralized) context-aware heterogeneous V2V communications algorithm that is technology and application agnostic, and that allows each vehicle to autonomously and dynamically select its communications technology taking into account its application requirements and the communication context conditions. This study demonstrates the potential of heterogeneous V2V communications, and the capability of the proposed algorithm to satisfy the vehicles' application requirements while approaching the estimated upper bound network capacity