Reinforcement Learning Scheduler for Vehicle-to-Vehicle Communications Outside Coverage

Radio resources in vehicle-to-vehicle (V2V) communication can be scheduled either by a centralized scheduler residing in the network (e.g., a base station in case of cellular systems) or a distributed scheduler, where the resources are autonomously selected by the vehicles. The former approach yields a considerably higher resource utilization in case the network coverage is uninterrupted. However, in case of intermittent or out-of-coverage, due to not having input from centralized scheduler, vehicles need to revert to distributed scheduling. Motivated by recent advances in reinforcement learning (RL), we investigate whether a centralized learning scheduler can be taught to efficiently pre-assign the resources to vehicles for out-of-coverage V2V communication. Specifically, we use the actor-critic RL algorithm to train the centralized scheduler to provide non-interfering resources to vehicles before they enter the out-of-coverage area. Our initial results show that a RL-based scheduler can achieve performance as good as or better than the state-of-art distributed scheduler, often outperforming it. Furthermore, the learning process completes within a reasonable time (ranging from a few hundred to a few thousand epochs), thus making the RL-based scheduler a promising solution for V2V communications with intermittent network coverage.Comment: Article published in IEEE VNC 201

Coherent, automatic address resolution for vehicular ad hoc networks

Published in: Int. J. of Ad Hoc and Ubiquitous Computing, 2017 Vol.25, No.3, pp.163 - 179. DOI: 10.1504/IJAHUC.2017.10001935The interest in vehicular communications has increased notably. In this paper, the use of the address resolution (AR) procedures is studied for vehicular ad hoc networks (VANETs). We analyse the poor performance of AR transactions in such networks and we present a new proposal called coherent, automatic address resolution (CAAR). Our approach inhibits the use of AR transactions and instead increases the usefulness of routing signalling to automatically match the IP and MAC addresses. Through extensive simulations in realistic VANET scenarios using the Estinet simulator, we compare our proposal CAAR to classical AR and to another of our proposals that enhances AR for mobile wireless networks, called AR+. In addition, we present a performance evaluation of the behaviour of CAAR, AR and AR+ with unicast traffic of a reporting service for VANETs. Results show that CAAR outperforms the other two solutions in terms of packet losses and furthermore, it does not introduce additional overhead.Postprint (published version