A REINFORCEMENT LEARNING APPROACH TO VEHICLE PATH OPTIMIZATION IN URBAN ENVIRONMENTS

Road traffic management in metropolitan cities and urban areas, in general, is an important component of Intelligent Transportation Systems (ITS). With the increasing number of world population and vehicles, a dramatic increase in road traffic is expected to put pressure on the transportation infrastructure. Therefore, there is a pressing need to devise new ways to optimize the traffic flow in order to accommodate the growing needs of transportation systems. This work proposes to use an Artificial Intelligent (AI) method based on reinforcement learning techniques for computing near-optimal vehicle itineraries applied to Vehicular Ad-hoc Networks (VANETs). These itineraries are optimized based on the vehicle’s travel distance, travel time, and traffic road congestion. The problem of traffic density is formulated as a Markov Decision Process (MDP). In particular, this work introduces a new reward function that takes into account the traffic congestion when learning about the vehicle’s best action (best turn) to take in different situations. To learn the effect of this approach, the work investigated different learning algorithms such as Q-Learning and SARSA in conjunction with two exploration strategies: (a) e-greedy and (b) Softmax. A comparative performance study of these methods is presented to determine the most effective solution that enables the vehicles to find a fast and reliable path. Simulation experiments illustrate the effectiveness of proposed methods in computing optimal itineraries allowing vehicles to avoid traffic congestion while maintaining reasonable travel times and distances

OBPF: Opportunistic Beaconless Packet Forwarding Strategy for Vehicular Ad Hoc Networks

[EN] In a vehicular ad hoc network, the communication links are unsteady due to the rapidly changing topology, high mobility and traffic density in the urban environment. Most of the existing geographical routing protocols rely on the continuous transmission of beacon messages to update the neighbors' presence, leading to network congestion. Source-based approaches have been proven to be inefficient in the inherently unstable network. To this end, we propose an opportunistic beaconless packet forwarding approach based on a modified handshake mechanism for the urban vehicular environment. The protocol acts differently between intersections and at the intersection to find the next forwarder node toward the destination. The modified handshake mechanism contains link quality, forward progress and directional greedy metrics to determine the best relay node in the network. After designing the protocol, we compared its performance with existing routing protocols. The simulation results show the superior performance of the proposed protocol in terms of packet delay and data delivery ratio in realistic wireless channel conditions.The authors would like to extend their sincere appreciation to the Deanship of Scientific Research at King Saud University for funding this research. The research is supported by Ministry of Education Malaysia (MOE) and conducted in collaboration with Research Management Center (RMC) at Universiti Teknologi Malaysia (UTM) under VOT NUMBER: QJ130000.2528.06H00.Qureshi, KN.; Abdullah, AH.; Lloret, J.; Altameem, A. (2016). OBPF: Opportunistic Beaconless Packet Forwarding Strategy for Vehicular Ad Hoc Networks. KSII Transactions on Internet and Information Systems. 10(5):2144-2165. https://doi.org/10.3837/tiis.2016.05.011S2144216510

Failure-Sentient Composition For Swarm-Based Drone Services

We propose a novel failure-sentient framework for swarm-based drone delivery services. The framework ensures that those drones that experience a noticeable degradation in their performance (called soft failure) and which are part of a swarm, do not disrupt the successful delivery of packages to a consumer. The framework composes a weighted continual federated learning prediction module to accurately predict the time of failures of individual drones and uptime after failures. These predictions are used to determine the severity of failures at both the drone and swarm levels. We propose a speed-based heuristic algorithm with lookahead optimization to generate an optimal set of services considering failures. Experimental results on real datasets prove the efficiency of our proposed approach in terms of prediction accuracy, delivery times, and execution times.Comment: 11 pages, 14 figures, This paper is accepted in the 2023 IEEE International Conference on Web Services (ICWS 2023

Cooperative Path-Planning for Multi-Vehicle Systems

In this paper, we propose a collision avoidance algorithm for multi-vehicle systems, which is a common problem in many areas, including navigation and robotics. In dynamic environments, vehicles may become involved in potential collisions with each other, particularly when the vehicle density is high and the direction of travel is unrestricted. Cooperatively planning vehicle movement can effectively reduce and fairly distribute the detour inconvenience before subsequently returning vehicles to their intended paths. We present a novel method of cooperative path planning for multi-vehicle systems based on reinforcement learning to address this problem as a decision process. A dynamic system is described as a multi-dimensional space formed by vectors as states to represent all participating vehicles’ position and orientation, whilst considering the kinematic constraints of the vehicles. Actions are defined for the system to transit from one state to another. In order to select appropriate actions whilst satisfying the constraints of path smoothness, constant speed and complying with a minimum distance between vehicles, an approximate value function is iteratively developed to indicate the desirability of every state-action pair from the continuous state space and action space. The proposed scheme comprises two phases. The convergence of the value function takes place in the former learning phase, and it is then used as a path planning guideline in the subsequent action phase. This paper summarizes the concept and methodologies used to implement this online cooperative collision avoidance algorithm and presents results and analysis regarding how this cooperative scheme improves upon two baseline schemes where vehicles make movement decisions independently

Performance enhancement of wireless communication systems through QoS optimisation

Providing quality of service (QoS) in a communication network is essential but challenging, especially when the complexities of wireless and mobile networks are added. The issues of how to achieve the intended performances, such as reliability and efficiency, at the minimal resource cost for wireless communications and networking have not been fully addressed. In this dissertation, we have investigated different data transmission schemes in different wireless communication systems such as wireless sensor network, device-to-device communications and vehicular networks. We have focused on cooperative communications through relaying and proposed a method to maximise the QoS performance by finding optimum transmission schemes. Furthermore, the performance trade-offs that we have identified show that both cooperative and non-cooperative transmission schemes could have advantages as well as disadvantages in offering QoS. In the analytical approach, we have derived the closed-form expressions of the outage probability, throughput and energy efficiency for different transmission schemes in wireless and mobile networks, in addition to applying other QoS metrics such as packet delivery ratio, packet loss rate and average end-to-end delay. We have shown that multi-hop relaying through cooperative communications can outperform non-cooperative transmission schemes in many cases. Furthermore, we have also analysed the optimum required transmission power for different transmission ranges to obtain the maximum energy efficiency or maximum achievable data rate with the minimum outage probability and bit error rate in cellular network. The proposed analytical and modelling approaches are used in wireless sensor networks, device-to-device communications and vehicular networks. The results generated have suggested an adaptive transmission strategy where the system can decide when and how each of transmission schemes should be adopted to achieve the best performance in varied conditions. In addition, the system can also choose proper transmitting power levels under the changing transmission distance to increase and maintain the network reliability and system efficiency accordingly. Consequently, these functions will lead to the optimized QoS in a given network