Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learning

This paper concerns automated vehicles negotiating with other vehicles, typically human driven, in crossings with the goal to find a decision algorithm by learning typical behaviors of other vehicles. The vehicle observes distance and speed of vehicles on the intersecting road and use a policy that adapts its speed along its pre-defined trajectory to pass the crossing efficiently. Deep Q-learning is used on simulated traffic with different predefined driver behaviors and intentions. The results show a policy that is able to cross the intersection avoiding collision with other vehicles 98% of the time, while at the same time not being too passive. Moreover, inferring information over time is important to distinguish between different intentions and is shown by comparing the collision rate between a Deep Recurrent Q-Network at 0.85% and a Deep Q-learning at 1.75%.Comment: 6 pages, 7 figures, Accepted to IEEE International Conference on Intelligent Transportation Systems (ITSC) 201

Evolving a rule system controller for automatic driving in a car racing competition

IEEE Symposium on Computational Intelligence and Games. Perth, Australia, 15-18 December 2008.The techniques and the technologies supporting Automatic Vehicle Guidance are important issues. Automobile manufacturers view automatic driving as a very interesting product with motivating key features which allow improvement of the car safety, reduction in emission or fuel consumption or optimization of driver comfort during long journeys. Car racing is an active research field where new advances in aerodynamics, consumption and engine power are critical each season. Our proposal is to research how evolutionary computation techniques can help in this field. For this work we have designed an automatic controller that learns rules with a genetic algorithm. This paper is a report of the results obtained by this controller during the car racing competition held in Hong Kong during the IEEE World Congress on Computational Intelligence (WCCI 2008).Publicad

Towards Social Autonomous Vehicles: Efficient Collision Avoidance Scheme Using Richardson's Arms Race Model

Background Road collisions and casualties pose a serious threat to commuters around the globe. Autonomous Vehicles (AVs) aim to make the use of technology to reduce the road accidents. However, the most of research work in the context of collision avoidance has been performed to address, separately, the rear end, front end and lateral collisions in less congested and with high inter-vehicular distances. Purpose The goal of this paper is to introduce the concept of a social agent, which interact with other AVs in social manners like humans are social having the capability of predicting intentions, i.e. mentalizing and copying the actions of each other, i.e. mirroring. The proposed social agent is based on a human-brain inspired mentalizing and mirroring capabilities and has been modelled for collision detection and avoidance under congested urban road traffic. Method We designed our social agent having the capabilities of mentalizing and mirroring and for this purpose we utilized Exploratory Agent Based Modeling (EABM) level of Cognitive Agent Based Computing (CABC) framework proposed by Niazi and Hussain. Results Our simulation and practical experiments reveal that by embedding Richardson's arms race model within AVs, collisions can be avoided while travelling on congested urban roads in a flock like topologies. The performance of the proposed social agent has been compared at two different levels.Comment: 48 pages, 21 figure