Evolving controllers for simulated car racing

This paper describes the evolution of controllers for racing a simulated radio-controlled car around a track, modelled on a real physical track. Five different controller architectures were compared, based on neural networks, force fields and action sequences. The controllers use either egocentric (first person), Newtonian (third person) or no information about the state of the car (open-loop controller). The only controller that is able to evolve good racing behaviour is based on a neural network acting on egocentric inputs

The 2007 IEEE CEC simulated car racing competition

This paper describes the simulated car racing competition that was arranged as part of the 2007 IEEE Congress on Evolutionary Computation. Both the game that was used as the domain for the competition, the controllers submitted as entries to the competition and its results are presented. With this paper, we hope to provide some insight into the efficacy of various computational intelligence methods on a well-defined game task, as well as an example of one way of running a competition. In the process, we provide a set of reference results for those who wish to use the simplerace game to benchmark their own algorithms. The paper is co-authored by the organizers and participants of the competitio

A human-like TORCS controller for the Simulated Car Racing Championship

Proceeding of: IEEE Congres on Computational Intelligence and Games (CIG'10), Copenhagen (Denmark), 18-21, August, 2010.This paper presents a controller for the 2010 Simulated Car Racing Championship. The idea is not to create the fastest controller but a human-like controller. In order to achieve this, first we have created a process to build a model of the tracks while the car is running and then we used several neural networks which predict the trajectory the car should follow and the target speed. A scripted policy is used for the gear change and to follow the predicted trajectory with the predicted speed. The neural networks are trained with data retrieved from a human player, and are evaluated in a new track. The results shows an acceptable performance of the controller in unknown tracks, more than 20% slower than the human in the same tracks because of the mistakes made when the controller tries to follow the trajectory.This work was supported in part by the University Carlos III of Madrid under grant PIF UC3M01-0809 and by the Ministry of Science and Innovation under project TRA2007- 67374-C02-02

Evolving robust and specialized car racing skills

Neural network-based controllers are evolved for racing simulated R/C cars around several tracks of varying difficulty. The transferability of driving skills acquired when evolving for a single track is evaluated, and different ways of evolving controllers able to perform well on many different tracks are investigated. It is further shown that such generally proficient controllers can reliably be developed into specialized controllers for individual tracks. Evolution of sensor parameters together with network weights is shown to lead to higher final fitness, but only if turned on after a general controller is developed, otherwise it hinders evolution. It is argued that simulated car racing is a scalable and relevant testbed for evolutionary robotics research, and that the results of this research can be useful for commercial computer games

Evolving controllers for simulated car racing using differential evolution

This paper presents an initial approach for creating autonomous controllers for the car racing game using a hybrid technique. The Differential Evolution (DE) algorithm is combined with Feed-forward Artificial Neural Networks (FFANNs) to generate the required intelligent controllers in a well-known car racing game, namely The Open Racing Car Simulator (TORCS). TORCS is used as a platform in most of the IEEE conference competitions. The main objective of this research is to test the feasibility of the DE implementation in TORCS platform. The literature showed that the application of DE in Real Time Strategy game returned promising results in evolving the required strategy gaming controllers. Interestingly, there is still no study thus far that has been conducted in applying DE into TORCS game platform. This research result shows that DE performed well in TORCS even though a very simple fitness function was used. This indicates that DE has well-tuned the neural network weights to generate optimal and sub-optimal controllers in TORCS

A Probabilistic Framework for Imitating Human Race Driver Behavior

Understanding and modeling human driver behavior is crucial for advanced vehicle development. However, unique driving styles, inconsistent behavior, and complex decision processes render it a challenging task, and existing approaches often lack variability or robustness. To approach this problem, we propose Probabilistic Modeling of Driver behavior (ProMoD), a modular framework which splits the task of driver behavior modeling into multiple modules. A global target trajectory distribution is learned with Probabilistic Movement Primitives, clothoids are utilized for local path generation, and the corresponding choice of actions is performed by a neural network. Experiments in a simulated car racing setting show considerable advantages in imitation accuracy and robustness compared to other imitation learning algorithms. The modular architecture of the proposed framework facilitates straightforward extensibility in driving line adaptation and sequencing of multiple movement primitives for future research.Comment: updated references [17] and [33]; added journal inf

Learning to Race through Coordinate Descent Bayesian Optimisation

In the automation of many kinds of processes, the observable outcome can often be described as the combined effect of an entire sequence of actions, or controls, applied throughout its execution. In these cases, strategies to optimise control policies for individual stages of the process might not be applicable, and instead the whole policy might have to be optimised at once. On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system. We consider the problem of optimising control policies to allow a robot to complete a given race track within a minimum amount of time. We assume that the robot has no prior information about the track or its own dynamical model, just an initial valid driving example. Localisation is only applied to monitor the robot and to provide an indication of its position along the track's centre axis. We propose a method for finding a policy that minimises the time per lap while keeping the vehicle on the track using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert space. We apply an algorithm to search more efficiently over high-dimensional policy-parameter spaces with BO, by iterating over each dimension individually, in a sequential coordinate descent-like scheme. Experiments demonstrate the performance of the algorithm against other methods in a simulated car racing environment.Comment: Accepted as conference paper for the 2018 IEEE International Conference on Robotics and Automation (ICRA

Controller for TORCS created by imitation

Proceeding of: IEEE Symposium on Computational Intelligence and Games, 2009. CIG 2009, september 7-10, 2009, Milano, ItalyThis paper is an initial approach to create a controller for the game TORCS by learning how another controller or humans play the game. We used data obtained from two controllers and from one human player. The first controller is the winner of the WCCI 2008 Simulated Car Racing Competition, and the second one is a hand coded controller that performs a complete lap in all tracks. First, each kind of controller is imitated separately, then a mix of the data is used to create new controllers. The imitation is performed by means of training a feed forward neural network with the data, using the backpropagation algorithm for learning.This work was supported in part by the University Carlos III of Madrid under grant PIF UC3M01-0809 and by the Ministry of Science and Innovation under project TRA2007- 67374-C02-02