Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

The majority of current studies on autonomous vehicle control via deep reinforcement learning (DRL) utilize point-mass kinematic models, neglecting vehicle dynamics which includes acceleration delay and acceleration command dynamics. The acceleration delay, which results from sensing and actuation delays, results in delayed execution of the control inputs. The acceleration command dynamics dictates that the actual vehicle acceleration does not rise up to the desired command acceleration instantaneously due to dynamics. In this work, we investigate the feasibility of applying DRL controllers trained using vehicle kinematic models to more realistic driving control with vehicle dynamics. We consider a particular longitudinal car-following control, i.e., Adaptive Cruise Control (ACC), problem solved via DRL using a point-mass kinematic model. When such a controller is applied to car following with vehicle dynamics, we observe significantly degraded car-following performance. Therefore, we redesign the DRL framework to accommodate the acceleration delay and acceleration command dynamics by adding the delayed control inputs and the actual vehicle acceleration to the reinforcement learning environment state, respectively. The training results show that the redesigned DRL controller results in near-optimal control performance of car following with vehicle dynamics considered when compared with dynamic programming solutions.Comment: Accepted to 2019 IEEE Intelligent Transportation Systems Conferenc

Adaptive driver modelling in ADAS to improve user acceptance: A study using naturalistic data

Accurate understanding of driver behaviour is crucial for future Advanced Driver Assistance Systems (ADAS) and autonomous driving. For user acceptance it is important that ADAS respect individual driving styles and adapt accordingly. Using data collected during a naturalistic driving study carried out at the University of Southampton, we assess existing models of driver acceleration and speed choice during car following and when cornering. We observe that existing models of driver behaviour that specify a preferred inter-vehicle spacing in car-following situations appear to be too prescriptive, with a wide range of acceptable spacings visible in the naturalistic data. Bounds on lateral acceleration during cornering from the literature are visible in the data, but appear to be influenced by the minimum cornering radii specified in design codes for UK roadway geometry. This analysis of existing driver models is used to suggest a small set of parameters that are sufficient to characterise driver behaviour in car-following and curve driving, which may be estimated in real-time by an ADAS to adapt to changing driver behaviour. Finally, we discuss applications to adaptive ADAS with the objectives of improving road safety and promoting eco-driving, and suggest directions for future researc