Multi-leader Adaptive Cruise Control Systems considering Sensor Measurement Uncertainties based on Deep Reinforcement Learning

Adaptive Cruise Control (ACC) relieves human drivers’ tasks by taking over the control of the throttle and braking of the vehicles automatically. However, it has been demonstrated in many empirical studies that current production ACC systems fail to guarantee string stability. It is believed that if vehicles can take the longitudinal dynamics further downstream into account and react to the propagating disturbance earlier, the string stability in the platoon may be improved. Instead of relying on inter-vehicle communication technologies, the ego-vehicle should be able to detect the second leading vehicle by leveraging the power of on-board sensors. Still, the second leader measurements can be highly erroneous. Therefore, it is important to consider the entailed measurement uncertainties when designing and evaluating such ACC systems. This study proposes several ACC systems which possess the property of multi-anticipation and uncertainty handling.The possible sensor technology which can collect the second leader measurements is first investigated. Based on the considered setup, the measurement uncertainties are modelled to reflect the real-world conditions. The ACC system architecture and control system design method are then proposed. Deep reinforcement learning is applied for the controller design in light of its great potential in describing the complex non-linear control task and handling the uncertainties. Kalman filters and recurrent policies with a Long-Short-Term-Memory network are applied to cope with uncertain measurements. The first method estimates the state information before feeding it back to the controller agent, while the latter incorporates the state estimator into the controller to actively consider the uncertainties while making decisions.A numerical simulation approach is adopted to theoretically assess the performance of the proposed ACC systems. A traffic disturbance event and multiple levels of measurement noise are considered in the experiment. To analyze the performance in terms of string stability and ride comfort and understand the car-following behavior mechanism resulted from the proposed systems, a quantitative analysis framework is developed.The evaluation results demonstrate the applied learning-based approach succeeds to train ACC control policies which can ensure string stability. It is also found that the multi-anticipation ability significantly improves the string stability and ride comfort performance. In the scenarios with measurement noise, systems using the tuned Kalman filters exhibit the ideal level of string stability performance. However, ride comfort cannot be guaranteed in scenarios with large measurement noise. On the other hand, systems using recurrent policies can better ensure ride comfort performance while maintaining string stability at certain levels. Based on the results, the performance limits of the proposed ACC systems in the handling of measurement uncertainties are explored. In addition, with the different policy training setups, the trade-off between these two performance aspects is shown.The findings of this study are anticipated to trigger the development of advanced multi-leader ACC system by automakers, sensor manufacturers, and traffic engineers. Future work can be directed to an enhanced controller design. Robustness of the systems with respect to other sources of measurement uncertainties, more types of traffic disturbance, and platoon heterogeneity is worth further design consideration and analysis.Civil Engineering | Transport and Plannin

An operational simulation framework for modelling the multi-interaction of two-wheelers on mixed-traffic road segments

In recent years, the interest in riding in cities using the two-wheeler (e.g., bicycles, electric bicycles, electric mopeds, etc.) increases. Mixed-traffic road segments are one of the most common traffic scenes where the mixed two-wheeler flows exist. Because the movements are often not restricted by lanes, the two-wheeler uses lateral road space more freely and shows obvious multilateral interactions (i.e. multi-interaction) with others, bringing issues that endanger traffic safety. A precise estimation of its impacts on traffic operation and safety is necessary, while the microscopic simulation model can satisfy the need as a helpful tool. However, most existing simulation models of these three types of two-wheelers are essentially focusing on handling the one-on-one interaction. The capability to deal with the two-wheeler multi-interaction in mixed traffic is still rare, and the description of what endogenous tasks are contained by the multi-interaction has also not given by literature. To this end, this paper first defines what the multi-interaction entails on the operational behaviour level, claiming that it contains three intertwined processes, namely a (mental) perception, a (mental) decision, and a physical process. The (mental) perception and decision processes represent the recognition of interactions and the response to traffic conditions, while the physical process refers to the execution of these mental activities. A three-layer simulation framework has then been developed, where each layer sequentially corresponds to one of the operational behaviour tasks. Integrated component models are also proposed in each layer to cover these operational tasks. A Comfort Zone model is hence put forward to dynamically perceive the multiple interactive road users, while a Bayesian network model is developed to deal with the decision-making process under multi-interaction situations. Meanwhile, a behaviour force model is also proposed to capture the non-lane based movements following the selected behaviour and current interaction states. Finally, we face validate the proposed models by the comparison between simulation results and observations obtained from trajectory dataset. Results indicate the model performance matches the observed interaction and motion well.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Transport and Plannin