Recognizing driver braking intention with vehicle data using unsupervised learning methods

Recently, the development of braking assistance system has largely benefit the safety of both driver and pedestrians. A robust prediction and detection of driver braking intention will enable driving assistance system response to traffic situation correctly and improve the driving experience of intelligent vehicles. Unsupervised machine learning algorithms has been widely used in clustering and pattern mining in previous researches. In this paper, a various unsupervised clustering methods will be used to build a driver braking intention predictor which can accurately recognize the braking maneuver based on vehicle data captured with CAN bus. The braking maneuver along with other driving maneuvers such as normal driving will be clustered and the results from different methods like K-means and Gaussian mixture model will be compared. Additionally, the evaluation of features from raw data, which are important to driving maneuvers clustering will be proposed. The experiment data are collected from one hybrid electric vehicle in real world. Final results show that the proposed method can detect driver’s braking intention in a very beginning moment with a high accuracy and the most important sets of feature for driving maneuver clustering will be discussed.Recently, the development of braking assistance system has largely benefit the safety of both driver and pedestrians. A robust prediction and detection of driver braking intention will enable driving assistance system response to traffic situation correctly and improve the driving experience of intelligent vehicles. Unsupervised machine learning algorithms has been widely used in clustering and pattern mining in previous researches. In this paper, a various unsupervised clustering methods will be used to build a driver braking intention predictor which can accurately recognize the braking maneuver based on vehicle data captured with CAN bus. The braking maneuver along with other driving maneuvers such as normal driving will be clustered and the results from different methods like K-means and Gaussian mixture model will be compared. Additionally, the evaluation of features from raw data, which are important to driving maneuvers clustering will be proposed. The experiment data are collected from one hybrid electric vehicle in real world. Final results show that the proposed method can detect driver’s braking intention in a very beginning moment with a high accuracy and the most important sets of feature for driving maneuver clustering will be discussed

Driver activity recognition for intelligent vehicles: a deep learning approach

Driver decisions and behaviors are essential factors that can affect the driving safety. To understand the driver behaviors, a driver activities recognition system is designed based on the deep convolutional neural networks (CNN) in this study. Specifically, seven common driving activities are identified, which are the normal driving, right mirror checking, rear mirror checking, left mirror checking, using in-vehicle radio device, texting, and answering the mobile phone, respectively. Among these activities, the first four are regarded as normal driving tasks, while the rest three are classified into the distraction group. The experimental images are collected using a low-cost camera, and ten drivers are involved in the naturalistic data collection. The raw images are segmented using the Gaussian mixture model (GMM) to extract the driver body from the background before training the behavior recognition CNN model. To reduce the training cost, transfer learning method is applied to fine tune the pre-trained CNN models. Three different pre-trained CNN models, namely, AlexNet, GoogLeNet, and ResNet50 are adopted and evaluated. The detection results for the seven tasks achieved an average of 81.6% accuracy using the AlexNet, 78.6% and 74.9% accuracy using the GoogLeNet and ResNet50, respectively. Then, the CNN models are trained for the binary classification task and identify whether the driver is being distracted or not. The binary detection rate achieved 91.4% accuracy, which shows the advantages of using the proposed deep learning approach. Finally, the real-world application are analysed and discussed

Driver workload estimation using a novel hybrid method of error reduction ratio causality and support vector machine

Measuring driver workload is of great significance for improving the understanding of driver behaviours and supporting the improvement of advanced driver assistance systems technologies. In this paper, a novel hybrid method for measuring driver workload estimation for real-world driving data is proposed. Error reduction ratio causality, a new nonlinear causality detection approach, is being proposed in order to assess the correlation of each measured variable to the variation of workload. A full model describing the relationship between the workload and the selected important measurements is then trained via a support vector regression model. Real driving data of 10 participants, comprising 15 measured physiological and vehicle-state variables are used for the purpose of validation. Test results show that the developed error reduction ratio causality method can effectively identify the important variables that relate to the variation of driver workload, and the support vector regression based model can successfully and robustly estimate workload

Advances in vision-based lane detection: algorithms, integration, assessment, and perspectives on ACP-based parallel vision

Lane detection is a fundamental aspect of most current advanced driver assistance systems (ADASs). A large number of existing results focus on the study of vision-based lane detection methods due to the extensive knowledge background and the low-cost of camera devices. In this paper, previous vision-based lane detection studies are reviewed in terms of three aspects, which are lane detection algorithms, integration, and evaluation methods. Next, considering the inevitable limitations that exist in the camera-based lane detection system, the system integration methodologies for constructing more robust detection systems are reviewed and analyzed. The integration methods are further divided into three levels, namely, algorithm, system, and sensor. Algorithm level combines different lane detection algorithms while system level integrates other object detection systems to comprehensively detect lane positions. Sensor level uses multi-modal sensors to build a robust lane recognition system. In view of the complexity of evaluating the detection system, and the lack of common evaluation procedure and uniform metrics in past studies, the existing evaluation methods and metrics are analyzed and classified to propose a better evaluation of the lane detection system. Next, a comparison of representative studies is performed. Finally, a discussion on the limitations of current lane detection systems and the future developing trends toward an Artificial Society, Computational experiment-based parallel lane detection framework is proposed

Cyber-physical system based optimization framework for intelligent powertrain control

The interactions between automatic controls, physics, and driver is an important step towards highly automated driving. This study investigates the dynamical interactions between human-selected driving modes, vehicle controller and physical plant parameters, to determine how to optimally adapt powertrain control to different human-like driving requirements. A cyber-physical system (CPS) based framework is proposed for co-design optimization of the physical plant parameters and controller variables for an electric powertrain, in view of vehicle’s dynamic performance, ride comfort, and energy efficiency under different driving modes. System structure, performance requirements and constraints, optimization goals and methodology are investigated. Intelligent powertrain control algorithms are synthesized for three driving modes, namely sport, eco, and normal modes, with appropriate protocol selections. The performance exploration methodology is presented. Simulation-based parameter optimizations are carried out according to the objective functions. Simulation results show that an electric powertrain with intelligent controller can perform its tasks well under sport, eco, and normal driving modes. The vehicle further improves overall performance in vehicle dynamics, ride comfort, and energy efficiency. The results validate the feasibility and effectiveness of the proposed CPS-based optimization framework, and demonstrate its advantages over a baseline benchmark

An ensemble deep learning approach for driver lane change intention inference

With the rapid development of intelligent vehicles, drivers are increasingly likely to share their control authorities with the intelligent control unit. For building an efficient Advanced Driver Assistance Systems (ADAS) and shared-control systems, the vehicle needs to understand the drivers’ intent and their activities to generate assistant and collaborative control strategies. In this study, a driver intention inference system that focuses on the highway lane change maneuvers is proposed. First, a high-level driver intention mechanism and framework are introduced. Then, a vision-based intention inference system is proposed, which captures the multi-modal signals based on multiple low-cost cameras and the VBOX vehicle data acquisition system. A novel ensemble bi-directional recurrent neural network (RNN) model with Long Short-Term Memory (LSTM) units is proposed to deal with the time-series driving sequence and the temporal behavioral patterns. Naturalistic highway driving data that consists of lane-keeping, left and right lane change maneuvers are collected and used for model construction and evaluation. Furthermore, the driver's pre-maneuver activities are statistically analyzed. It is found that for situation-aware, drivers usually check the mirrors for more than six seconds before they initiate the lane change maneuver, and the time interval between steering the handwheel and crossing the lane is about 2 s on average. Finally, hypothesis testing is conducted to show the significant improvement of the proposed algorithm over existing ones. With five-fold cross-validation, the EBiLSTM model achieves an average accuracy of 96.1% for the intention that is inferred 0.5 s before the maneuver starts

Analysis of autopilot disengagements occurring during autonomous vehicle testing

In present-day highly-automated vehicles, there are occasions when the driving system disengages and the human driver is required to take-over. This is of great importance to a vehicle U+02BC s safety and ride comfort. In the U.S state of California, the Autonomous Vehicle Testing Regulations require every manufacturer testing autonomous vehicles on public roads to submit an annual report summarizing the disengagements of the technology experienced during testing. On 1 January 2016, seven manufacturers submitted their first disengagement reports: Bosch, Delphi, Google, Nissan, Mercedes-Benz, Volkswagen, and Tesla Motors. This work analyses the data from these disengagement reports with the aim of gaining abetter understanding of the situations in which a driver is required to takeover, as this is potentially useful in improving the Society of Automotive Engineers U+0028 SAE U+0029 Level 2 and Level 3 automation technologies. Disengagement events from testing are classified into different groups based on attributes and the causes of disengagement are investigated and compared in detail. The mechanisms and time taken for take-over transition occurred in disengagements are studied. Finally, recommendations for OEMs, manufacturers, and government organizations are also discussed

Driver lane change intention inference for intelligent vehicles: framework, survey, and challenges

Intelligent vehicles and advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver status since ADAS share the vehicle control authorities with the human driver. This study provides an overview of the ego-vehicle driver intention inference (DII), which mainly focus on the lane change intention on highways. First, a human intention mechanism is discussed in the beginning to gain an overall understanding of the driver intention. Next, the ego-vehicle driver intention is classified into different categories based on various criteria. A complete DII system can be separated into different modules, which consists of traffic context awareness, driver states monitoring, and the vehicle dynamic measurement module. The relationship between these modules and the corresponding impacts on the DII are analyzed. Then, the lane change intention inference (LCII) system is reviewed from the perspective of input signals, algorithms, and evaluation. Finally, future concerns and emerging trends in this area are highlighted

Characterization of driver neuromuscular dynamics for human-automation collaboration design of automated vehicles

In order to design an advanced human-automation collaboration system for highly automated vehicles, research into the driver's neuromuscular dynamics is needed. In this paper a dynamic model of drivers' neuromuscular interaction with a steering wheel is firstly established. The transfer function and the natural frequency of the systems are analyzed. In order to identify the key parameters of the driver-steering-wheel interacting system and investigate the system properties under different situations, experiments with driver-in-the-loop are carried out. For each test subject, two steering tasks, namely the passive and active steering tasks, are instructed to be completed. Furthermore, during the experiments, subjects manipulated the steering wheel with two distinct postures and three different hand positions. Based on the experimental results, key parameters of the transfer function model are identified by using the Gauss-Newton algorithm. Based on the estimated model with identified parameters, investigation of system properties is then carried out. The characteristics of the driver neuromuscular system are discussed and compared with respect to different steering tasks, hand positions and driver postures. These experimental results with identified system properties provide a good foundation for the development of a haptic take-over control system for automated vehicles