An ensemble deep learning approach for driver lane change intention inference

With the rapid development of intelligent vehicles, drivers are increasingly likely to share their control authorities with the intelligent control unit. For building an efficient Advanced Driver Assistance Systems (ADAS) and shared-control systems, the vehicle needs to understand the drivers’ intent and their activities to generate assistant and collaborative control strategies. In this study, a driver intention inference system that focuses on the highway lane change maneuvers is proposed. First, a high-level driver intention mechanism and framework are introduced. Then, a vision-based intention inference system is proposed, which captures the multi-modal signals based on multiple low-cost cameras and the VBOX vehicle data acquisition system. A novel ensemble bi-directional recurrent neural network (RNN) model with Long Short-Term Memory (LSTM) units is proposed to deal with the time-series driving sequence and the temporal behavioral patterns. Naturalistic highway driving data that consists of lane-keeping, left and right lane change maneuvers are collected and used for model construction and evaluation. Furthermore, the driver's pre-maneuver activities are statistically analyzed. It is found that for situation-aware, drivers usually check the mirrors for more than six seconds before they initiate the lane change maneuver, and the time interval between steering the handwheel and crossing the lane is about 2 s on average. Finally, hypothesis testing is conducted to show the significant improvement of the proposed algorithm over existing ones. With five-fold cross-validation, the EBiLSTM model achieves an average accuracy of 96.1% for the intention that is inferred 0.5 s before the maneuver starts

Holistic Vehicle Control Using Learning MPC

In recent years, learning MPC schemes have been introduced to address these challenges of traditional MPC. They typically leverage different machine learning techniques to learn the system dynamics directly from data, allowing it to handle model uncertainty more effectively. Besides, they can adapt to changes by continuously updating the learned model using real-time data, ensuring that the controller remains effective even as the system evolves. However, there are some challenges for the existing learning MPC techniques. Firstly, learning-based control approaches often lack interpretability. Understanding and interpreting the learned models and their learning and prediction processes are crucial for safety critical systems such as vehicle stability systems. Secondly, existing learning MPC techniques rely solely on learned models, which might result in poor performance or instability if the model encounters scenarios that differ significantly from the training data. Thirdly, existing learning MPC techniques typically require large amounts of high-quality data for training accurate models, which can be expensive or impractical in the vehicle stability control domain. To address these challenges, this thesis proposes a novel hybrid learning MPC approach for HVC. The main objective is to leverage the capabilities of machine learning algorithms to learn accurate and adaptive models of vehicle dynamics from data, enabling enhanced control strategies for improved stability and maneuverability. The hybrid learning MPC scheme maintains a traditional physics-based vehicle model and a data-based learning model. In the learned model, a variety of machine-learning techniques can be used to predict vehicle dynamics based on learning from collected vehicle data. The performance of the developed hybrid learning MPC controller using torque vectoring (TV) as the actuator is evaluated through the Matlab/Simulink and CarSim co-simulation with a high-fidelity Chevy Equinox vehicle model under a series of harsh maneuvers. Extensive real-world experiments using a Chevy Equinox electric testing vehicle are conducted. Both simulation results and experimental results show that the developed hybrid learning MPC approach consistently outperforms existing MPC methods with better yaw rate tracking performance and smaller vehicle sideslip under various driving conditions

Vehicle Stability Control Considering the Driver-in-the-Loop

A driver‐in‐the‐loop modeling framework is essential for a full analysis of vehicle stability systems. In theory, knowing the vehicle’s desired path (driver’s intention), the problem is reduced to a standard control system in which one can use different methods to produce a (sub) optimal solution. In practice, however, estimation of a driver’s desired path is a challenging – if not impossible – task. In this thesis, a new formulation of the problem that integrates the driver and the vehicle model is proposed to improve vehicle performance without using additional information from the future intention of the driver. The driver’s handling technique is modeled as a general function of the road preview information as well as the dynamic states of the vehicle. In order to cover a variety of driving styles, the time‐ varying cumulative driver's delay and model uncertainties are included in the formulation. Given that for practical implementations, the driver’s future road preview data is not accessible, this information is modeled as bounded uncertainties. Subsequently, a state feedback controller is designed to counteract the negative effects of a driver’s lag while makes the system robust to modeling and process uncertainties. The vehicle’s performance is improved by redesigning the controller to consider a parameter varying model of the driver‐vehicle system. An LPV controller robust to unknown time‐varying delay is designed and the disturbance attenuation of the closed loop system is estimated. An approach is constructed to identify the time‐varying parameters of the driver model using past driving information. The obtained gains are clustered into several modes and the transition probability of switching between different driving‐styles (modes) is calculated. Based on this analysis, the driver‐vehicle system is modeled as a Markovian jump dynamical system. Moreover, a complementary analysis is performed on the convergence properties of the mode‐dependent controller and a tighter estimation for the maximum level of disturbance rejection of the LPV controller is obtained. In addition, the effect of a driver’s skills in controlling the vehicle while the tires are saturated is analyzed. A guideline for analysis of the nonlinear system performance with consideration to the driver’s skills is suggested. Nonlinear controller design techniques are employed to attenuate the undesirable effects of both model uncertainties and tire saturation

Predictive Model of Driver\u27s Eye Fixation for Maneuver Prediction in the Design of Advanced Driving Assistance Systems

Over the last few years, Advanced Driver Assistance Systems (ADAS) have been shown to significantly reduce the number of vehicle accidents. Accord- ing to the National Highway Traffic Safety Administration (NHTSA), driver errors contribute to 94% of road collisions. This research aims to develop a predictive model of driver eye fixation by analyzing the driver eye and head information (cephalo-ocular) for maneuver prediction in an Advanced Driving Assistance System (ADAS). Several ADASs have been developed to help drivers to perform driving tasks in complex environments and many studies were conducted on improving automated systems. Some research has relied on the fact that the driver plays a crucial role in most driving scenarios, recognizing the driver’s role as the central element in ADASs. The way in which a driver monitors the surrounding environment is at least partially descriptive of the driver’s situation awareness. This thesis’s primary goal is the quantitative and qualitative analysis of driver behavior to determine the relationship between driver intent and actions. The RoadLab initiative provided an instrumented vehicle equipped with an on-board diagnostic system, an eye-gaze tracker, and a stereo vision system for the extraction of relevant features from the driver, the vehicle, and the environment. Several driver behavioral features are investigated to determine whether there is a relevant relation between the driver’s eye fixations and the prediction of driving maneuvers

Constraint-based navigation for safe, shared control of ground vehicles

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 138-147).Human error in machine operation is common and costly. This thesis introduces, develops, and experimentally demonstrates a new paradigm for shared-adaptive control of human-machine systems that mitigates the effects of human error without removing humans from the control loop. Motivated by observed human proclivity toward navigation in fields of safe travel rather than along specific trajectories, the planning and control framework developed in this thesis is rooted in the design and enforcement of constraints rather than the more traditional use of reference paths. Two constraint-planning methods are introduced. The first uses a constrained Delaunay triangulation of the environment to identify, cumulatively evaluate, and succinctly circumscribe the paths belonging to a particular homotopy with a set of semi autonomously enforceable constraints on the vehicle's position. The second identifies a desired homotopy by planning - and then laterally expanding - the optimal path that traverses it. Simulated results show both of these constraint-planning methods capable of improving the performance of one or multiple agents traversing an environment with obstacles. A method for predicting the threat posed to the vehicle given the current driver action, present state of the environment, and modeled vehicle dynamics is also presented. This threat assessment method, and the shared control approach it facilitates, are shown in simulation to prevent constraint violation or vehicular loss of control with minimal control intervention. Visual and haptic driver feedback mechanisms facilitated by this constraint-based control and threat-based intervention are also introduced. Finally, a large-scale, repeated measures study is presented to evaluate this control framework's effect on the performance, confidence, and cognitive workload of 20 drivers teleoperating an unmanned ground vehicle through an outdoor obstacle course. In 1,200 trials, the constraint-based framework developed in this thesis is shown to increase vehicle velocity by 26% while reducing the occurrence of collisions by 78%, improving driver reaction time to a secondary task by 8.7%, and increasing overall user confidence and sense of control by 44% and 12%, respectively. These performance improvements were realized with the autonomous controller usurping less than 43% of available vehicle control authority, on average.by Sterling J. Anderson.Ph.D

A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

In this paper a review is presented of the research on eye gaze estimation techniques and applications, that has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for performance evaluation of gaze tracking systems and achieve consistency in their specification and comparative evaluation. To address this need, the concept of a methodological framework for practical evaluation of different gaze tracking systems is proposed.Comment: 25 pages, 13 figures, Accepted for publication in IEEE Access in July 201