Recognizing driver braking intention with vehicle data using unsupervised learning methods

Recently, the development of braking assistance system has largely benefit the safety of both driver and pedestrians. A robust prediction and detection of driver braking intention will enable driving assistance system response to traffic situation correctly and improve the driving experience of intelligent vehicles. Unsupervised machine learning algorithms has been widely used in clustering and pattern mining in previous researches. In this paper, a various unsupervised clustering methods will be used to build a driver braking intention predictor which can accurately recognize the braking maneuver based on vehicle data captured with CAN bus. The braking maneuver along with other driving maneuvers such as normal driving will be clustered and the results from different methods like K-means and Gaussian mixture model will be compared. Additionally, the evaluation of features from raw data, which are important to driving maneuvers clustering will be proposed. The experiment data are collected from one hybrid electric vehicle in real world. Final results show that the proposed method can detect driver’s braking intention in a very beginning moment with a high accuracy and the most important sets of feature for driving maneuver clustering will be discussed.Recently, the development of braking assistance system has largely benefit the safety of both driver and pedestrians. A robust prediction and detection of driver braking intention will enable driving assistance system response to traffic situation correctly and improve the driving experience of intelligent vehicles. Unsupervised machine learning algorithms has been widely used in clustering and pattern mining in previous researches. In this paper, a various unsupervised clustering methods will be used to build a driver braking intention predictor which can accurately recognize the braking maneuver based on vehicle data captured with CAN bus. The braking maneuver along with other driving maneuvers such as normal driving will be clustered and the results from different methods like K-means and Gaussian mixture model will be compared. Additionally, the evaluation of features from raw data, which are important to driving maneuvers clustering will be proposed. The experiment data are collected from one hybrid electric vehicle in real world. Final results show that the proposed method can detect driver’s braking intention in a very beginning moment with a high accuracy and the most important sets of feature for driving maneuver clustering will be discussed

Characterization of driver neuromuscular dynamics for human-automation collaboration design of automated vehicles

In order to design an advanced human-automation collaboration system for highly automated vehicles, research into the driver's neuromuscular dynamics is needed. In this paper a dynamic model of drivers' neuromuscular interaction with a steering wheel is firstly established. The transfer function and the natural frequency of the systems are analyzed. In order to identify the key parameters of the driver-steering-wheel interacting system and investigate the system properties under different situations, experiments with driver-in-the-loop are carried out. For each test subject, two steering tasks, namely the passive and active steering tasks, are instructed to be completed. Furthermore, during the experiments, subjects manipulated the steering wheel with two distinct postures and three different hand positions. Based on the experimental results, key parameters of the transfer function model are identified by using the Gauss-Newton algorithm. Based on the estimated model with identified parameters, investigation of system properties is then carried out. The characteristics of the driver neuromuscular system are discussed and compared with respect to different steering tasks, hand positions and driver postures. These experimental results with identified system properties provide a good foundation for the development of a haptic take-over control system for automated vehicles

Intelligent Operation System for the Autonomous Vehicle Fleet

Modular vehicles are vehicles with interchangeable substantial components also known as modules. Fleet modularity provides extra operational flexibility through on-field actions, in terms of vehicle assembly, disassembly, and reconfiguration (ADR). The ease of assembly and disassembly of modular vehicles enables them to achieve real-time fleet reconfiguration, which is proven as beneficial in promoting fleet adaptability and in saving ownership costs. The objective of military fleet operation is to satisfy uncertain demands on time while providing vehicle maintenance. To quantify the benefits and burdens from modularity in military operation, a decision support system is required to yield autonomously operation strategies for comparing the (near) optimal fleet performance for different vehicle architectures under diverse scenarios. The problem is challenging because: 1) fleet operation strategies are numerous, especially when modularity is considered; 2) operation actions are time-delayed and time-varying; 3) vehicle damages and demands are highly uncertain; 4) available capacity for ADR actions and vehicle repair is constrained. Finally, to explore advanced tactics enabled by fleet modularity, the competition between human-like and adversarial forces is required, where each force is capable to autonomously perceive and analyze field information, learn enemy's behavior, forecast enemy's actions, and prepare an operation plan accordingly. Currently, methodologies developed specifically for fleet competition are only valid for single type of resources and simple operation rules, which are impossible to implement in modular fleet operation. This dissertation focuses on a new general methodology to yield decisions in operating a fleet of autonomous military vehicles/robots in both conventional and modular architectures. First, a stochastic state space model is created to represent the changes in fleet dynamics caused by operation actions. Then, a stochastic model predictive control is customized to manage the system dynamics, which is capable of real-time decision making. Including modularity increases the complexity of fleet operation problem, a novel intelligent agent based model is proposed to ensure the computational efficiency and also imitate the collaborative decisions making process of human-like commanders. Operation decisions are distributed to several agents with distinct responsibility. Agents are designed in a specific way to collaboratively make and adjust decisions through selectively sharing information, reasoning the causality between events, and learning the other's behavior, which are achieved by real-time optimization and artificial intelligence techniques. To evaluate the impacts from fleet modularity, three operation problems are formulated: (i) simplified logistic mission scenario: operate a fleet to guarantee the readiness of vehicles at battlefields considering the stochasticity in inventory stocks and mission requirements; (ii) tactical mission scenario: deliver resources to battlefields with stochastic requirements of vehicle repairs and maintenance; (iii) attacker-defender game: satisfy the mission requirements with minimized losses caused by uncertain assaults from an enemy. The model is also implemented for a civilian application, namely the real-time management of reconfigurable manufacturing systems (RMSs). As the number of RMS configurations increases exponentially with the size of the line and demand changes frequently, two challenges emerge: how to efficiently select the optimal configuration given limited resources, and how to allocate resources among lines. According to the ideas in modular fleet operation, a new mathematical approach is presented for distributing the stochastic demands and exchanging machines or modules among lines (which are groups of machines) as a bidding process, and for adaptively configuring these lines and machines for the resulting shared demand under a limited inventory of configurable components.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147588/1/lixingyu_2.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/147588/2/lixingyu_1.pd

Influence of accelerator pedal force feedback on truck drivers' speed control

The UK government has set clear targets for 80% reductions (compared with 1990 levels) in greenhouse gas emissions by 2050 and pressure is increasing on the road transport industry to reduce the fuel consumption and harmful exhaust emissions of Heavy Goods Vehicles (HGVs). Vehicle manufacturers and operators alike are having to investigate and find new ways of making reductions. It is thought that improving driver behaviour offers significant potential for these reductions in fuel consumption and emissions. This thesis considers the use of Active Accelerator Pedals (AAPs) and the potential for improved driver performance that they may offer by providing pedal force feedback to the driver. In order to develop understanding of the interactions between the human driver and accelerator pedal, two near identical tractor units, operated by Turners of Soham Ltd, were fitted with the a data logger. Data was collected and stored over a period of four months as they operated on the road. This data provided the basis for a vehicle model to be developed using real-world conditions, rather than strictly controlled test track conditions. Analysis of the behaviour of the two drivers also identified differences is styles, and explained 7% fuel consumption differences between the two drivers when negotiating roundabouts. A new mathematical model of the human driver’s longitudinal control was also developed to include the driver’s cognitive control of the accelerator pedal. Model Predictive Control theory, commonly used for modelling the driver’s steering control, was used and different driving styles were replicated by varying the weightings in a cost function, and a series of driving simulator experiments were performed to validate the model. Nine human drivers, two of which were professionals, performed two driving scenarios (drive cycle and car-following). The driver model was fitted to each driver individually to mathematically express the differences in their styles. The simulated RMS pedal forces from the fitted driver models lay within 20% of the measured simulator values. The driver model was also extended to include the interactions between a human driver and an AAP using mathematical game theory. Three frameworks were proposed: decentralised, cooperative and one-sided cooperative, but, as the cooperative framework would have been very difficult to implement experimentally, it was only considered theoretically. The same nine human drivers were presented with drive cycle and car-following scenarios whilst being assisted by pedal feedback to validate the model. Both decentralised and one-sided cooperative frameworks were applied to the fitting and compared. In the drive cycle scenario, the one-sided cooperative framework output an identical controller to the decentralised framework. In the car-following scenario, the one-sided cooperative framework produced the best fit, suggesting that the human drivers adapted their strategy to reflect the guidance from the AAP. It was noted in both scenarios that the peak pedal displacement decreased by approximately 20% with the presence of pedal force feedback. Further work is suggested to improve the mass and road gradient data obtained from the data loggers in vehicles in order to reduce the uncertainty in the traction force and fuel rate maps. With a model for the interactions of a human driver with an AAP now in place, the pedal feedback strategy can now be optimised to improve the performance of the human driver.Centre for Sustainable Road Freight and EPSR

Development of cognitive workload models to detect driving impairment

Tesi redactada en castellàDriving a vehicle is a complex activity exposed to continuous changes such as speed limits and vehicular traffic. Drivers require a high degree of concentration when performing this activity, increasing the amount of mental demand known as cognitive workload, causing vehicular accidents to the minimum negligence. In fact, human error is the leading contributing factor in over 90% of road accidents. In recent years, the subjects' cognitive workload levels while driving a vehicle have been predicted using subjective and vehicle performance tools. Other research has emphasized the use and analysis of physiological information, where electroencephalographic (EEG) signals are the most used to identify cognitive states due to their high precision. Although significant progress has been made in this area, these investigations have been based on traditional techniques or data analysis from a specific source due to the information's complexity. A new trend has been opened in the study of the internal behavior of subjects by implementing machine learning techniques to analyze information from various sources. However, there are still several challenges to face in this new line of research. This doctoral thesis presents a new model to predict the states of low and high cognitive workload of subjects when facing scenarios of driving a vehicle called GALoRSI-SVMRBF (Genetic Algorithms and Logistic Regression for the Structuring of Information-Support Vector Machine with Radial Basis Function Kernel). GALoRSI-SVMRBF is developed using machine learning algorithms based on information from EEG signals. Also, the information collected from NASA-TLX, instant online self-assessment and the error rate measure are implemented in the model. First, GALoRSI-SVMRBF proposes a new method for pattern recognition based on feature selection that combines statistical tests, genetic algorithms, and logistic regression. This method consists mainly of selecting an EEG dataset and exploring the information to identify the key features that recognize cognitive states. The selected data are defined as an index for pattern recognition and used to structure a new dataset capable of optimizing the model's learning and classification process. Second, the methodology and development of a classifier for the prediction model are presented, implementing machine learning algorithms. The classifier is developed mainly in two phases, defined as training and testing. Once the prediction model has been developed, this thesis presents the validation phase of GALoRSI-SVMRBF. The validation consists of evaluating the model's adaptability to new datasets, maintaining a high prediction rate. Finally, an analysis of the performance of GALoRSI-SVMRBF is presented. The objective is to know the model's scope and limitations, evaluating various performance metrics to find the optimal configuration for GALoRSI-SVMRBF. We found that GALoRSI-SVMRBF successfully predicts low and high cognitive workload of subjects while driving a vehicle. In general, it is observed that the model uses the information extracted from multiple EEG signals, reducing the original dataset by more than 50%, maximizing its predictive capacity, achieving a precision rate of >90% in the classification of the information. During this thesis, the experiments showed that obtaining a high percentage of prediction depends on several factors, from applying a useful collection technique data until the last step of the prediction model.La conducción de un vehículo es una actividad compleja que está expuesta a demandas que cambian continuamente por diferentes factores, tales como, el límite de velocidad, obstáculos en la vía, tráfico vehicular, entre otros. Al desempeñar esta actividad, los conductores requieren un alto grado de concentración incrementando la cantidad de demanda mental conocida como carga. En los últimos años, se han propuesto mecanismos para monitorear y/o predecir los niveles de carga cognitiva de los sujetos al conducir un vehículo, centrándose en el uso de herramientas subjetivas y de rendimiento vehicular. Otras investigaciones, han enfatizado en el uso y análisis de la información fisiológica, siendo las señales electroencefalográficas (EEG) las más utilizadas para identificar los estados cognitivos por su alta precisión. A pesar del gran avance realizado, estas investigaciones se han basado en técnicas tradicionales o en el análisis de la información proveniente de fuentes específicas para identificar el estado interno del sujeto, obteniendo modelos sobreentrenados o robustos, incrementando el tiempo de análisis afectando el desempeño del modelo. En esta tesis doctoral se presenta un nuevo modelo para predecir los estados de baja y alta carga cognitiva de los sujetos al enfrentarse a escenarios de la conducción de un vehículo denominado GALoRSI-SVMRBF (Genetic Algorithms and Logistic Regression for the Structuring of Information-Support Vector Machine with Radial Basis Function Kernel). GALoRSI-SVMRBF fue desarrollado utilizando los algoritmos de aprendizaje automático y técnicas estadísticas basado en la información proveniente de las señales EEG. Primero, GALoRSI-SVMRBF crea una base de datos extrayendo las características que serán utilizadas en el modelo a través de técnicas estadísticas. Posteriormente, propone un nuevo método para el reconocimiento de patrones basado en la selección de características que combina pruebas estadísticas, algoritmos genéticos y regresión logística. Este método consiste principalmente en seleccionar un conjunto de datos EEG y explorar la combinación de la información para identificar las características claves que contribuyan al reconocimiento de dos estados cognitivos. Después, la información seleccionada es definida como un índice para el reconocimiento de patrones y utilizada para estructurar un nuevo conjunto de datos que soporta información de uno o múltiples canales para optimizar el proceso de aprendizaje y clasificación del modelo. Por último, es desarrollado el clasificador del modelo de predicciones el cual consiste en dos etapas definidas como entrenamiento y prueba. Nosotros encontramos que GALoRSI-SVMRBF predice de manera exitosa la carga cognitiva baja y alta de los sujetos durante la conducción de un vehículo. En general, se observó que el modelo utiliza la información extraída de una o múltiples señales EEG y logrando una tasa de precisión >90% en la clasificación de la informaciónPostprint (published version

Vehicle Stability Control Considering the Driver-in-the-Loop

A driver‐in‐the‐loop modeling framework is essential for a full analysis of vehicle stability systems. In theory, knowing the vehicle’s desired path (driver’s intention), the problem is reduced to a standard control system in which one can use different methods to produce a (sub) optimal solution. In practice, however, estimation of a driver’s desired path is a challenging – if not impossible – task. In this thesis, a new formulation of the problem that integrates the driver and the vehicle model is proposed to improve vehicle performance without using additional information from the future intention of the driver. The driver’s handling technique is modeled as a general function of the road preview information as well as the dynamic states of the vehicle. In order to cover a variety of driving styles, the time‐ varying cumulative driver's delay and model uncertainties are included in the formulation. Given that for practical implementations, the driver’s future road preview data is not accessible, this information is modeled as bounded uncertainties. Subsequently, a state feedback controller is designed to counteract the negative effects of a driver’s lag while makes the system robust to modeling and process uncertainties. The vehicle’s performance is improved by redesigning the controller to consider a parameter varying model of the driver‐vehicle system. An LPV controller robust to unknown time‐varying delay is designed and the disturbance attenuation of the closed loop system is estimated. An approach is constructed to identify the time‐varying parameters of the driver model using past driving information. The obtained gains are clustered into several modes and the transition probability of switching between different driving‐styles (modes) is calculated. Based on this analysis, the driver‐vehicle system is modeled as a Markovian jump dynamical system. Moreover, a complementary analysis is performed on the convergence properties of the mode‐dependent controller and a tighter estimation for the maximum level of disturbance rejection of the LPV controller is obtained. In addition, the effect of a driver’s skills in controlling the vehicle while the tires are saturated is analyzed. A guideline for analysis of the nonlinear system performance with consideration to the driver’s skills is suggested. Nonlinear controller design techniques are employed to attenuate the undesirable effects of both model uncertainties and tire saturation

Driver lane change intention inference using machine learning methods.

Lane changing manoeuvre on highway is a highly interactive task for human drivers. The intelligent vehicles and the advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver. The ADAS also need to understand the driver potential intent correctly since it shares the control authority with the human driver. This study provides a research on the driver intention inference, particular focus on the lane change manoeuvre on highways. This report is organised in a paper basis, where each chapter corresponding to a publication, which is submitted or to be submitted. Part Ⅰ introduce the motivation and general methodology framework for this thesis. Part Ⅱ includes the literature survey and the state-of-art of driver intention inference. Part Ⅲ contains the techniques for traffic context perception that focus on the lane detection. A literature review on lane detection techniques and its integration with parallel driving framework is proposed. Next, a novel integrated lane detection system is designed. Part Ⅳ contains two parts, which provides the driver behaviour monitoring system for normal driving and secondary tasks detection. The first part is based on the conventional feature selection methods while the second part introduces an end-to-end deep learning framework. The design and analysis of driver lane change intention inference system for the lane change manoeuvre is proposed in Part Ⅴ. Finally, discussions and conclusions are made in Part Ⅵ. A major contribution of this project is to propose novel algorithms which accurately model the driver intention inference process. Lane change intention will be recognised based on machine learning (ML) methods due to its good reasoning and generalizing characteristics. Sensors in the vehicle are used to capture context traffic information, vehicle dynamics, and driver behaviours information. Machine learning and image processing are the techniques to recognise human driver behaviour.PhD in Transpor