Driver lane change intention inference for intelligent vehicles: framework, survey, and challenges

Intelligent vehicles and advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver status since ADAS share the vehicle control authorities with the human driver. This study provides an overview of the ego-vehicle driver intention inference (DII), which mainly focus on the lane change intention on highways. First, a human intention mechanism is discussed in the beginning to gain an overall understanding of the driver intention. Next, the ego-vehicle driver intention is classified into different categories based on various criteria. A complete DII system can be separated into different modules, which consists of traffic context awareness, driver states monitoring, and the vehicle dynamic measurement module. The relationship between these modules and the corresponding impacts on the DII are analyzed. Then, the lane change intention inference (LCII) system is reviewed from the perspective of input signals, algorithms, and evaluation. Finally, future concerns and emerging trends in this area are highlighted

Driver lane change intention inference using machine learning methods.

Lane changing manoeuvre on highway is a highly interactive task for human drivers. The intelligent vehicles and the advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver. The ADAS also need to understand the driver potential intent correctly since it shares the control authority with the human driver. This study provides a research on the driver intention inference, particular focus on the lane change manoeuvre on highways. This report is organised in a paper basis, where each chapter corresponding to a publication, which is submitted or to be submitted. Part Ⅰ introduce the motivation and general methodology framework for this thesis. Part Ⅱ includes the literature survey and the state-of-art of driver intention inference. Part Ⅲ contains the techniques for traffic context perception that focus on the lane detection. A literature review on lane detection techniques and its integration with parallel driving framework is proposed. Next, a novel integrated lane detection system is designed. Part Ⅳ contains two parts, which provides the driver behaviour monitoring system for normal driving and secondary tasks detection. The first part is based on the conventional feature selection methods while the second part introduces an end-to-end deep learning framework. The design and analysis of driver lane change intention inference system for the lane change manoeuvre is proposed in Part Ⅴ. Finally, discussions and conclusions are made in Part Ⅵ. A major contribution of this project is to propose novel algorithms which accurately model the driver intention inference process. Lane change intention will be recognised based on machine learning (ML) methods due to its good reasoning and generalizing characteristics. Sensors in the vehicle are used to capture context traffic information, vehicle dynamics, and driver behaviours information. Machine learning and image processing are the techniques to recognise human driver behaviour.PhD in Transpor

INQUIRIES IN INTELLIGENT INFORMATION SYSTEMS: NEW TRAJECTORIES AND PARADIGMS

Rapid Digital transformation drives organizations to continually revitalize their business models so organizations can excel in such aggressive global competition. Intelligent Information Systems (IIS) have enabled organizations to achieve many strategic and market leverages. Despite the increasing intelligence competencies offered by IIS, they are still limited in many cognitive functions. Elevating the cognitive competencies offered by IIS would impact the organizational strategic positions. With the advent of Deep Learning (DL), IoT, and Edge Computing, IISs has witnessed a leap in their intelligence competencies. DL has been applied to many business areas and many industries such as real estate and manufacturing. Moreover, despite the complexity of DL models, many research dedicated efforts to apply DL to limited computational devices, such as IoTs. Applying deep learning for IoTs will turn everyday devices into intelligent interactive assistants. IISs suffer from many challenges that affect their service quality, process quality, and information quality. These challenges affected, in turn, user acceptance in terms of satisfaction, use, and trust. Moreover, Information Systems (IS) has conducted very little research on IIS development and the foreseeable contribution for the new paradigms to address IIS challenges. Therefore, this research aims to investigate how the employment of new AI paradigms would enhance the overall quality and consequently user acceptance of IIS. This research employs different AI paradigms to develop two different IIS. The first system uses deep learning, edge computing, and IoT to develop scene-aware ridesharing mentoring. The first developed system enhances the efficiency, privacy, and responsiveness of current ridesharing monitoring solutions. The second system aims to enhance the real estate searching process by formulating the search problem as a Multi-criteria decision. The system also allows users to filter properties based on their degree of damage, where a deep learning network allocates damages in 12 each real estate image. The system enhances real-estate website service quality by enhancing flexibility, relevancy, and efficiency. The research contributes to the Information Systems research by developing two Design Science artifacts. Both artifacts are adding to the IS knowledge base in terms of integrating different components, measurements, and techniques coherently and logically to effectively address important issues in IIS. The research also adds to the IS environment by addressing important business requirements that current methodologies and paradigms are not fulfilled. The research also highlights that most IIS overlook important design guidelines due to the lack of relevant evaluation metrics for different business problems

A Context Aware Classification System for Monitoring Driver’s Distraction Levels

Understanding the safety measures regarding developing self-driving futuristic cars is a concern for decision-makers, civil society, consumer groups, and manufacturers. The researchers are trying to thoroughly test and simulate various driving contexts to make these cars fully secure for road users. Including the vehicle’ surroundings offer an ideal way to monitor context-aware situations and incorporate the various hazards. In this regard, different studies have analysed drivers’ behaviour under different case scenarios and scrutinised the external environment to obtain a holistic view of vehicles and the environment. Studies showed that the primary cause of road accidents is driver distraction, and there is a thin line that separates the transition from careless to dangerous. While there has been a significant improvement in advanced driver assistance systems, the current measures neither detect the severity of the distraction levels nor the context-aware, which can aid in preventing accidents. Also, no compact study provides a complete model for transitioning control from the driver to the vehicle when a high degree of distraction is detected. The current study proposes a context-aware severity model to detect safety issues related to driver’s distractions, considering the physiological attributes, the activities, and context-aware situations such as environment and vehicle. Thereby, a novel three-phase Fast Recurrent Convolutional Neural Network (Fast-RCNN) architecture addresses the physiological attributes. Secondly, a novel two-tier FRCNN-LSTM framework is devised to classify the severity of driver distraction. Thirdly, a Dynamic Bayesian Network (DBN) for the prediction of driver distraction. The study further proposes the Multiclass Driver Distraction Risk Assessment (MDDRA) model, which can be adopted in a context-aware driving distraction scenario. Finally, a 3-way hybrid CNN-DBN-LSTM multiclass degree of driver distraction according to severity level is developed. In addition, a Hidden Markov Driver Distraction Severity Model (HMDDSM) for the transitioning of control from the driver to the vehicle when a high degree of distraction is detected. This work tests and evaluates the proposed models using the multi-view TeleFOT naturalistic driving study data and the American University of Cairo dataset (AUCD). The evaluation of the developed models was performed using cross-correlation, hybrid cross-correlations, K-Folds validation. The results show that the technique effectively learns and adopts safety measures related to the severity of driver distraction. In addition, the results also show that while a driver is in a dangerous distraction state, the control can be shifted from driver to vehicle in a systematic manner

Mobility and Aging: Older Drivers’ Visual Searching, Lane Keeping and Coordination

This thesis examined older drivers’ mobility and behaviour through comprehensive measurements of driver-vehicle-environment interaction and investigated the associations between driving behaviour and cognitive functions. Data were collected and analysed for 50 older drivers using eye tracking, GNSS tracking, and GIS. Results showed that poor selective attention, spatial ability and executive function in older drivers adversely affect lane keeping, visual search and coordination. Visual-motor coordination measure is sensitive and effective for driving assessment in older drivers

Constraint-based navigation for safe, shared control of ground vehicles

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 138-147).Human error in machine operation is common and costly. This thesis introduces, develops, and experimentally demonstrates a new paradigm for shared-adaptive control of human-machine systems that mitigates the effects of human error without removing humans from the control loop. Motivated by observed human proclivity toward navigation in fields of safe travel rather than along specific trajectories, the planning and control framework developed in this thesis is rooted in the design and enforcement of constraints rather than the more traditional use of reference paths. Two constraint-planning methods are introduced. The first uses a constrained Delaunay triangulation of the environment to identify, cumulatively evaluate, and succinctly circumscribe the paths belonging to a particular homotopy with a set of semi autonomously enforceable constraints on the vehicle's position. The second identifies a desired homotopy by planning - and then laterally expanding - the optimal path that traverses it. Simulated results show both of these constraint-planning methods capable of improving the performance of one or multiple agents traversing an environment with obstacles. A method for predicting the threat posed to the vehicle given the current driver action, present state of the environment, and modeled vehicle dynamics is also presented. This threat assessment method, and the shared control approach it facilitates, are shown in simulation to prevent constraint violation or vehicular loss of control with minimal control intervention. Visual and haptic driver feedback mechanisms facilitated by this constraint-based control and threat-based intervention are also introduced. Finally, a large-scale, repeated measures study is presented to evaluate this control framework's effect on the performance, confidence, and cognitive workload of 20 drivers teleoperating an unmanned ground vehicle through an outdoor obstacle course. In 1,200 trials, the constraint-based framework developed in this thesis is shown to increase vehicle velocity by 26% while reducing the occurrence of collisions by 78%, improving driver reaction time to a secondary task by 8.7%, and increasing overall user confidence and sense of control by 44% and 12%, respectively. These performance improvements were realized with the autonomous controller usurping less than 43% of available vehicle control authority, on average.by Sterling J. Anderson.Ph.D

Prediction of drivers’ performance in highly automated vehicles

Purpose: The aim of this research was to assess the predictability of driver’s response to critical hazards during the transition from automated to manual driving in highly automated vehicles using their physiological data.Method: A driving simulator experiment was conducted to collect drivers’ physiological data before, during and after the transition from automated to manual driving. A total of 33 participants between 20 and 30 years old were recruited. Participants went through a driving scenario under the influence of different non-driving related tasks. The repeated measures approach was used to assess the effect of repeatability on the driver’s physiological data. Statistical and machine learning methods were used to assess the predictability of drivers’ response quality based on their physiological data collected before responding to a critical hazard. Findings: - The results showed that the observed physiological data that was gathered before the transition formed strong indicators of the drivers’ ability to respond successfully to a potential hazard after the transition. In addition, physiological behaviour was influenced by driver’s secondary tasks engagement and correlated with the driver’s subjective measures to the difficulty of the task. The study proposes new quality measures to assess the driver’s response to critical hazards in highly automated driving. Machine learning results showed that response time is predictable using regression methods. In addition, the classification methods were able to classify drivers into low, medium and high-risk groups based on their quality measures values. Research Implications: Proposed models help increase the safety of automated driving systems by providing insights into the drivers’ ability to respond to future critical hazards. More research is required to find the influence of age, drivers’ experience of the automated vehicles and traffic density on the stability of the proposed models. Originality: The main contribution to knowledge of this study is the feasibility of predicting drivers’ ability to respond to critical hazards using the physiological behavioural data collected before the transition from automated to manual driving. With the findings, automation systems could change the transition time based on the driver’s physiological state to allow for the safest transition possible. In addition, it provides an insight into driver’s readiness and therefore, allows the automated system to adopt the correct driving strategy and plan to enhance drivers experience and make the transition phase safer for everyone.</div

Multimodal machine learning for intelligent mobility

Scientific problems are solved by finding the optimal solution for a specific task. Some problems can be solved analytically while other problems are solved using data driven methods. The use of digital technologies to improve the transportation of people and goods, which is referred to as intelligent mobility, is one of the principal beneficiaries of data driven solutions. Autonomous vehicles are at the heart of the developments that propel Intelligent Mobility. Due to the high dimensionality and complexities involved in real-world environments, it needs to become commonplace for intelligent mobility to use data-driven solutions. As it is near impossible to program decision making logic for every eventuality manually. While recent developments of data-driven solutions such as deep learning facilitate machines to learn effectively from large datasets, the application of techniques within safety-critical systems such as driverless cars remain scarce.Autonomous vehicles need to be able to make context-driven decisions autonomously in different environments in which they operate. The recent literature on driverless vehicle research is heavily focused only on road or highway environments but have discounted pedestrianized areas and indoor environments. These unstructured environments tend to have more clutter and change rapidly over time. Therefore, for intelligent mobility to make a significant impact on human life, it is vital to extend the application beyond the structured environments. To further advance intelligent mobility, researchers need to take cues from multiple sensor streams, and multiple machine learning algorithms so that decisions can be robust and reliable. Only then will machines indeed be able to operate in unstructured and dynamic environments safely. Towards addressing these limitations, this thesis investigates data driven solutions towards crucial building blocks in intelligent mobility. Specifically, the thesis investigates multimodal sensor data fusion, machine learning, multimodal deep representation learning and its application of intelligent mobility. This work demonstrates that mobile robots can use multimodal machine learning to derive driver policy and therefore make autonomous decisions.To facilitate autonomous decisions necessary to derive safe driving algorithms, we present an algorithm for free space detection and human activity recognition. Driving these decision-making algorithms are specific datasets collected throughout this study. They include the Loughborough London Autonomous Vehicle dataset, and the Loughborough London Human Activity Recognition dataset. The datasets were collected using an autonomous platform design and developed in house as part of this research activity. The proposed framework for Free-Space Detection is based on an active learning paradigm that leverages the relative uncertainty of multimodal sensor data streams (ultrasound and camera). It utilizes an online learning methodology to continuously update the learnt model whenever the vehicle experiences new environments. The proposed Free Space Detection algorithm enables an autonomous vehicle to self-learn, evolve and adapt to new environments never encountered before. The results illustrate that online learning mechanism is superior to one-off training of deep neural networks that require large datasets to generalize to unfamiliar surroundings. The thesis takes the view that human should be at the centre of any technological development related to artificial intelligence. It is imperative within the spectrum of intelligent mobility where an autonomous vehicle should be aware of what humans are doing in its vicinity. Towards improving the robustness of human activity recognition, this thesis proposes a novel algorithm that classifies point-cloud data originated from Light Detection and Ranging sensors. The proposed algorithm leverages multimodality by using the camera data to identify humans and segment the region of interest in point cloud data. The corresponding 3-dimensional data was converted to a Fisher Vector Representation before being classified by a deep Convolutional Neural Network. The proposed algorithm classifies the indoor activities performed by a human subject with an average precision of 90.3%. When compared to an alternative point cloud classifier, PointNet[1], [2], the proposed framework out preformed on all classes. The developed autonomous testbed for data collection and algorithm validation, as well as the multimodal data-driven solutions for driverless cars, is the major contributions of this thesis. It is anticipated that these results and the testbed will have significant implications on the future of intelligent mobility by amplifying the developments of intelligent driverless vehicles.</div

In-Vehicle information systems-related multiple task performance and driver

Doutoramento em Motricidade Humana, Especialidade em ErgonomiaThe presence of new technologies inside vehicles is becoming more common. Due to this fact, the potential changes produced on the driving task and also on the road safety must be examined. With the intention of contributing to amplify this knowledge, the present research aimed to study the impact of multiple visual and auditory inputs from in-vehicle information systems on the driver behaviour. It was investigated the interaction with more than one invehicle system (a guidance system and a mobile phone device) and verified its consequences on the drivers’ activity. To accomplish this goal two experimental moments were planned: one conducted in a real context and another in a simulated environment. Results revealed that the interaction with two in-vehicle systems produced considerable changes on drivers’behaviour once subjects assumed more frequently unsafe actions like: inadequate indication of their actions; abrupt and unexpected adoption of determined behaviours; and also negligence of some road information from the environment. It was also verified that this situation produced more severe consequences to the driving task performance of elderly drivers. The management of all sources of information induced them to compromise their safety and be more frequently involved in dangerous situations.O surgimento das novas tecnologias embarcadas e a sua contínua evolução têm alterado o contexto rodoviário. Como consequência, a cada vez maior aceitação e utilização deste tipo de equipamentos tem sido motivo de estudo, uma vez que é essencial conhecer as potenciais alterações produzidas na tarefa de condução e na segurança rodoviária. Com o intuito de contribuir para ampliar o conhecimento relativo a este tema, a presente investigação pretendeu estudar o impacto que múltiplas mensagens visuais e auditivas provenientes de sistemas embarcados possam ter no comportamento do condutor. Foi investigada a interacção com mais que um sistema embarcado (sistema de navegação e telemóvel) e verificadas as consequências na actividade dos condutores. Para cumprir este objectivo, dois momentos experimentais foram desenvolvidos: um em ambiente real e outro em envolvimento simulado. Os resultados revelaram que a interacção com os dois sistemas embarcados produziram alterações consideráveis no comportamento dos condutores uma vez que estes adoptaram mais frequentemente actos inseguros como: indicação inadequada das suas acções; comportamentos bruscos e inesperados; bem como negligência de determinada informação proveniente do envolvimento rodoviário. Foi igualmente verificado que esta situação produziu consequências mais gravosas no desempenho dos condutores idosos. A gestão de todas as fontes de informação impeliu este grupo de condutores a comprometer a sua segurança e a estar mais frequentemente envolvidos em situações perigosas