Advancing Deep Learning-based Driver Intention Recognition : Towards a safe integration framework of high-risk AI systems

Progress in artificial intelligence (AI), onboard computation capabilities, and the integration of advanced sensors in cars have facilitated the development of Advanced Driver Assistance Systems (ADAS). These systems aim to continuously minimize human driving errors. {An example application of an ADAS could be to support a human driver by informing if an intended driving maneuver is safe to pursue given the current state of the driving environment. One of the components enabling such an ADAS is recognizing the driver's intentions. Driver intention recognition (DIR) concerns the identification of what driving maneuver a driver aspires to perform in the near future, commonly spanning a few seconds. A challenging aspect of integrating such a system into a car is the ability of the ADAS to handle unseen scenarios. Deploying any AI-based system in an environment where mistakes can cause harm to human beings is considered a high-risk AI system. Upcoming AI regulations require a car manufacturer to motivate the design, performance-complexity trade-off, and the understanding of potential blind spots of a high-risk AI system.} Therefore, this licentiate thesis focuses on AI-based DIR systems and presents an overview of the current state of the DIR research field. Additionally, experimental results are included that demonstrate the process of empirically motivating and evaluating the design of deep neural networks for DIR. To avoid the reliance on sequential Monte Carlo sampling techniques to produce an uncertainty estimation, we evaluated a surrogate model to reproduce uncertainty estimations learned from probabilistic deep-learning models. Lastly, to contextualize the results within the broader scope of safely integrating future high-risk AI-based systems into a car, we propose a foundational conceptual framework.Ett av tre delarbeten (övriga se rubriken Delarbeten/List of papers):Vellenga, Koen, H. Joe Steinhauer et al. (2024). "Designing deep neural networks for driver intention recognition". Under submission.</p

Advancing Deep Learning-based Driver Intention Recognition : Towards a safe integration framework of high-risk AI systems

Progress in artificial intelligence (AI), onboard computation capabilities, and the integration of advanced sensors in cars have facilitated the development of Advanced Driver Assistance Systems (ADAS). These systems aim to continuously minimize human driving errors. {An example application of an ADAS could be to support a human driver by informing if an intended driving maneuver is safe to pursue given the current state of the driving environment. One of the components enabling such an ADAS is recognizing the driver's intentions. Driver intention recognition (DIR) concerns the identification of what driving maneuver a driver aspires to perform in the near future, commonly spanning a few seconds. A challenging aspect of integrating such a system into a car is the ability of the ADAS to handle unseen scenarios. Deploying any AI-based system in an environment where mistakes can cause harm to human beings is considered a high-risk AI system. Upcoming AI regulations require a car manufacturer to motivate the design, performance-complexity trade-off, and the understanding of potential blind spots of a high-risk AI system.} Therefore, this licentiate thesis focuses on AI-based DIR systems and presents an overview of the current state of the DIR research field. Additionally, experimental results are included that demonstrate the process of empirically motivating and evaluating the design of deep neural networks for DIR. To avoid the reliance on sequential Monte Carlo sampling techniques to produce an uncertainty estimation, we evaluated a surrogate model to reproduce uncertainty estimations learned from probabilistic deep-learning models. Lastly, to contextualize the results within the broader scope of safely integrating future high-risk AI-based systems into a car, we propose a foundational conceptual framework.Ett av tre delarbeten (övriga se rubriken Delarbeten/List of papers):Vellenga, Koen, H. Joe Steinhauer et al. (2024). "Designing deep neural networks for driver intention recognition". Under submission.</p

Evaluation of Video Masked Autoencoders' Performance and Uncertainty Estimations for Driver Action and Intention Recognition

Traffic fatalities remain among the leading death causes worldwide. To reduce this figure, car safety is listed as one of the most important factors. To actively support human drivers, it is essential for advanced driving assistance systems to be able to recognize the driver's actions and intentions. Prior studies have demonstrated various approaches to recognize driving actions and intentions based on in-cabin and external video footage. Given the performance of self-supervised video pre-trained (SSVP) Video Masked Autoencoders (VMAEs) on multiple action recognition datasets, we evaluate the performance of SSVP VMAEs on the Honda Research Institute Driving Dataset for driver action recognition (DAR) and on the Brain4Cars dataset for driver intention recognition (DIR). Besides the performance, the application of an artificial intelligence system in a safety-critical environment must be capable to express when it is uncertain about the produced results. Therefore, we also analyze uncertainty estimations produced by a Bayes-by-Backprop last-layer (BBB-LL) and Monte-Carlo (MC) dropout variants of an VMAE. Our experiments show that an VMAE achieves a higher overall performance for both offline DAR and end-to-end DIR compared to the state-of-the-art. The analysis of the BBB-LL and MC dropout models show higher uncertainty estimates for incorrectly classified test instances compared to correctly predicted test instances

Surrogate Deep Learning to Estimate Uncertainties for Driver Intention Recognition

Real-world applications of artificial intelligence that can potentially harm human beings should be able to express uncertainty about the made predictions. Probabilistic deep learning (DL) methods (e.g., variational inference [VI], VI last layer [VI-LL], Monte-Carlo [MC] dropout, stochastic weight averaging - Gaussian [SWA-G], and deep ensembles) can produce a predictive uncertainty but require expensive MC sampling techniques. Therefore, we evaluated if the probabilistic DL methods are uncertain when making incorrect predictions for an open-source driver intention recognition dataset and if a surrogate DL model can reproduce the uncertainty estimates. We found that all probabilistic DL methods are significantly more uncertain when making incorrect predictions at test time, but there are still instances where the models are very certain but completely incorrect. The surrogate DL models trained on the MC dropout and VI uncertainty estimates were capable of reproducing a significantly higher uncertainty estimate when making incorrect predictions.CC BY-NC-SA 4.0CORRESPONDING AUTHOR: K. VELLENGA (e-mail: [email protected])This work was supported by the Intention Recognition in Real Time for Automotive 3D Situation Awareness (IRRA) Project (https://www.vinnova.se/p/intention-recognition-i-realtid-for-automotive-3d-situation-awareness-irra/).Intention recognition for real-time automotive 3D situation awarenes

PT-HMC : Optimization-based Pre-Training with Hamiltonian Monte-Carlo Sampling for Driver Intention Recognition

Driver intention recognition (DIR) methods mostly rely on deep neural networks (DNNs). To use DNNs in asafety-critical real-world environment it is essential to quantify how confident the model is about the producedpredictions. Therefore, this study evaluates the performance and calibration of a temporal convolutionalnetwork (TCN) for multiple probabilistic deep learning (PDL) methods (Bayes-by-Backprop, Monte-Carlodropout, Deep ensembles, Stochastic Weight averaging - Gaussian, Multi SWA-G, cyclic Stochastic GradientHamiltonian Monte Carlo). Notably, we formalize an approach that combines optimization-based pre-trainingwith Hamiltonian Monte-Carlo (PT-HMC) sampling, aiming to leverage the strengths of both techniques. Ouranalysis, conducted on two pre-processed open-source DIR datasets, reveals that PT-HMC not only matchesbut occasionally surpasses the performance of existing PDL methods. One of the remaining challenges thatprohibits the integration of a PDL-based DIR system into an actual car is the computational requirements toperform inference. Therefore, future work could focus on optimizing PDL methods to be more computationallyefficient without sacrificing performance or the ability to estimate uncertainties

Designing deep neural networks for driver intention recognition

Driver intention recognition studies increasingly rely on deep neural networks. Deep neural networks have achieved top performance for many different tasks, but it is not a common practice to explicitly analyse the complexity and performance of the network's architecture. Therefore, this paper applies neural architecture search to investigate the effects of the deep neural network architecture on a real-world safety critical application with limited computational capabilities. We explore a pre-defined search space for three deep neural network layer types that are capable to handle sequential data (a long-short term memory, temporal convolution, and a time-series transformer layer), and the influence of different data fusion strategies on the driver intention recognition performance. A set of eight search strategies are evaluated for two driver intention recognition datasets. For the two datasets, we observed that there is no search strategy clearly sampling better deep neural network architectures. However, performing an architecture search does improve the model performance compared to the original manually designed networks. Furthermore, we observe no relation between increased model complexity and higher driver intention recognition performance. The result indicate that multiple architectures yield similar performance, regardless of the deep neural network layer type or fusion strategy.CC BY-NC-ND 4.0arXiv: Submitted on 7 Feb 2024Koen Vellenga"Under submission"</p

Driver intention recognition : state-of-the-art review

Every year worldwide more than one million people die and a further 50 million people are injured in traffic accidents. Therefore, the development of car safety features that actively support the driver in preventing accidents, is of utmost importance to reduce the number of injuries and fatalities. However, to establish this support it is necessary that the advanced driver assistance system (ADAS) understands the driver’s intended behavior in advance. The growing variety of sensors available for vehicles together with improved computer vision techniques, hence led to increased research directed towards inferring the driver’s intentions. This article reviews 64 driver intention recognition studies with regard to the maneuvers considered, the driving features included, the AI methods utilized, the achieved performance within the presented experiments, and the open challenges identified by the respected researchers. The article provides a high level analysis of the current technology readiness level of driver intention recognition technology to address the challenges to enable reliable driver intention recognition, such as the system architecture, implementation, and the purpose of the technology.CC BY-NC-ND 4.0CORRESPONDING AUTHOR: K. VELLENGA (e-mail: [email protected])This work was supported by the Intention Recognition in Real Time for Automotive 3D Situation Awareness (IRRA) Project (https://www.vinnova.se/p/intention-recognition-i-realtid-for-automotive-3d-situation-awareness-irra/).Intention recognition for real-time automotive 3D situation awarenes

Autophagy Proteins ATG5 and ATG7 Are Essential for the Maintenance of Human CD34+ Hematopoietic Stem-Progenitor Cells

Autophagy is a highly regulated catabolic process that involves sequestration and lysosomal degradation of cytosolic components such as damaged organelles and misfolded proteins. While autophagy can be considered to be a general cellular housekeeping process, it has become clear that it may also play cell type-dependent functional roles. In this study, we analyzed the functional importance of autophagy in human hematopoietic stem/progenitor cells (HSPCs), and how this is regulated during differentiation. Western blot-based analysis of LC3-II and p62 levels, as well as flow cytometry-based autophagic vesicle quantification, demonstrated that umbilical cord blood-derived CD34+/CD38- immature hematopoietic progenitors show a higher autophagic flux than CD34+/CD38+ progenitors and more differentiated myeloid and erythroid cells. This high autophagic flux was critical for maintaining stem and progenitor function since knockdown of autophagy genes ATG5 or ATG7 resulted in reduced HSPC frequencies in vitro as well as in vivo. The reduction in HSPCs was not due to impaired differentiation, but at least in part due to reduced cell cycle progression and increased apoptosis. This is accompanied by increased expression of p53, proapoptotic genes BAX and PUMA, and the cell cycle inhibitor p21, as well as increased levels of cleaved caspase-3 and reactive oxygen species. Taken together, our data demonstrate that autophagy is an important regulatory mechanism for human HSCs and their progeny, reducing cellular stress and promoting survival