An integrated framework for multi-state driver monitoring using heterogeneous loss and attention-based feature decoupling

Multi-state driver monitoring is a key technique in building human-centric intelligent driving systems. This paper presents an integrated visual-based multi-state driver monitoring framework that incorporates head rotation, gaze, blinking, and yawning. To solve the challenge of head pose and gaze estimation, this paper proposes a unified network architecture that tackles these estimations as soft classification tasks. A feature decoupling module was developed to decouple the extracted features from different axis domains. Furthermore, a cascade cross-entropy was designed to restrict large deviations during the training phase, which was combined with the other features to form a heterogeneous loss function. In addition, gaze consistency was used to optimize its estimation, which also informed the model architecture design of the gaze estimation task. Finally, the proposed method was verified on several widely used benchmark datasets. Comprehensive experiments were conducted to evaluate the proposed method and the experimental results showed that the proposed method could achieve a state-of-the-art performance compared to other methods

Domain adaptation for driver's gaze mapping for different drivers and new environments

Distracted driving is a leading cause of traffic accidents, and often arises from a lack of visual attention on the road. To enhance road safety, monitoring a driver's visual attention is crucial. Appearance-based gaze estimation using deep learning and Convolutional Neural Networks (CNN) has shown promising results, but it faces challenges when applied to different drivers and environments. In this paper, we propose a domain adaptation-based solution for gaze mapping, which aims to accurately estimate a driver's gaze in diverse drivers and new environments. Our method consists of three steps: pre-processing, facial feature extraction, and gaze region classification. We explore two strategies for input feature extraction, one utilizing the full appearance of the driver and environment and the other focusing on the driver's face. Through unsupervised domain adaptation, we align the feature distributions of the source and target domains using a conditional Generative Adversarial Network (GAN). We conduct experiments on the Driver Gaze Mapping (DGM) dataset and the Columbia Cave-DB dataset to evaluate the performance of our method. The results demonstrate that our proposed method reduces the gaze mapping error, achieves better performance on different drivers and camera positions, and outperforms existing methods. We achieved an average Strictly Correct Estimation Rate (SCER) accuracy of 81.38% and 93.53% and Loosely Correct Estimation Rate (LCER) accuracy of 96.69% and 98.9% for the two strategies, respectively, indicating the effectiveness of our approach in adapting to different domains and camera positions. Our study contributes to the advancement of gaze mapping techniques and provides insights for improving driver safety in various driving scenarios

An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.PostprintPeer reviewe

Implementación de una herramienta de aprendizaje automático como apoyo para el diagnóstico temprano de enfermedades neurodegenerativas

La tecnología de seguimiento ocular es utilizada en todo tipo de ámbitos para establecer una relación entre la función cognitiva y el movimiento ocular: desde el marketing en grandes empresas, hasta los estudios conductuales en psicología y una enorme variedad de aplicaciones biomédicas. La diversidad en modos de funcionamiento de los aparatos, y el tratamiento de las series temporales que ofrece cada uno como resultado, resultan en un complejo escoyo en el uso de esta técnica. Por lo tanto, el objetivo de este trabajo es la creación y validación de un protocolo sencillo de apoyo en el diagnóstico de enfermedades neurodegenerativas a partir de las series temporales completas resultantes del seguimiento ocular para evitar la pérdida de información. Los datos se obtuvieron de la colaboración con una residencia y un hospital. Para generar las series temporales a partir de la lectura de pequeños textos se utilizó una aplicación previamente desarrollada de funcionamiento intuitivo y sencillo. Para esta fase, debido a sus especificaciones técnicas, se escogió el Eye-tracker Tobii X2-30 para la recogida de datos. En este proyecto se usaron dos algoritmos de aprendizaje automático para hacer una comparativa entre ellos y usar el de mayor precisión para el apoyo en el diagnóstico médico. Se utilizaron los algoritmos Multivariate Time Series Classification with Learned Discretization (SMTS) y una red neuronal Long Short Term Memory (LSTM). Tras el preprocesamiento de los datos, y la comparativa de los modelos de machine learning, se obtuvo una precisión de un 92,30% al diferenciar pacientes con enfermedad neurodegenerativa de aquellos sanos y una precisión de un 89,6% para asignar el tipo de enfermedad específica a cada paciente. Por consiguiente, aunque los resultados finales estén influenciados por el uso de una cohorte pequeña, este estudio nos da una prometedora perspectiva respecto al uso de esta tecnología como apoyo en el diagnóstico de enfermedades neurodegenerativas, mediante un protocolo rápido y sencillo