17 research outputs found
VIENA2: A Driving Anticipation Dataset
Action anticipation is critical in scenarios where one needs to react before
the action is finalized. This is, for instance, the case in automated driving,
where a car needs to, e.g., avoid hitting pedestrians and respect traffic
lights. While solutions have been proposed to tackle subsets of the driving
anticipation tasks, by making use of diverse, task-specific sensors, there is
no single dataset or framework that addresses them all in a consistent manner.
In this paper, we therefore introduce a new, large-scale dataset, called
VIENA2, covering 5 generic driving scenarios, with a total of 25 distinct
action classes. It contains more than 15K full HD, 5s long videos acquired in
various driving conditions, weathers, daytimes and environments, complemented
with a common and realistic set of sensor measurements. This amounts to more
than 2.25M frames, each annotated with an action label, corresponding to 600
samples per action class. We discuss our data acquisition strategy and the
statistics of our dataset, and benchmark state-of-the-art action anticipation
techniques, including a new multi-modal LSTM architecture with an effective
loss function for action anticipation in driving scenarios.Comment: Accepted in ACCV 201
Driver Distraction Identification with an Ensemble of Convolutional Neural Networks
The World Health Organization (WHO) reported 1.25 million deaths yearly due
to road traffic accidents worldwide and the number has been continuously
increasing over the last few years. Nearly fifth of these accidents are caused
by distracted drivers. Existing work of distracted driver detection is
concerned with a small set of distractions (mostly, cell phone usage).
Unreliable ad-hoc methods are often used.In this paper, we present the first
publicly available dataset for driver distraction identification with more
distraction postures than existing alternatives. In addition, we propose a
reliable deep learning-based solution that achieves a 90% accuracy. The system
consists of a genetically-weighted ensemble of convolutional neural networks,
we show that a weighted ensemble of classifiers using a genetic algorithm
yields in a better classification confidence. We also study the effect of
different visual elements in distraction detection by means of face and hand
localizations, and skin segmentation. Finally, we present a thinned version of
our ensemble that could achieve 84.64% classification accuracy and operate in a
real-time environment.Comment: arXiv admin note: substantial text overlap with arXiv:1706.0949
A Fuzzy-Logic Approach to Dynamic Bayesian Severity Level Classification of Driver Distraction Using Image Recognition
open access articleDetecting and classifying driver distractions is crucial in the prevention of road accidents. These distractions impact both driver behavior and vehicle dynamics. Knowing the degree of driver distraction can aid in accident prevention techniques, including transitioning of control to a level 4 semi- autonomous vehicle, when a high distraction severity level is reached. Thus, enhancement of Advanced Driving Assistance Systems (ADAS) is a critical component in the safety of vehicle drivers and other road users. In this paper, a new methodology is introduced, using an expert knowledge rule system to predict the severity of distraction in a contiguous set of video frames using the Naturalistic Driving American University of Cairo (AUC) Distraction Dataset. A multi-class distraction system comprises the face orientation, drivers’ activities, hands and previous driver distraction, a severity classification model is developed as a discrete dynamic Bayesian (DDB). Furthermore, a Mamdani-based fuzzy system was implemented to detect multi- class of distractions into a severity level of safe, careless or dangerous driving. Thus, if a high level of severity is reached the semi-autonomous vehicle will take control. The result further shows that some instances of driver’s distraction may quickly transition from a careless to dangerous driving in a multi-class distraction context
Beyond just keeping hands on the wheel: Towards visual interpretation of driver hand motion patterns
Abstract — Observing hand activity in the car provides a rich set of patterns relating to vehicle maneuvering, secondary tasks, driver distraction, and driver intent inference. This work strives to develop a vision-based framework for analyzing such patterns in real-time. First, hands are detected and tracked from a monocular camera. This provides position information of the left and right hands with no intrusion over long, naturalistic drives. Second, the motion trajectories are studied in settings of activity recognition, prediction, and higher-level semantic categorization. I
Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces
With the recently increasing capabilities of modern vehicles, novel
approaches for interaction emerged that go beyond traditional touch-based and
voice command approaches. Therefore, hand gestures, head pose, eye gaze, and
speech have been extensively investigated in automotive applications for object
selection and referencing. Despite these significant advances, existing
approaches mostly employ a one-model-fits-all approach unsuitable for varying
user behavior and individual differences. Moreover, current referencing
approaches either consider these modalities separately or focus on a stationary
situation, whereas the situation in a moving vehicle is highly dynamic and
subject to safety-critical constraints. In this paper, I propose a research
plan for a user-centered adaptive multimodal fusion approach for referencing
external objects from a moving vehicle. The proposed plan aims to provide an
open-source framework for user-centered adaptation and personalization using
user observations and heuristics, multimodal fusion, clustering,
transfer-of-learning for model adaptation, and continuous learning, moving
towards trusted human-centered artificial intelligence
Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual Drivers
The rapid advancement of the automotive industry towards automated and
semi-automated vehicles has rendered traditional methods of vehicle
interaction, such as touch-based and voice command systems, inadequate for a
widening range of non-driving related tasks, such as referencing objects
outside of the vehicle. Consequently, research has shifted toward gestural
input (e.g., hand, gaze, and head pose gestures) as a more suitable mode of
interaction during driving. However, due to the dynamic nature of driving and
individual variation, there are significant differences in drivers' gestural
input performance. While, in theory, this inherent variability could be
moderated by substantial data-driven machine learning models, prevalent
methodologies lean towards constrained, single-instance trained models for
object referencing. These models show a limited capacity to continuously adapt
to the divergent behaviors of individual drivers and the variety of driving
scenarios. To address this, we propose \textit{IcRegress}, a novel
regression-based incremental learning approach that adapts to changing behavior
and the unique characteristics of drivers engaged in the dual task of driving
and referencing objects. We suggest a more personalized and adaptable solution
for multimodal gestural interfaces, employing continuous lifelong learning to
enhance driver experience, safety, and convenience. Our approach was evaluated
using an outside-the-vehicle object referencing use case, highlighting the
superiority of the incremental learning models adapted over a single trained
model across various driver traits such as handedness, driving experience, and
numerous driving conditions. Finally, to facilitate reproducibility, ease
deployment, and promote further research, we offer our approach as an
open-source framework at \url{https://github.com/amrgomaaelhady/IcRegress}.Comment: Accepted for publication in the Proceedings of the 29th International
Conference on Intelligent User Interfaces (IUI'24), March 18--21, 2024, in
Greenville, SC, US