Multimodal Polynomial Fusion for Detecting Driver Distraction

Distracted driving is deadly, claiming 3,477 lives in the U.S. in 2015 alone. Although there has been a considerable amount of research on modeling the distracted behavior of drivers under various conditions, accurate automatic detection using multiple modalities and especially the contribution of using the speech modality to improve accuracy has received little attention. This paper introduces a new multimodal dataset for distracted driving behavior and discusses automatic distraction detection using features from three modalities: facial expression, speech and car signals. Detailed multimodal feature analysis shows that adding more modalities monotonically increases the predictive accuracy of the model. Finally, a simple and effective multimodal fusion technique using a polynomial fusion layer shows superior distraction detection results compared to the baseline SVM and neural network models.Comment: INTERSPEECH 201

Jerk as a Method of Identifying Physical Fatigue and Skill Level in Construction Work

Researchers have shown that physically demanding work, characterized by forceful exertions, repetition, and prolonged duration can result in fatigue. Physical fatigue has been identified as a risk factor for both acute and cumulative injuries. Thus, monitoring worker fatigue levels is highly important in health and safety programs as it supports proactive measures to prevent or reduce instances of injury to workers. Recent advancements in sensing technologies, including inertial measurement units (IMUs), present an opportunity for the real-time assessment of individuals' physical exposures. These sensors also exceed the ability of mature motion capture technologies to accurately provide fundamental parameters such as acceleration and its derivative, jerk. Although jerk has been used for a variety of clinical application to assess motor control, it has seldom been studied for applications in physically-demanding occupations that are directly related to physical fatigue detection. This research uses IMU-based motion tracking suits to evaluate the use of jerk to detect changes in motor control. Since fatigue degrades motor control, and thus motion smoothness, it is expected that jerk values will increase with fatigue. Jerk can be felt as the change in force on the body leading to biomechanical injuries over time. Although it is known that fatigue contributes to a decline in motor control, there are no explicit studies that show the relationship between jerk and fatigue. In addition, jerk as it relates to skill level of highly repetitive and demanding work has also remained unexplored. To examine these relationships, our first study evaluates: 1) the use of jerk to detect changes in motor control arising from physical exertion and 2) differences in jerk values between motions performed by workers with varying skill levels. Additionally, we conducted a second study to assess the suitability of machine learning techniques for automated physical fatigue monitoring. Bricklaying experiments were conducted with participants recruited from the Ontario Brick and Stone Mason apprenticeship program. Participants were classified into four groups based on their level of masonry experience including novices, first-year apprentices, third-year apprentices, and journeymen who have greater than five years of experience. In our first study, jerk analysis was carried out on eleven body segments, namely the pelvis, and the dominant and non-dominant upper and lower limb segments. Our findings show that jerk values were consistently lowest for journeymen and highest for third-year apprentices across all eleven body segments. These findings suggest that the experience that journeymen gain over the course of their career improves their ability to perform repetitive heavy lifts with smoother motions and greater control. Third-year apprentices performed lifts with the greatest jerk values, indicating poor motor performance. Attributed to this finding was the pressure that third-year apprentices felt to match their production levels to that of journeymen’s, leading third-year apprentices to use jerkier, less controlled motions. Novices and first-year apprentices showed more caution towards risks of injury, moving with greater motor control, compared to the more experienced third-year apprentices. However, the production levels of novices and first-year apprentices falter far behind the production levels of other experience groups. Detectable increases between jerk values during the beginning (rested) and end (exerted) of the task were found only for the journeymen, which is attributed to their greater interpersonal similarities in learned technique and work pace. In our second study, we investigated the use of support-vector machines (SVM) to automate the monitoring of physical exertion levels using jerk. The jerk values of the pelvis, upper arms, and thighs were used to classify inter-and intra-subject rested and exerted states. As expected, classification results demonstrated a significantly higher intra-subject rested/exerted classification than the inter-subject classification. On average, intra-subject classification achieved an accuracy of 94% for the wall building experiment and 80% for the first-course-of-masonry-units experiment. The thesis findings lead us to conclude that: 1) jerk changes resulting from physical exertion and skill level can be assessed using IMUs, and 2) SVMs have the ability to automatically classify rested and exerted movements. The investigated jerk analysis holds promise for in-situ and real-time monitoring of physical exertion and fatigue which can help in reducing work-related injuries and illnesses

An enhanced computational integrated decision model for prime decision-making in driving

Recent development of technology has led to the invention of driver assistance systems that support driving and prevent accidents. These systems employ Recognition-Primed Decision (RPD) model that use driver prior experience to make prime decision during emergencies. However, the existing RPD model does not include necessary training factors. Although, there is existing integrated RPD-SA model known as Integrated Decision-making Model (IDM) that includes training factors from Situation Awareness (SA) model, the training factors were not detailed (IDM has only six training factors). Hence, the model could not provide reasoning capability. Therefore, this study enhanced the IDM by proposing Computational-Rabi’s Driver Training (C-RDT) model that improves the RPD component with 18 additional training factors obtained from cognitive theories. The designed model is realized by identifying factors for prime decision-making in driving domain, designing the conceptual model of the RDT and formalizing it using differential equation. The model is verified through simulation, mathematical and automated analyses and then validated by human experiment. Verification result shows positive equilibrium conditions of the model (stability) and confirms the structural and theoretical correctness of the model. Furthermore, the validation result shows that the inclusion of the 18 training factors in the RPD training component of the IDM can improve driver’s prime decision-making. This study demonstrated the ability of the enhanced C-RDT model to backtrack and provide reasoning on the undertaking decisions. Hence, the model can also serve as a guideline for software developers in developing driving assistance systems