Cross-correlation based performance measures for characterizing the influence of in-vehicle interfaces on driving and cognitive workload

Driving is a cognitively loading task which requires drivers\u27 full attention and coordination of both mind and body. However, drivers often engage in side activities which can negatively impact safety. A typical approach for analyzing the influences of side activities on driving is to conduct experiments in which various driving performance measures are collected, such as steering wheel angle and lane position. Those measures are then transformed, typically using means and variances, before being analyzed statistically. However, the problem is that those transformations perform averaging of the acquired data, which can result in missing short, but important events (such as glances directed off-road). As a consequence, statistically significant differences may not be observed between the tested conditions. Nevertheless, just because the influences of in-vehicle interactions do not show in the averages, it does not mean that they do not exist or should be neglected, especially if the nature of the interactions is such that they can be performed frequently (for example, with an infotainment system). This can create a false conclusion about the lack of influence of the tested side activity on driving. The main contribution of this research is in developing two new performance measures inspired by the mathematical function of cross-correlation: one which evaluates the cumulative effect and the other which evaluates the effects of individual instances of in-vehicle interactions on driving and cognitive load. The results from three driving simulator studies demonstrate that our cumulative measure provides more sensitivity to the effects of in-vehicle interactions, even when they are not detected through average-based measures. Additionally, our instance-based measure provides a low-level insight into the nature of the influence of individual in-vehicle interactions. Both measures produce results that can be ranked, which allows determining the relative size of the effect that various in-vehicle interactions have on driving. Finally, we demonstrate a set of variables which can be used for predicting the cumulative and instance-based results. This predictive ability is important, because it may allow obtaining quick simulation results without performing actual experiments, which can be used in the early stages of an interface or experiment design process

Task constraints distinguish perspective inferences from perspective use during discourse interpretation in a false belief task

Interpreting other peoples’ actions relies on an understanding of their current mental states (e.g. beliefs, desires and intentions). In this paper, we distinguish between listeners’ ability to infer others’ perspectives and their explicit use of this knowledge to predict subsequent actions. In a visual-world study, two groups of participants (passive observers vs. active participants) watched short videos, depicting transfer events, where one character (‘Jane’) either held a true or false belief about an object’s location. We tracked participants’ eye-movements around the final visual scene, time-locked to related auditory descriptions (e.g. “Jane will look for the chocolates in the container on the left”.). Results showed that active participants had already inferred the character’s belief in the 1 s preview period prior to auditory onset, before it was possible to use this information to predict an outcome. Moreover, they used this inference to correctly anticipate reference to the object’s initial location on false belief trials at the earliest possible point (i.e. from “Jane” onwards). In contrast, passive observers only showed evidence of a belief inference from the onset of “Jane”, and did not show reliable use of this inference to predict Jane’s behaviour on false belief trials until much later, when the location (“left/right”) was auditorily available. These results show that active engagement in a task activates earlier inferences about others’ perspectives, and drives immediate use of this information to anticipate others’ actions, compared to passive observers, who are susceptible to influences from egocentric or reality biases. Finally, we review evidence that using other peoples’ perspectives to predict their behaviour is more cognitively effortful than simply using one’s own

Listen Carefully protocol : an exploratory case–control study of the association between listening effort and cognitive function

Introduction A growing body of evidence suggests that hearing loss is a significant and potentially modifiable risk factor for cognitive impairment. Although the mechanisms underlying the associations between cognitive decline and hearing loss are unclear, listening effort has been posited as one of the mechanisms involved with cognitive decline in older age. To date, there has been a lack of research investigating this association, particularly among adults with mild cognitive impairment (MCI).Methods and analysis 15–25 cognitively healthy participants and 15–25 patients with MCI (age 40–85 years) will be recruited to participate in an exploratory study investigating the association between cognitive functioning and listening effort. Both behavioural and objective measures of listening effort will be investigated. The sentence-final word identification and recall (SWIR) test will be administered with single talker non-intelligible speech background noise while monitoring pupil dilation. Evaluation of cognitive function will be carried out in a clinical setting using a battery of neuropsychological tests. This study is considered exploratory and proof of concept, with information taken to help decide the validity of larger-scale trials.Ethics and dissemination Written approval exemption was obtained by the Scientific Ethics Committee in the central region of Denmark (De Videnskabsetiske Komiteer i Region Hovedstaden), reference 19042404, and the project is registered pre-results at clinicaltrials.gov, reference NCT04593290, Protocol ID 19042404. Study results will be disseminated in peer-reviewed journals and conferences

Preliminary study for the measurement of Biosignals in Driving Simulators

openThis preliminary study focuses on the goal of developing and testing a setup and method for non-invasive monitoring of individuals using biosensors in a professional driving simulator (VI-grade Compact Simulator). This involves the synchronization and integration of hardware and software components. To detect the emotional and cognitive state of the driver, it is crucial to identify which signals provide reliable information about their condition. The objective of this study is to observe individuals in a controlled and repeatable environment designed to stimulate cognitive workload. This was achieved using a multimodal assessment method (iMotions), which includes eye tracking, galvanic skin response (GSR), electromyography (EMG), and respiration measurements, all conducted during two distinct controlled driving simulation scenarios. Four healthy subjects (average age = 24, standard deviation = ±2) were monitored during the first scenario, a highway with repeated emergency maneuvers (slalom through cones and double lane change), and the second, five laps of the Paul Ricard circuit. All of this for a total duration of approximately 20 minutes. The participants were not aware that the scenarios were designed to provoke different reactions. This experimental thesis aims to be the continuation and evolution of a testing phase previously conducted during an internship at iMotions, a company that develops multimodal streaming software and distributes commercial hardware. The hardware was supplied to the NAVLAB at the University of Padua, where the simulator is located. The results obtained, at first analysis, appear to be consistent with the literature, suggesting that a multimodal approach to physiological signals may characterize emotional and cognitive states in driving scenarios.This preliminary study focuses on the goal of developing and testing a setup and method for non-invasive monitoring of individuals using biosensors in a professional driving simulator (VI-grade Compact Simulator). This involves the synchronization and integration of hardware and software components. To detect the emotional and cognitive state of the driver, it is crucial to identify which signals provide reliable information about their condition. The objective of this study is to observe individuals in a controlled and repeatable environment designed to stimulate cognitive workload. This was achieved using a multimodal assessment method (iMotions), which includes eye tracking, galvanic skin response (GSR), electromyography (EMG), and respiration measurements, all conducted during two distinct controlled driving simulation scenarios. Four healthy subjects (average age = 24, standard deviation = ±2) were monitored during the first scenario, a highway with repeated emergency maneuvers (slalom through cones and double lane change), and the second, five laps of the Paul Ricard circuit. All of this for a total duration of approximately 20 minutes. The participants were not aware that the scenarios were designed to provoke different reactions. This experimental thesis aims to be the continuation and evolution of a testing phase previously conducted during an internship at iMotions, a company that develops multimodal streaming software and distributes commercial hardware. The hardware was supplied to the NAVLAB at the University of Padua, where the simulator is located. The results obtained, at first analysis, appear to be consistent with the literature, suggesting that a multimodal approach to physiological signals may characterize emotional and cognitive states in driving scenarios

Listening in a second language: a pupillometric investigation of the effect of semantic and acoustic cues on listening effort

Non-native listeners live a great part of their day immersed in a second language environment. Challenges arise because many linguistic interactions happen in noisy environments, and because their linguistic knowledge is imperfect. Pupillometry was shown to provide a reliable measure of cognitive effort during listening. This research aims to investigate by means of pupillometry how listening effort is modulated by the intelligibility level of the listening task, the availability of contextual and acoustic cues and by the language background of listeners (native vs non-native). In Study 1, listening effort in native and non-native listeners was evaluated during a sentence perception task in noise across different intelligibility levels. Results indicated that listening effort was increased for non-native compared to native listeners, when the intelligibility levels were equated across the two groups. In Study 2, using a similar method, materials included predictable and semantically anomalous sentences, presented in a plain and a clear speaking style. Results confirmed an increased listening effort for non-native compared to native listeners. Listening effort was overall reduced when participants attended to clear speech. Moreover, effort reduction after the sentence ended was delayed for less proficient non-native listeners. In Study 3, the contribution of semantic content spanning over several sentences was evaluated using lists of semantically related and unrelated stimuli. The presence of semantic cues across sentences led to a reduction in listening effort for native listeners as reflected by the peak pupil dilation, while non-native listeners did not show the same benefit. In summary, this research consistently showed an increased listening effort for non-native compared to native listeners, at equated levels of intelligibility. Additionally, the use of a clear speaking style proved to be an effective strategy to enhance comprehension and to reduce cognitive effort in native and non-native listeners

Feel, Don\u27t Think Review of the Application of Neuroscience Methods for Conversational Agent Research

Conversational agents (CAs) equipped with human-like features (e.g., name, avatar) have been reported to induce the perception of humanness and social presence in users, which can also increase other aspects of users’ affection, cognition, and behavior. However, current research is primarily based on self-reported measurements, leaving the door open for errors related to the self-serving bias, socially desired responding, negativity bias and others. In this context, applying neuroscience methods (e.g., EEG or MRI) could provide a means to supplement current research. However, it is unclear to what extent such methods have already been applied and what future directions for their application might be. Against this background, we conducted a comprehensive and transdisciplinary review. Based on our sample of 37 articles, we find an increased interest in the topic after 2017, with neural signal and trust/decision-making as upcoming areas of research and five separate research clusters, describing current research trends

Investigating supra-intelligibility aspects of speech

158 p.Synthetic and recorded speech form a great part of oureveryday listening experience, and much of our exposure tothese forms of speech occurs in potentially noisy settings such as on public transport, in the classroom or workplace, while driving, and in our homes. Optimising speech output to ensure that salient information is both correctly and effortlessly received is a main concern for the designers of applications that make use of the speech modality. Most of the focus in adapting speech output to challenging listening conditions has been on intelligibility, and specifically on enhancing intelligibility by modifying speech prior to presentation. However, the quality of the generated speech is not always satisfying for the recipient, which might lead to fatigue, or reluctance in using this communication modality. Consequently, a sole focus on intelligibility enhancement provides an incomplete picture of a listener¿s experience since the effect of modified or synthetic speech on other characteristics risks being ignored. These concerns motivate the study of 'supra-intelligibility' factors such as the additional cognitive demand that modified speech may well impose upon listeners, as well as quality, naturalness, distortion and pleasantness. This thesis reports on an investigation into two supra-intelligibility factors: listening effort and listener preferences. Differences in listening effort across four speech types (plain natural, Lombard, algorithmically-enhanced, and synthetic speech) were measured using existing methods, including pupillometry, subjective judgements, and intelligibility scores. To explore the effects of speech features on listener preferences, a new tool, SpeechAdjuster, was developed. SpeechAdjuster allows the manipulation of virtually any aspect of speech and supports the joint elicitation of listener preferences and intelligibility measures. The tool reverses the roles of listener and experimenter by allowing listeners direct control of speech characteristics in real-time. Several experiments to explore the effects of speech properties on listening preferences and intelligibility using SpeechAdjuster were conducted. Participants were permitted to change a speech feature during an open-ended adjustment phase, followed by a test phase in which they identified speech presented with the feature value selected at the end of the adjustment phase. Experiments with native normal-hearing listeners measured the consequences of allowing listeners to change speech rate, fundamental frequency, and other features which led to spectral energy redistribution. Speech stimuli were presented in both quiet and masked conditions. Results revealed that listeners prefer feature modifications similar to those observed in naturally modified speech in noise (Lombard speech). Further, Lombard speech required the least listening effort compared to either plain natural, algorithmically-enhanced, or synthetic speech. For stationary noise, as noise level increased listeners chose slower speech rates and flatter tilts compared to the original speech. Only the choice of fundamental frequency was not consistent with that observed in Lombard speech. It is possible that features such as fundamental frequency that talkers naturally modify are by-products of the speech type (e.g. hyperarticulated speech) and might not be advantageous for the listener.Findings suggest that listener preferences provide information about the processing of speech over and above that measured by intelligibility. One of the listeners¿ concerns was to maximise intelligibility. In noise, listeners preferred the feature values for which more information survived masking, choosing speech rates that led to a contrast with the modulation rate of the masker, or modifications that led to a shift of spectral energy concentration to higher frequencies compared to those of the masker. For all features being modified by listeners, preferences were evident even when intelligibility was at or close to ceiling levels. Such preferences might result from a desire to reduce the cognitive effort of understanding speech, or from a desire to reproduce the sound of typical speech features experienced in real-world noisy conditions, or to optimise the quality of the modified signal. Investigation of supra-intelligibility aspects of speech promises to improve the quality of speech enhancement algorithms, bringing with it the potential of reducing the effort of understanding artificially-modified or generated forms of speech

Trends in Electrodermal Activity, Heart Rate and Temperature during Distracted Driving among Young Novice Drivers.

Driver distraction, defined as the scattering of attention from critical activities for safe driving, is among the key globally recognized contributing factors to road crashes. The trend keeps increasing with in-vehicle information systems and hand-held devices, leading to inattention. Of people in all age groups, young novice teenagers are prone to the risk of road crashes and are also more likely to exhibit risky and unsafe driving behavior. Data shows that the involvement of distracted drivers in fatal & injury collisions is higher for people aged between 16 -34, which is about 55%. Therefore, young drivers are of great concern for the research about driving and evaluation of safe driving conditions, which is vital in upcoming advancements in autonomous vehicles. Several research studies have explored the effects of distracted driving using face tracking and eye glance monitoring. Previous research [50] did not consider much about the effect of distraction on physiological factors and their impact during driving. The current study used data collected from a previous thesis work titled “Detection of Driver Cognitive Distraction Using Machine Learning Methods” by Apurva Misra and conducted new data analysis focusing on new research questions. The main objective of this thesis is to study, identify and discuss the effects on physiological factors like heart rate (HR), electrodermal activity (EDA), body temperature, and motion sickness during distracted driving among young drivers. The data was collected from a driving simulator study comprising 42 participants aged 16 – 23 under normal and distracted driving conditions. Their driving experience ranges from 0 to a maximum of 5 years. Each participant navigated six scenarios, three with distraction and the rest without distraction. Each scenario has a hidden, latent hazard depending on the surrounding; for example, in the work zone scenario, a worker is hidden behind the bulldozer in the work zone. The distraction task is a spoken task for which the driver has to respond verbally, which exerts a workload similar to that observed in conversations using a hands-free mobile phone. The physiological data collected through the Empatica4 wristband was analyzed and compared against age, gender, driver experience, and another parameter like motion sickness score (MSS) obtained from a questionnaire the participants completed after the experiment. Of the physiological factors stated above, it was found that HR and EDA play a significant role while studying distraction. Data analysis showed that HR and EDA increase more during distraction than baseline events. Nearly 80% of drivers with 0 or 1 year of experience tend to have a higher range of HR and EDA, which reveals that they are more distracted than their peers with more experience. From the results of the Load index questionnaire and Motion Sickness susceptibility questionnaire, it is inferred that when MSS increases, there is an increase in HR and EDA. These findings will provide insights into physiological factors for developing distraction mitigation systems or in-vehicle warning systems for distracted drivers

The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study

Carminati MN, Knoeferle P. The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study. Presented at the Architectures and Mechanisms of Language and Processing (AMLaP), Riva del Garda, Italy