Demonstrating Brain-Level Interactions Between Visuospatial Attentional Demands and Working Memory Load While Driving Using Functional Near-Infrared Spectroscopy

Driving is a complex task concurrently drawing on multiple cognitive resources. Yet, there is a lack of studies investigating interactions at the brain-level among different driving subtasks in dual-tasking. This study investigates how visuospatial attentional demands related to increased driving difficulty interacts with different working memory load (WML) levels at the brain level. Using multichannel whole-head high density functional near-infrared spectroscopy (fNIRS) brain activation measurements, we aimed to predict driving difficulty level, both separate for each WML level and with a combined model. Participants drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. In half of the time, the course led through a construction site with reduced lane width, increasing visuospatial attentional demands. Concurrently, participants performed a modified version of the n-back task with five different WML levels (from 0-back up to 4-back), forcing them to continuously update, memorize, and recall the sequence of the previous ‘n’ speed signs and adjust their speed accordingly. Using multivariate logistic ridge regression, we were able to correctly predict driving difficulty in 75.0% of the signal samples (1.955 Hz sampling rate) across 15 participants in an out-of-sample cross-validation of classifiers trained on fNIRS data separately for each WML level. There was a significant effect of the WML level on the driving difficulty prediction accuracies [range 62.2–87.1%; χ2(4) = 19.9, p < 0.001, Kruskal–Wallis H test] with highest prediction rates at intermediate WML levels. On the contrary, training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46.8%). Activation changes in the bilateral dorsal frontal (putative BA46), bilateral inferior parietal (putative BA39), and left superior parietal (putative BA7) areas were most predictive to increased driving difficulty. These discriminative patterns diminished at higher WML levels indicating that visuospatial attentional demands and WML involve interacting underlying brain processes. The changing pattern of driving difficulty related brain areas across WML levels could indicate potential changes in the multitasking strategy with level of WML demand, in line with the multiple resource theory

Monitoring driver’s mental workload for user adaptive aid

Gebruikers-adaptieve machines gebruiken sensortechnieken en computer algoritmes om interne toestanden van de gebruiker af te leiden en beslissingen te nemen op basis van deze informatie. Toekomstige generaties auto’s zouden hiermee in staat kunnen worden gesteld actie te ondernemen indien de rijcapaciteit van de bestuurder suboptimaal wordt tijdens het autorijden, zelfs voordat de rijprestatie noemenswaardig verslechtert. De grote uitdaging van zo’n ondersteuning is dat het automatisch, onmiddellijk en op individueel niveau informatie moet kunnen interpreteren, terwijl dit individu beïnvloed wordt door hetzelfde systeem. Mijn proefschrift was gericht op deze uitdaging, waarbij de focus is gelegd op mentale inspanning.In hoofdstuk drie wordt betoogd dat een betrouwbaar systeem waarschijnlijk meerdere typen informatie nodig heeft om de interne toestand vast te kunnen stellen, zoals rijgedrag, fysiologie, en subjectieve ervaringen. Het belangrijkste inzicht uit hoofdstuk vier is dat gebruikers de ondersteuningsacties van adaptief systeem als waarschuwingssignaal kunnen gebruiken, en daarmee het systeem anders gebruikten dan bedoeld. In hoofdstuk vijf werd gekeken naar de mogelijkheid automatische muziekselectie in te zetten teneinde mentale inspanning te beïnvloeden, maar een direct verband tussen inspanning en muzieksoort werd niet aangetoond. In hoofdstuk zes werd de stap gemaakt naar individuele data-analyses uit hersengolven, met zeer goede classificatie resultaten. Uiteindelijk leidde dit tot een op hersengolven gebaseerde cruise control die tijdens (gesimuleerd) autorijden de mentale inspanning van de bestuurder classificeerde, terwijl de rijprestatie ook gemonitord werd (hoofdstuk zeven). Hieruit bleek dat vervolgonderzoek zich zou moeten richten op het verbeteren van de monitorbetrouwbaarheid door het verlagen van de tijds- en contextafhankelijkheid.User adaptive machines use sensor technology and computer algorithms to infer the user’s internal state and make decisions based on this information. Future cars could use this technology to intervene if the capacity of the driver to drive safely is degraded, even before performance starts to break down. The main challenge for such a support system is that it needs to interpret individual data automatically and immediately, while the individual is influenced by the same system. My thesis aims at this challenge, focussing on mental workload. In chapter three it is argued that a reliable system probably needs multiple types of measures to infer the user’s internal state, such as driving performance, physiology, and subjective experiences. The main result from chapter four is that users may use support actions of an adaptive system as a warning signal, and thereby not use the system as intended by the designers. In chapter five the potential was explored to use automatic music selection to influence mental workload was, but a direct link between mental effort and music type was not confirmed. Individual data analyses from brainwaves were the topic of chapter six, resulting in highly accurate workload classifications. This inspired the development of a performance and brain-based cruise control described in chapter seven. The adaptive performance of this system led to the conclusion that future research should focus on decreasing the context and time dependency of workload monitors for user adaptive systems