Prefrontal cortex activation upon a demanding virtual hand-controlled task: A new frontier for neuroergonomics

open9noFunctional near-infrared spectroscopy (fNIRS) is a non-invasive vascular-based functional neuroimaging technology that can assess, simultaneously from multiple cortical areas, concentration changes in oxygenated-deoxygenated hemoglobin at the level of the cortical microcirculation blood vessels. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, could represent a valid tool for monitoring cortical responses in the research field of neuroergonomics. In virtual reality (VR) real situations can be replicated with greater control than those obtainable in the real world. Therefore, VR is the ideal setting where studies about neuroergonomics applications can be performed. The aim of the present study was to investigate, by a 20-channel fNIRS system, the dorsolateral/ventrolateral prefrontal cortex (DLPFC/VLPFC) in subjects while performing a demanding VR hand-controlled task (HCT). Considering the complexity of the HCT, its execution should require the attentional resources allocation and the integration of different executive functions. The HCT simulates the interaction with a real, remotely-driven, system operating in a critical environment. The hand movements were captured by a high spatial and temporal resolution 3-dimensional (3D) hand-sensing device, the LEAP motion controller, a gesture-based control interface that could be used in VR for tele-operated applications. Fifteen University students were asked to guide, with their right hand/forearm, a virtual ball (VB) over a virtual route (VROU) reproducing a 42 m narrow road including some critical points. The subjects tried to travel as long as possible without making VB fall. The distance traveled by the guided VB was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Nevertheless, a bilateral VLPFC activation, in response to the HCT execution, was observed in all the subjects. No correlation was found between the distance traveled by the guided VB and the corresponding cortical activation. These results confirm the suitability of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators during the simulation of different demanding/fatiguing activities.openCarrieri, Marika; Petracca, Andrea; Lancia, Stefania; Basso Moro, Sara; Brigadoi, Sabrina; Spezialetti, Matteo; Ferrari, Marco; Placidi, Giuseppe; Quaresima, ValentinaCarrieri, Marika; Petracca, Andrea; Lancia, Stefania; BASSO MORO, Sara; Brigadoi, Sabrina; Spezialetti, Matteo; Ferrari, Marco; Placidi, Giuseppe; Quaresima, Valentin

Multimodal Neuroergonomic Approaches to Human Behavior and Cognitive Workload in Complex High-Risk Semantically Rich Environments: A Case Study of Local & En-Route Air Traffic Controllers

Fast-paced technology advancements have enabled us to create ecologically valid simulations of high risk, complex, and semantically rich environments in which human interaction and decision-making are the keys to increase system performance. These advances have improved our capabilities of exploring, quantifying, and measuring the underlying mechanisms that guide human behavior using sophisticated neuroergonomic devices; and in turn, improve human performance and reduce human errors. In this thesis, multimodal approaches consisted of a self-report analysis, eye-tracking analysis, and functional near-infrared spectroscopy analysis were used to investigate how veteran local & en-route air traffic controllers carry out their operational tasks. Furthermore, the correlations among the cognitive workload and physiological measures (i.e. eye movement characteristics and brain activities) were investigated. Combining the results of these experiments, we can observe that the multimodal approaches show promise on exploring the underlying mechanisms of workload and human interaction in a complex, high-risk, and semantically rich environment. This is because cognitive workload can be considered as a multidimensional construct and different devices or approaches might be more effective in sensing changes in either the task difficulty or complexity. The results can be used to find ways to better train the novices

Using Functional Near Infrared Spectroscopy to Assess Cognitive Workload

Quantification of mental workload is a significant aspect of monitoring and adaptive aiding systems that are intended to improve the efficiency and safety of human–machine systems. Functional near Infrared (fNIR) spectroscopy is a field-deployable brain monitoring device that provides a measures of cerebral hemodynamic within the prefrontal cortex. The purpose of this study was to assess the cognitive load by using Performance (reaction time), Behavioral metrics (NASA TLX) and Neuro-Cognitive Measures (Hemodynamic response). To observe the activation in prefrontal cortex, we employed Functional Near Infrared (fNIR) Spectroscopy with a Standard Stroop task. A total of 25 healthy participants (N 18 Male and N 07 Female, M Age 25.5 SD 7.6), participated in the study. For statistical analysis, a repeated measure t-test was computed to compare the Oxy (Δ[HbO2]) and De-Oxy (Δ[hHb]) changes under Congruent and In-Congruent task conditions. For Classification, Binary logistic regression model applied to identify how accurately classifying the varied workload conditions. The finding shows that fNIR measures had adequate predictive power for estimating task performance in workload conditions. In this paper, we have found evidence that fNIR can be used as indicator of cognitive load which is important for optimal human performance

Real-Time State Estimation in a Flight Simulator Using fNIRS

Working memory is a key executive function for flying an aircraft. This function is particularly critical when pilots have to recall series of air traffic control instructions. However, working memory limitations may jeopardize flight safety. Since the functional near-infrared spectroscopy (fNIRS) method seems promising for assessing working memory load, our objective is to implement an on-line fNIRS-based inference system that integrates two complementary estimators. The first estimator is a real-time state estimation MACD-based algorithm dedicated to identifying the pilot’s instantaneous mental state (not-on-task vs. on-task). It does not require a calibration process to perform its estimation. The second estimator is an on-line SVM-based classifier that is able to discriminate task difficulty (low working memory load vs. high working memory load). These two estimators were tested with 19 pilots who were placed in a realistic flight simulator and were asked to recall air traffic control instructions. We found that the estimated pilot’s mental state matched significantly better than chance with the pilot’s real state (62% global accuracy, 58% specificity, and 72% sensitivity). The second estimator, dedicated to assessing single trial working memory loads, led to 80% classification accuracy, 72% specificity, and 89% sensitivity. These two estimators establish reusable blocks for further fNIRS-based passive brain computer interface development

Human Factors and Neurophysiological Metrics in Air Traffic Control: a Critical Review

International audienceThis article provides the reader a focused and organised review of the research progresses on neurophysiological indicators, also called “neurometrics”, to show how neurometrics could effectively address some of the most important Human Factors (HFs) needs in the Air Traffic Management (ATM) field. The state of the art on the most involved HFs and related cognitive processes (e.g. mental workload, cognitive training) is presented together with examples of possible applications in the current and future ATM scenarios, in order to better understand and highlight the available opportunities of such neuroscientific applications. Furthermore, the paper will discuss the potential enhancement that further research and development activities could bring to the efficiency and safety of the ATM service

Measuring cognitive load and cognition: metrics for technology-enhanced learning

This critical and reflective literature review examines international research published over the last decade to summarise the different kinds of measures that have been used to explore cognitive load and critiques the strengths and limitations of those focussed on the development of direct empirical approaches. Over the last 40 years, cognitive load theory has become established as one of the most successful and influential theoretical explanations of cognitive processing during learning. Despite this success, attempts to obtain direct objective measures of the theory's central theoretical construct – cognitive load – have proved elusive. This obstacle represents the most significant outstanding challenge for successfully embedding the theoretical and experimental work on cognitive load in empirical data from authentic learning situations. Progress to date on the theoretical and practical approaches to cognitive load are discussed along with the influences of individual differences on cognitive load in order to assess the prospects for the development and application of direct empirical measures of cognitive load especially in technology-rich contexts

Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty based on Brain State

We present Brain Automated Chorales (BACh), an adaptive brain-computer system that dynamically increases the levels of difficulty in a musical learning task based on pianists\u27 cognitive workload measured by functional near-infrared spectroscopy. As users\u27 cognitive workload fell below a certain threshold, suggesting that they had mastered the material and could handle more cognitive information, BACh automatically increased the difficulty of the learning task. We found that learners played with significantly increased accuracy and speed in the brain-based adaptive task compared to our control condition. Participant feedback indicated that they felt they learned better with BACh and they liked the timings of the level changes. The underlying premise of BACh can be applied to learning situations where a task can be broken down into increasing levels of difficulty

Applying Functional Near Infrared (fNIR) Spectroscopy to Enhance MIS Research

This review paper introduces the emerging technology of optical brain imaging, also known as functional near infrared (fNIR) spectroscopy, and discusses its potential role in enhancing theory and methodology used in MIS research. We discuss basic fNIR principles including the technique’s safe and portable nature, which allows ambulatory brain activity assessment in real world environments. We then touch on the neural correlates that fNIR measures, and the cortical oxygenation changes in the dorsal and anterior regions of the prefrontal cortex. We compare fNIR with traditional neuroimaging methods such as fMRI and PET. We also list case studies, future directions, and potential approaches relevant to MIS. fNIR may be used to inform theory and improve assessments in MIS-based studies, including informing theory, by identifying neural correlates, studying constructs that could not easily if at all be measured with traditional methods, applying objective constructs that subjects are unaware of, and designing better surveys

Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights

Monitoring pilots' cognitive states becomes increasingly important in aviation. Physiological measurement can detect increased mental workload (MWL) even before performance declines.Yet, changes in MWL are rarely varied systematically and few studies control for confounding effects of other cognitive states. The present study targets these shortcomings by analysing the effects of stepwise increased MWL on cortical activation, while controlling for mental fatigue (MF). 35 participants conducted a simulated flight with an incorporated adapted n‑back and monitoring task. We recorded cortical activation with concurrent EEG and fNIRS measurement, performance, self‑reported MWL and MF. Our results show the successful manipulation of MWL without confounding effects of MF. Higher task difficulty elicited higher subjective MWL ratings, performance decline, higher frontal theta activity and reduced frontal deoxyhaemoglobin (Hbr) concentration. Using both EEG and fNIRS, we could discriminate all induced MWL levels. fNIRS was more sensitive to tasks with low difficulty, and EEG to tasks with high difficulty. Our findings further suggest a plateau effect for high MWL that could present an upper boundary to individual cognitive capacity. Our results highlight the benefits of physiological measurement in aviation, both for assessment of cognitive states and as a data source for adaptive assistance systems