Gaming behavior and brain activation using functional near‐infrared spectroscopy, Iowa gambling task, and machine learning techniques

Abstract Introduction The current study investigates the utilization and performance of machine learning (ML) algorithms in the cognitive task of finding the correlation between numerical parameters of the human brain activation during gaming. We hypothesize that our integrated feature extraction platform is able to distinguish between different psychosomatic conditions in the gaming process as measured by the functional near‐infrared brain imaging technique. Methods For demonstration, the decision‐making process was constructed in the experiment environment that combined gaming simulator, such as the Iowa Gaming Task (IGT), with functional near‐infrared spectroscopy (fNIRS) as the neuroimaging technique. Features of fNIRS levels were extracted, averaged, and synchronized by time with the IGT dataset to predict the task score inside ML algorithms, such as multiple regression, classification and regression trees, support vector machine, artificial neural network, and random forest. For findings validation, the experiment data were resampled by training and testing sets. Further, a training dataset was used to train the ML algorithms, and prediction accuracy was estimated by repeated cross‐validation methods and compared by R squared and root mean square error (RMSE). The model with the best accuracy was used with the testing dataset and finalized the experiment. Results During the experiment, the highest correlation was identified in the fourth block between the oxy‐hemoglobin signal and IGT score in average value (0.24) and signal feature (0.57). Such relationship is due to block 4 characterization as “conceptual” period when participants task experience reaches the maximum, and rewards raise accordingly. Simultaneously, ML algorithms, constructed based on training data set, demonstrate acceptable performance, and RMSE as the primary performance metric dynamically increases from block 1 to block 5, from the state of uncertainty and unknown to the certainty and risky. In contrast, R squared decreases during the same transition. In most IGT blocks, the best fitted model was determined as support vector machine with radial bases function kernel, and predictions were made with the highest accuracy (lowest RMSE) than in training models. Conclusion Obtained findings showed the applicability and capability of ML models as a powerful technique to evaluate the cognitive neuroimaging task result. Moreover, in terms of features it was identified that the hemodynamic response reacts to the acceleration decision‐making process and raises more significance than it was observed before

Exploring the role of task performance and learning style on prefrontal hemodynamics during a working memory task.

Existing literature outlines the quality and location of activation in the prefrontal cortex (PFC) during working memory (WM) tasks. However, the effects of individual differences on the underlying neural process of WM tasks are still unclear. In this functional near infrared spectroscopy study, we administered a visual and auditory n-back task to examine activation in the PFC while considering the influences of task performance, and preferred learning strategy (VARK score). While controlling for age, results indicated that high performance (HP) subjects (accuracy > 90%) showed task dependent lower activation compared to normal performance subjects in PFC region Specifically HP groups showed lower activation in left dorsolateral PFC (DLPFC) region during performance of auditory task whereas during visual task they showed lower activation in the right DLPFC. After accounting for learning style, we found a correlation between visual and aural VARK score and level of activation in the PFC. Subjects with higher visual VARK scores displayed lower activation during auditory task in left DLPFC, while those with higher visual scores exhibited higher activation during visual task in bilateral DLPFC. During performance of auditory task, HP subjects had higher visual VARK scores compared to NP subjects indicating an effect of learning style on the task performance and activation. The results of this study show that learning style and task performance can influence PFC activation, with applications toward neurological implications of learning style and populations with deficits in auditory or visual processing

Structured sparse multiset canonical correlation analysis of simultaneous fNIRS and EEG provides new insights into the human action-observation network

The action observation network (AON) is a network of brain regions involved in the execution and observation of a given action. The AON has been investigated in humans using mostly electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), but shared neural correlates of action observation and action execution are still unclear due to lack of ecologically valid neuroimaging measures. In this study, we used concurrent EEG and functional Near Infrared Spectroscopy (fNIRS) to examine the AON during a live-action observation and execution paradigm. We developed structured sparse multiset canonical correlation analysis (ssmCCA) to perform EEG-fNIRS data fusion. MCCA is a generalization of CCA to more than two sets of variables and is commonly used in medical multimodal data fusion. However, mCCA suffers from multi-collinearity, high dimensionality, unimodal feature selection, and loss of spatial information in interpreting the results. A limited number of participants (small sample size) is another problem in mCCA, which leads to overfitted models. Here, we adopted graph-guided (structured) fused least absolute shrinkage and selection operator (LASSO) penalty to mCCA to conduct feature selection, incorporating structural information amongst the variables (i.e., brain regions). Benefitting from concurrent recordings of brain hemodynamic and electrophysiological responses, the proposed ssmCCA finds linear transforms of each modality such that the correlation between their projections is maximized. Our analysis of 21 right-handed participants indicated that the left inferior parietal region was active during both action execution and action observation. Our findings provide new insights into the neural correlates of AON which are more fine-tuned than the results from each individual EEG or fNIRS analysis and validate the use of ssmCCA to fuse EEG and fNIRS datasets

Simultaneous multimodal fNIRS-EEG recordings reveal new insights in neural activity during motor execution, observation, and imagery

Abstract Motor execution, observation, and imagery are important skills used in motor learning and rehabilitation. The neural mechanisms underlying these cognitive-motor processes are still poorly understood. We used a simultaneous recording of functional near-infrared spectroscopy (fNIRS) and electroencephalogram (EEG) to elucidate the differences in neural activity across three conditions requiring these processes. Additionally, we used a new method called structured sparse multiset Canonical Correlation Analysis (ssmCCA) to fuse the fNIRS and EEG data and determine the brain regions of neural activity consistently detected by both modalities. Unimodal analyses revealed differentiated activation between conditions; however, the activated regions did not fully overlap across the two modalities (fNIRS: left angular gyrus, right supramarginal gyrus, as well as right superior and inferior parietal lobes; EEG: bilateral central, right frontal, and parietal). These discrepancies might be because fNIRS and EEG detect different signals. Using fused fNIRS-EEG data, we consistently found activation over the left inferior parietal lobe, superior marginal gyrus, and post-central gyrus during all three conditions, suggesting that our multimodal approach identifies a shared neural region associated with the Action Observation Network (AON). This study highlights the strengths of using the multimodal fNIRS-EEG fusion technique for studying AON. Neural researchers should consider using the multimodal approach to validate their findings

Differences between high performance and normal performance groups in visual and aural VARK scores.

<p>(a) during performance of visual task and (b) during performance of auditory task.</p

Estimated marginal means of WM activation level across all tasks in different PFC regions.

<p>Estimated marginal means of WM activation level across all tasks in different PFC regions.</p

Sensor position on the prefrontal cortex region with respect to Fpz (International 10–20 system).

<p>Yellow dots: Sources, Blue dots: Detectors. Numbers indicate the loaction of the channels based on the source-detector pairs. The dotted squares indicate channel parinings.</p

Differences in activation between high performance (HP) and normal performance (NP) groups.

<p>Both auditory and visual NP groups showed higher activation in PFC compared to their high-performance cohorts in right DLPFC during visual (a) and left DLPFC and MPFC during auditory (b) tasks.</p