Machine Learning Methods for functional Near Infrared Spectroscopy

Identification of user state is of interest in a wide range of disciplines that fall under the umbrella of human machine interaction. Functional Near Infra-Red Spectroscopy (fNIRS) device is a relatively new device that enables inference of brain activity through non-invasively pulsing infra-red light into the brain. The fNIRS device is particularly useful as it has a better spatial resolution than the Electroencephalograph (EEG) device that is most commonly used in Human Computer Interaction studies under ecologically valid settings. But this key advantage of fNIRS device is underutilized in current literature in the fNIRS domain. We propose machine learning methods that capture this spatial nature of the human brain activity using a novel preprocessing method that uses `Region of Interest\u27 based feature extraction. Experiments show that this method outperforms the F1 score achieved previously in classifying `low\u27 vs `high\u27 valence state of a user. We further our analysis by applying a Convolutional Neural Network (CNN) to the fNIRS data, thus preserving the spatial structure of the data and treating the data similar to a series of images to be classified. Going further, we use a combination of CNN and Long Short-Term Memory (LSTM) to capture the spatial and temporal behavior of the fNIRS data, thus treating it similar to a video classification problem. We show that this method improves upon the accuracy previously obtained by valence classification methods using EEG or fNIRS devices. Finally, we apply the above model to a problem in classifying combined task-load and performance in an across-subject, across-task scenario of a Human Machine Teaming environment in order to achieve optimal productivity of the system

What we can and cannot (yet) do with functional near infrared spectroscopy

Functional near infrared spectroscopy (NIRS) is a relatively new technique complimentary to EEG for the development of brain-computer interfaces (BCIs). NIRS-based systems for detecting various cognitive and affective states such as mental and emotional stress have already been demonstrated in a range of adaptive human–computer interaction (HCI) applications. However, before NIRS-BCIs can be used reliably in realistic HCI settings, substantial challenges oncerning signal processing and modeling must be addressed. Although many of those challenges have been identified previously, the solutions to overcome them remain scant. In this paper, we first review what can be currently done with NIRS, specifically, NIRS-based approaches to measuring cognitive and affective user states as well as demonstrations of passive NIRS-BCIs. We then discuss some of the primary challenges these systems would face if deployed in more realistic settings, including detection latencies and motion artifacts. Lastly, we investigate the effects of some of these challenges on signal reliability via a quantitative comparison of three NIRS models. The hope is that this paper will actively engage researchers to acilitate the advancement of NIRS as a more robust and useful tool to the BCI community

A Neurophysiologic Study Of Visual Fatigue In Stereoscopic Related Displays

Two tasks were investigated in this study. The first study investigated the effects of alignment display errors on visual fatigue. The experiment revealed the following conclusive results: First, EEG data suggested the possibility of cognitively-induced time compensation changes due to a corresponding effect in real-time brain activity by the eyes trying to compensate for the alignment. The magnification difference error showed more significant effects on all EEG band waves, which were indications of likely visual fatigue as shown by the prevalence of simulator sickness questionnaire (SSQ) increases across all task levels. Vertical shift errors were observed to be prevalent in theta and beta bands of EEG which probably induced alertness (in theta band) as a result of possible stress. Rotation errors were significant in the gamma band, implying the likelihood of cognitive decline because of theta band influence. Second, the hemodynamic responses revealed that significant differences exist between the left and right dorsolateral prefrontal due to alignment errors. There was also a significant difference between the main effect for power band hemisphere and the ATC task sessions. The analyses revealed that there were significant differences between the dorsal frontal lobes in task processing and interaction effects between the processing lobes and tasks processing. The second study investigated the effects of cognitive response variables on visual fatigue. Third, the physiologic indicator of pupil dilation was 0.95mm that occurred at a mean time of 38.1min, after which the pupil dilation begins to decrease. After the average saccade rest time of 33.71min, saccade speeds leaned toward a decrease as a possible result of fatigue on-set. Fourth, the neural network classifier showed visual response data from eye movement were identified as the best predictor of visual fatigue with a classification accuracy of 90.42%. Experimental data confirmed that 11.43% of the participants actually experienced visual fatigue symptoms after the prolonged task

Attachment classification, psychophysiology and frontal EEG asymmetry across the lifespan: a review

In recent years research on physiological response and brain reactivity in different patterns of infant and adult attachment has increased. We review research findings regarding associations between attachment classifications and reactivity of the prefrontal cortex, the autonomic nervous system and the hypothalamic-pituitary-adrenocortical axis. Studies indicate that insecure attachment is related to a heightened adrenocortical activity, heart rate and skin conductance in response to stress, which is consistent with the hypothesis that attachment insecurity leads to impaired emotion regulation. Research on frontal EEG asymmetry also shows a clear difference in the emotional arousal between the attachment groups evidenced by specific frontal asymmetry changes. Furthermore, we discuss neurophysiological evidence of attachment organization and present up-to-date findings of EEG-research with adults. Based on the overall patterns of results presented in this article we identify some major areas of interest and directions for future research

The diagnosticity of psychophysiological signatures: Can we disentangle mental workload from acute stress with ECG and fNIRS?

The ability to identify reliable and sensitive physiological signatures of psychological dimensions is key to developing intelligent adaptive systems that may in turn help to mitigate human error in complex operations. The challenge of this endeavor lies with diagnosticity. Despite different underlying causes, the physiological correlates of workload and acute psychological stress manifest in rather similar ways and can be easily confounded. The current work aimed to build a diagnostic model of mental state through the simultaneous classification of mental workload (varied through three levels of the n-back task) and acute stress (the presence/absence of aversive sounds) with machine learning. Using functional near infrared spectroscopy (fNIRS) and electrocardiography (ECG), the model's classifiers was above-chance to disentangle variations of mental workload from variations of acute stress. Both ECG and fNIRS could predict mental workload level, the best accuracy resulted from the two measures in combination. Stress level could not be accurately diagnosed through ECG alone, only with fNIRS or ECG and fNIRS combined. Individual calibration may be important since stress classification was more accurate for those with higher subjective state anxiety, perhaps due to a greater sensitivity to stress. Mental workload and stress were both better classified with activity in lateral prefrontal regions of the cortex than the medial areas, and the HbO2 signal generally lead to better classification than HHB. The current model represents a step forward to finely discriminate different mental states despite their rather analog physiological correlates

The development of sensitivity to threat among children and adolescents

Several theories of adolescent brain development suggest that adolescence is a sensitive period of development characterized by the onset of internalizing problems, such as anxiety. Sensitivity to threat, a heightened responsiveness to aversive situations, has been suggested to be a precursor to anxiety, highlighting the importance of understanding sensitivity to threat among children and adolescents. Yet relatively little is known about the development of sensitivity to threat. Further, identifying the neural indicators that are associated with heightened sensitivity to threat would help classify which youth are most at risk for anxiety. The primary goals of my dissertation were: 1) to explore whether adolescents, compared to children, have heightened sensitive to threat, 2) assess which neural indicators are associated with heightened sensitivity to threat, and 3) assess whether individual differences (e.g., in consistency of sensitivity to threat across time and situation) help predict which youth are most at risk for anxiety-related problems. Study 1 of my dissertation examined, with concurrent data, whether adolescents have greater neural sensitivity to negative feedback compared to children. Study 2 examined whether children and adolescents differ in their longitudinal trajectories of sensitivity to threat (e.g., consistency across time). I also was interested in whether these trajectories were associated with frontal asymmetry, a neural indicator associated with avoidance motivations. Study 3 extended the findings from Study 2 to examine consistency across threatening situations. While Studies 1 through 3 investigated whether adolescence is a period of heightened sensitivity to threat, Study 4 of my dissertation used a latent class analysis to investigate whether individual differences in sensitivity to threat, impulsivity, and emotion dysregulation are associated with anxiety and/or risk taking. Results indicated that adolescence (especially when defined by pubertal status), may be a normative period for sensitivity to threat. At the same time, not all youth who are sensitive to threat go on to develop anxiety; thus, it may be that for many adolescents, sensitivity to threat is an adolescent-limited phenomenon, meaning that threat sensitivity may peak in adolescence, but then tapers off into adulthood. Importantly, neural indicators associated with threat sensitivity helped identify which youth may have the highest levels of threat sensitivity. Overall, my dissertation shows that while some level of sensitivity to threat is normative, it is less common for youth to be consistently sensitive to threats and importantly, these youth who are consistently sensitive appear to be most at risk. Taken together, the four studies of my dissertation incorporate EEG, longitudinal designs, multiple indicators of development (age and pubertal status), and self-report data to gain a holistic understanding of sensitivity to threat from childhood to adolescence

Manipulating Paradigm and Attention via a Mindfulness Meditation Training Program Improves P300-Based BCI.

To date, only one study has situationally bolstered attentional resources in an effort to improve P300-BCI performance. The current study implements a 4-week Mindfulness Meditation Training Program (MMTP) as a nonmedicinal means to increase concentrative attention and to reduce lapses of attention; MMTP is expected to improve P300-BCI performance by enhancing attentional resources and reducing distractibility. A second aim is to test the efficacy of the checkerboard paradigm (CBP) against the standard row-column paradigm (RCP). Online results show that MMTP had greater accuracies than CTRL and that CBP outperformed the RCP. MMTP participants provided greater amplitude positive target responses, but these differences were not statistically significant. CBP had greater positive amplitude peaks and negative peaks than RCP. The discussion focuses on potential benefits of MMTP for P300-based BCIs, provides further support for the construct validity of mindfulness, and addresses future directions of the translational applicability of MMTP to in-home settings

THE DEVELOPMENT OF A MULTIMODAL NEUROADAPTIVE GAMING TECHNOLOGY TO DISTRACT FROM PAINFUL EXPERIENCES.

Painful experiences can be mitigated by distraction techniques such as video game distraction, due to limited available attentional resources. There are many benefits to using video games as a non-pharmacological intervention, including their cost-effectiveness and absence of side effects or withdrawal symptoms. However, video games cannot provide a distraction which is sufficient for pain management if they are not engaging. This work aims to discuss how and why video games capture attention and explore how modulating game factors can affect the response to pain. The aim of this work in its entirety is to develop a neuroadaptive game which is tailored to reorient attention away from a painful experience, and towards the distraction technique. The neuroadaptive element of this technology will enable a balance of challenge and skill which make a unique and playable game for each participant. The development of the neuroadaptive game was supported by two studies. Study One focused on the determination of optimal game difficulty level for pain distraction, and Study Two furthered this research, alongside determining optimal neurological sites for the monitoring of attention and attentional reorientation. Study 3 explored the use of a neuroadaptive gaming technology to distract from pain – a bespoke, real-time data processing pipeline was developed for this purpose. The limitations of the neuroadaptive game are discussed in detail with considerations for future work and development. The results of the three studies carried out during the course of this work indicate that real-time pre-processing and classification of fNIRS data to a good standard is possible. The studies also revealed that the montage for data collection and features used for data collection are crucial considerations for classification accuracy. This thesis also has implications for further work into neuroadaptive technologies and how these systems can be tested and verified. Statistical significance between a non-neuroadaptive game and a neuroadaptive game was not found throughout the course of this work, although the potential explanations and future considerations are discussed in detail. Overall, we were able to confirm that pain tolerance can be improved with the use of a distraction task, but that the balance of task difficulty and skill level is delicate and requires further exploration

Evaluation of EEG-based depth of anaesthesia monitoring

In 2001 a University of Bristol team patented a novel data reduction method of the EEG for characterising categorical changes in consciousness. After pre-whitening the EEG signal with Gaussian white noise a parametric spectral estimation technique was applied. Two frequency domain indices were then proposed: the relative power found between 8Hz to 12Hz and 0.5Hz to 32Hz termed the 'alpha index', and the relative power between 0.5Hz to 4Hz and 0.5Hz to 32Hz termed the 'delta index'. The research and development of a precision EEG monitoring device designed to embody the novel algorithm is described in this thesis. The efficacy of the technique was evaluated using simulated and real EEG data recorded during Propofol anaesthesia. The simulated data showed improvements could be made to the patented method. Real EEG data collected whilst patients were wakeful and data from patients unresponsive to noxious stimuli were cleaned of obvious artefacts and analysed using the proposed algorithm. A Bayesian diagnostic test showed the alpha index had 65% sensitivity and selectivity to patient state. The delta index showed 72% sensitivity and selectivity. Taking a pragmatic approach, the literature is reviewed in this thesis to evaluate the use of EEG in depth of anaesthesia monitoring. Pertinent aspects of the sciences are profiled to identify physiological links to the characteristics of the EEG signal. Methods of data reduction are also reviewed to identify useful features and possible sources of error. In conclusion it is shown that the proposed indices do not provide a robust measure of depth of anaesthesia. An approach for further research is proposed based on the review work.EThOS - Electronic Theses Online ServiceGBUnited Kingdo