Hemodynamic Analysis for Cognitive Load Assessment and Classification in Motor Learning Tasks Using Type-2 Fuzzy Sets

The paper addresses a novel approach to assess and classify the cognitive load of subjects from their hemodynamic response while engaged in motor learning tasks, such as vehicle-driving. A set of complex motor-activity-learning stimuli for braking, steering-control and acceleration is prepared to experimentally measure and classify the cognitive load of the car-drivers in three distinct classes: High, Medium and Low. New models of General and Interval Type-2 Fuzzy classifiers are proposed to reduce the scope of uncertainty in cognitive load classification due to the fluctuation of the hemodynamic features within and across sessions. The proposed classifiers offer high classification accuracy over 96%, leaving behind the traditional type-1/type-2 fuzzy and other standard classifiers. Experiments undertaken also offer a deep biological insight concerning the shift of brain-activations from the orbito-frontal to the ventro-lateral prefrontal cortex during high-to-low transition in cognitive load. Further, the activation of the dorsolateral prefrontal cortex is also reduced during low cognitive load of subjects. The proposed research outcome may directly be utilized to identify driving learners with low cognitive load for difficult motor learning tasks, such as taking a U-turn in a narrow space and motion control on the top of a bridge to avoid possible collision with the car ahead

Hemodynamic Analysis for Olfactory Perceptual Degradation Assessment Using Generalized Type-2 Fuzzy Regression

Olfactory perceptual degradation refers to the inability of people to recognize the variation in concentration levels of olfactory stimuli. The paper attempts to assess the degree of olfactory perceptual degradation of subjects from their hemodynamic response to olfactory stimuli. This is done in 2 phases. In the first (training) phase, a regression model is developed to assess the degree of concentration levels of an olfactory stimulus by a subject from her hemodynamic response to the stimulus. In the second (test) phase, the model is employed to predict the possible concentration level experienced by the subject in [0, 100] scale. The difference between the model-predicted response and the oral response (the center value of the qualitative grades) of the subject about her perceived concentration level is regarded as the quantitative measure of the degree of subject's olfactory degradation. The novelty of the present research lies in the design of a General Type-2 fuzzy regression model, which is capable of handling uncertainty due to the presence of intra- and inter-session variations in the brain responses to olfactory stimuli. The attractive feature of the paper lies in adaptive tuning of secondary membership functions to reduce model prediction error in an evolutionary optimization setting. The effect of such adaptation in secondary measures is utilized to adjust the corresponding primary memberships in order to reduce the uncertainty involved in the regression process. The proposed regression model has good prediction accuracy and high time-efficiency as evident from average percentage success rate (PSR) and run-time complexity analysis respectively. The Friedman test undertaken also confirms the superior performance of the proposed technique with other competitive techniques at 95% confidence level

Olfactory Perceptual-Ability Assessment by Near-Infrared Spectroscopy using Vertical-Slice based Fuzzy Reasoning

The paper introduced a novel approach for automatic assessment of olfactory perceptual-ability of human-subjects using a functional Near Infrared Spectroscopy device. The assessment requires fuzzy functional mapping from spectroscopic measurement to perceptual-ability using Type-2 fuzzy reasoning. The novelty of the work lies in Vertical Slice Based General Type-2 Fuzzy Reasoning which employs fuzzy meet and union between the planes of type-2 measurement and observation spaces using the classical definition of t-norms and s-norms. The results of the meet and the union computation are later used as the Lower and Upper Firing Strength of the fired rule to determine the structure of the inference. Experiments undertaken confirm the efficacy of the proposed technique over traditional functional mapping, involving neural networks, regression analysis, and the like. The proposed technique of olfactory perceptual-ability can be directly employed to determine the thresholds for recognition-probability and discrimination-probability, when submitted to the subject in presence of aromatic noise. An analysis is undertaken to measure the computational overhead, which is found of the order of O(m.n) and run-time complexity of 94.78 ms, where m and n respectively represent discretizations in the vertical slice and features respectively. A statistical test undertaken confirms the superior performance of the proposed system with others at 95% confidence level

Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of certain types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. Different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or the brain activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Further, potential topics in relation to driving behaviour are identified that could benefit from the adoption of neuroimaging methods in future studies

Speech Processes for Brain-Computer Interfaces

Speech interfaces have become widely used and are integrated in many applications and devices. However, speech interfaces require the user to produce intelligible speech, which might be hindered by loud environments, concern to bother bystanders or the general in- ability to produce speech due to disabilities. Decoding a usera s imagined speech instead of actual speech would solve this problem. Such a Brain-Computer Interface (BCI) based on imagined speech would enable fast and natural communication without the need to actually speak out loud. These interfaces could provide a voice to otherwise mute people. This dissertation investigates BCIs based on speech processes using functional Near In- frared Spectroscopy (fNIRS) and Electrocorticography (ECoG), two brain activity imaging modalities on opposing ends of an invasiveness scale. Brain activity data have low signal- to-noise ratio and complex spatio-temporal and spectral coherence. To analyze these data, techniques from the areas of machine learning, neuroscience and Automatic Speech Recog- nition are combined in this dissertation to facilitate robust classification of detailed speech processes while simultaneously illustrating the underlying neural processes. fNIRS is an imaging modality based on cerebral blood flow. It only requires affordable hardware and can be set up within minutes in a day-to-day environment. Therefore, it is ideally suited for convenient user interfaces. However, the hemodynamic processes measured by fNIRS are slow in nature and the technology therefore offers poor temporal resolution. We investigate speech in fNIRS and demonstrate classification of speech processes for BCIs based on fNIRS. ECoG provides ideal signal properties by invasively measuring electrical potentials artifact- free directly on the brain surface. High spatial resolution and temporal resolution down to millisecond sampling provide localized information with accurate enough timing to capture the fast process underlying speech production. This dissertation presents the Brain-to- Text system, which harnesses automatic speech recognition technology to decode a textual representation of continuous speech from ECoG. This could allow to compose messages or to issue commands through a BCI. While the decoding of a textual representation is unparalleled for device control and typing, direct communication is even more natural if the full expressive power of speech - including emphasis and prosody - could be provided. For this purpose, a second system is presented, which directly synthesizes neural signals into audible speech, which could enable conversation with friends and family through a BCI. Up to now, both systems, the Brain-to-Text and synthesis system are operating on audibly produced speech. To bridge the gap to the final frontier of neural prostheses based on imagined speech processes, we investigate the differences between audibly produced and imagined speech and present first results towards BCI from imagined speech processes. This dissertation demonstrates the usage of speech processes as a paradigm for BCI for the first time. Speech processes offer a fast and natural interaction paradigm which will help patients and healthy users alike to communicate with computers and with friends and family efficiently through BCIs

Psychophysiology-based QoE assessment : a survey

We present a survey of psychophysiology-based assessment for quality of experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarize multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardized experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research

Using non-invasive stimulation of the undamaged brain to guide the identification of lesion sites that predict language outcome after stroke

Disrupting the neural activity in the left anterior supramarginal gyrus (SMG) or opercular part of the left inferior frontal gyrus (pOp) with repetitive transcranial magnetic stimulation (TMS) has been demonstrated to cause a transient slowing of response times during phonologically more than semantically demanding tasks. Likewise, a wealth of functional magnetic resonance imaging (fMRI) studies have shown increased activation in SMG and/or pOp for phonological relative to semantic processing. Here I set out to investigate whether, and how frequently, stroke damage to SMG and/or pOp results in persistent phonological processing impairments in a large sample of 262 right-handed English-speaking adults, who were tested at least 1 year after a left-hemisphere stroke. In Experiment I, I compared the effect of damage to different parts of SMG and pOp that were defined by regions of interest from either TMS or fMRI studies of phonological processing in neurologically-normal individuals. I found that the incidence of phonological processing impairments was predicted significantly better by the presence or absence of damage to SMG and pOp regions defined by TMS studies than SMG and pOp regions defined by fMRI studies. Moreover, the discriminatory power (for segregating patients with and without phonological abilities) of the TMS sites was not improved further when combined with the fMRI sites. In Experiment II, I adapted the borders of the TMS SMG and pOp regions to include the surrounding grey and white matter where the presence or absence of stroke damage was consistently associated with the presence or absence of phonological processing impairments. The presence or absence of damage to these new TMS-guided regions was able to explain the incidence of phonological impairments better than the original TMS regions, even in a new sample of patients that was entirely independent of the region identification process. In Experiment III, I showed that damage to the TMS-guided regions accounted for the incidence of phonological impairments substantially better than damage to an alternative set of regions derived from voxel-based lesion-deficit mapping techniques that search the whole brain for areas that are most frequently damaged in those with phonological impairments. However, the best classification accuracy was observed when the analysis took into account a combination of regions from TMS-guided and voxel-based lesion-deficit mapping approaches. In Experiment IV, I investigated the nature of the functional impairment caused by SMG or pOp lesions and found that damage to either region impaired covert and overt phonological processing abilities more than semantic processing abilities, as predicted by prior TMS and fMRI studies of neurologically-normal subjects. Finally, the behavioural effects of damage were remarkably similar (i.e. no statistically significant differences) for both TMS-guided sites (i.e. pOp and SMG). In conclusion, the fact that damage to the TMS-guided SMG and pOp regions impaired phonological processing abilities years after stroke onset, suggests that these regions are critical for accurate phonological processing (both overt and covert) and that other brain areas are not typically able to fully compensate for the contribution that these regions make to language processing. More broadly, the results illustrate how non-invasive stimulation of the undamaged brain can be used to guide the identification of regions where brain damage is likely to cause persistent behavioural effects. By combining these regions of interest with those derived from other lesion-deficit mapping approaches, I was not only able to explain the presence, but also the absence, of phonological processing impairments in a large cohort of patients

Development of a Unique Whole-Brain Model for Upper Extremity Neuroprosthetic Control

Neuroprostheses are at the forefront of upper extremity function restoration. However, contemporary controllers of these neuroprostheses do not adequately address the natural brain strategies related to planning, execution and mediation of upper extremity movements. These lead to restrictions in providing complete and lasting restoration of function. This dissertation develops a novel whole-brain model of neuronal activation with the goal of providing a robust platform for an improved upper extremity neuroprosthetic controller. Experiments (N=36 total) used goal-oriented upper extremity movements with real-world objects in an MRI scanner while measuring brain activation during functional magnetic resonance imaging (fMRI). The resulting data was used to understand neuromotor strategies using brain anatomical and temporal activation patterns. The study\u27s fMRI paradigm is unique and the use of goal-oriented movements and real-world objects are crucial to providing accurate information about motor task strategy and cortical representation of reaching and grasping. Results are used to develop a novel whole-brain model using a machine learning algorithm. When tested on human subject data, it was determined that the model was able to accurately distinguish functional motor tasks with no prior knowledge. The proof of concept model created in this work should lead to improved prostheses for the treatment of chronic upper extremity physical dysfunction