Network-based brain computer interfaces: principles and applications

Brain-computer interfaces (BCIs) make possible to interact with the external environment by decoding the mental intention of individuals. BCIs can therefore be used to address basic neuroscience questions but also to unlock a variety of applications from exoskeleton control to neurofeedback (NFB) rehabilitation. In general, BCI usability critically depends on the ability to comprehensively characterize brain functioning and correctly identify the user s mental state. To this end, much of the efforts have focused on improving the classification algorithms taking into account localized brain activities as input features. Despite considerable improvement BCI performance is still unstable and, as a matter of fact, current features represent oversimplified descriptors of brain functioning. In the last decade, growing evidence has shown that the brain works as a networked system composed of multiple specialized and spatially distributed areas that dynamically integrate information. While more complex, looking at how remote brain regions functionally interact represents a grounded alternative to better describe brain functioning. Thanks to recent advances in network science, i.e. a modern field that draws on graph theory, statistical mechanics, data mining and inferential modelling, scientists have now powerful means to characterize complex brain networks derived from neuroimaging data. Notably, summary features can be extracted from these networks to quantitatively measure specific organizational properties across a variety of topological scales. In this topical review, we aim to provide the state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability

The Inter-Subject Correlation of EEG in Response to Naturalistic Stimuli

Inter-subject correlation is a measure of the similarity of the brain activity of a group of people as they respond to the same naturalistic stimulus, typically a story or video, meant to simulate a real world experience. This thesis tests the hypothesis that the correlation of the brain responses of a group of people is indicative of stimulus engagement. The rationale is that the content of the stimulus drives brain activity in a consistent manner, while internal thoughts are divergent and result in uncorrelated activity. The inter-subject correlation (ISC) of neural responses have previously been assessed with fMRI, EEG, and MEG. Here, EEG will assess ISC, thereby examining the correlation of the early responses to a stimulus. Engagement has been examined previously with self-report assessments of interest. These ratings are noisy, subject to bias, and do not measure how engagement evolves over time. In this thesis, engagement is defined as a commitment to devote a scarce resource, such as attention or time, to a stimulus. In the experiment presented here, subjects were allowed limited time with the stimuli, thus forcing them to engage with the content they determined to be most compelling. This behavioral metric strongly correlated with ISC of the EEG, thus validating it as a measure of neural engagement. Interestingly, higher ISC was also indicative of a shared perception of the passage of time across subjects. This suggests that when people are engaged with a stimulus, their perception of time is also driven by that stimulus, rather than by an internal sense of time. If people are more engaged at the time of encoding, it is likely that they will better remember their experiences. Memory was therefore assessed three weeks after subjects heard salient emotional narratives. Individuals whose EEG responses during the stories correlated more strongly with their peers had stronger memories of the events in the stories. ISC was also tested as a predictor of retention in the context of online educational videos. Again, the similarity between each subject’s brain activity and that of his or her peers corresponded with memory for factual information in a subsequent test. It is possible that people with different backgrounds do not engage with the world in similar ways, and their neural responses will therefore correlate more strongly with people who are most similar to them. To address this notion, ISC was compared across the dimensions of age and gender. In a population with ages ranging from 5 - 44 years old, ISC weakens with age and is stronger in males than it is in females. This result is consistent with the idea that age and experience are marked by an increase in the repertoire of neural representations. Adults may therefore have more variable interpretations that mediate their sensory responses to stimuli. Alternatively, if ISC is truly assessing engagement in this context, the result may demonstrate that adults are less susceptible to the influence of outside stimuli since they have more powerful internal voices that distract them. Whichever the ultimate reason for this change, the gender disparity may also be related to a developmental difference because the deviation between males and females in ISC is strongest in young ages, a period when anatomical findings show that young males are less neurally mature than young females. Although ISC is implicated in fundamental processes such as engagement, memory, and development, the neural underpinnings of this signal are unclear. The spatial distribution of the EEG signal that drives ISC appears similar on the surface of the scalp across stimuli with different narrative content, and between different stimulus modalities. The similarity of the topography of correlated activity across sensory modalities may indicate that this activity is supramodal and is therefore generated by a region that is impervious to the stimulus modality. To assess ISC’s dependence on stimulus modality and stimulus type, the modulation of ISC was compared with the fMRI BOLD responses to the same stimuli. This analysis revealed that ISC is mostly modulated by sensory regions, and that the extent of the regions involved depends on the content of the stimulus. These areas, which are largely driven by immediate processing of the stimulus at a fast timescale, are therefore implicated in higher-level behaviors such as engagement and memory

Neuroimaging of endogenous lapses of responsiveness,

Attention lapses (ALs) and microsleeps (MSs) are complete lapses of responsiveness in which performance is completely disrupted for a short period of time, but consciousness is retained in the case of ALs. ALs are behaviourally different from MSs, as in an AL the eyes remain open whereas in a MS eyes are partially or completely closed. Both ALs and MSs can result in catastrophic consequences, especially in the transportation sector. Research over the past two decades has investigated the AL and MS phenomena using behavioural and physiological means. However, both ALs and MSs need further investigation to separate the different types of ALs physiologically, and to explore the neural signature of MSs in relation to normal sleep and drowsiness. Hence, the objective of this project was to understand the underlying physiological substrates of endogenous (internal) ALs and MSs which could potentially result in differentiating types of ALs and provide more understanding of MSs. Data from two previous Christchurch Neurotechnology Research Programme (NeuroTech™) studies (C and D) were combined resulting in a total of 40 subjects. During each session, subjects performed a 2-D continuous visuomotor tracking (CVT) task for 50 min (Study C) and 20 min (Study D). For each participant, tracking performance, eye-video, EEG, and fMRI were simultaneously collected. A human expert visually inspected the tracking performance and eye-video recordings to identify and categorize lapses of responsiveness for each participant. Participants performed the 2-D CVT task without interruptions. The repetitive nature of the task and the lack of a motivational factor made the task monotonous and fatiguing. As a result, it was more likely to introduce boredom leading to task-unrelated thoughts (TUTs), which divides attention between the task and the internal thoughts unrelated to the task, also fatigue which will introduce a trend of vigilance decrement over time. The project had hypotheses focusing on the changes in the brain’s activity compared to the baseline of good responsiveness tracking. We expected a decrease in dorsal attention network (DAN) activity during ALs due to a decoupling of attention from the external environment. Furthermore, we hypothesized that the ALs were due to involuntary mind-blanks. As such, we expected no change in default mode network (DMN) activity, as would have otherwise been expected if the ALs were due to mind-wandering. Functional connectivity (FC) of the brain was also investigated between the networks of interest which were the DMN, DAN, frontoparietal network (FPN), sensorimotor network (SMN), visual network (VSN), salience network (SN), eye-movement network (EMN), and working memory network (WMN), by analysing data from fMRI. EEG data were also used to perform analysis on ALs and MSs, by analysing changes in power in the delta, theta, alpha, beta, and gamma bands. Voxel-wise fMRI throughout the whole brain, group-ICA, haemodynamic response (HR) over the regions of interest (ROIs), and FC analyses were performed to reveal the neural signature during ALs. In voxel-wise analysis, a significant increase in activity was found in two regions: the dorsal anterior cingulate cortex (dACC) and the supplementary motor area (SMA). The group-ICA analysis did not show any significant results but did show a trend of increased activity in an independent component (IC) that was spatially correlated with SMN. Dynamic HR analysis was performed to further investigate findings from the voxel-wise analysis. Our results were not significant but there were strong trends of change. There was a trend of increased HR 7.5 s after the onset of the AL in the left intraparietal sulcus (IPS) of the DAN. There was also a decrease of 2.5 s before the onset of the AL in the right posterior parietal cortex (PPC) of the FPN. There was also an increase in the HR 5 s after the onset of the AL in the dACC of the SN. Finally, an increase in the HR 15 s before the onset of ALs in the left inferior parietal lobule (IPL) of the DMN is a major finding, as it is an indication that a lapse is about to happen. The HR analysis provided consistent findings with the voxel-wise analysis. FC analysis showed increases in FC within all networks of interest during the ALs. On looking at FC between networks, there was an increase in FC between the DMN and the FPN, no change between the DAN and the FPN, a decrease in FC between the SMN and the FPN, and an increase in FC between the FPN and the VSN. The EMN had an increased FC with the DMN, while it had both increases and decreases in FC with the DAN. There was also an increase in FC between the SN and the DAN, and no change between the SN and the DMN. Finally, a decrease in FC was found between the WMN and the DMN. These findings indicate an overlap between decoupling due to ALs and the process of recovery from ALs. The EEG analysis showed no significant change in the relative difference between average spectral power during ALs and their average baselines for any band of interest for ALs. During MSs, there was a significant increase in power relative to responsive baselines in the delta, theta, alpha, beta, and gamma bands. However, we could not be completely sure that all motion-related artefacts had been removed. Hence, we investigated this further by removing the effect of the global signal, which left only an increase in gamma activity, in addition to a trend of decreased activity in the alpha band. The significant increase in BOLD seen in the voxel-wise analysis is considered to represent the recovery of responsiveness following ALs. This was also seen in trends in group ICA and HR analyses. Overall, findings from the FC analysis show that, in addition to decoupling during ALs, and recovery from ALs, it is highly likely that the ALs during the 2-D CVT task were due to involuntary mind-blanks. This is supported by three major findings: (1) no significant increase in DMN activity in both voxel-wise and HR analyses, (2) the decrease in the HR in the FPN prior to the onset of the AL, and (3) the decrease in FC between the DMN and the WMN. This is further supported behaviourally by the short average duration of ALs (~ 1.7 s), in contrast to what would be likely during mind-wandering. Finally, the significant results from the EEG analysis of MSs, agreed with the literature in delta, theta, and alpha bands. However, increased power in beta and gamma bands was an important finding. We consider this increased high-frequency activity reflects unconscious ‘cognitive’ activity during a MS aimed at restoring consciousness after having fallen asleep during an active task. This highlights a key behavioural and physiological difference between MSs and sleep. Even after removing the effect of the global signal, we still believe that MSs and sleep are physiologically different in the recovery process. To summarize our key findings: (1) this is the first study to demonstrate that ALs during a continuous task are likely to be due to involuntary mind-blanks, (2) the increase in the HR in the DMN 15 s before the onset of AL could be a predictive signature of these lapses, and finally (3) MSs are physiologically different from sleep in terms of the recovery process. This project has improved our understanding of endogenous ALs and MSs and taken us a step closer to accurate detection/prediction systems which can increase prevention of fatal accidents

Network science and the effects of music on the human brain

Most people choose to listen to music that they prefer or like such as classical, country or rock. Previous research has focused on how different characteristics of music (i.e., classical versus country) affect the brain. Yet, when listening to preferred music regardless of the type--people report they often experience personal thoughts and memories. To date, understanding how this occurs in the brain has remained elusive. Using network science methods, I evaluated differences in functional brain connectivity when individuals listened to complete songs. Here the results reveal that a circuit important for internally focused thoughts, known as the default mode network, was most connected when listening to preferred music. The results also reveal that listening to a favorite song alters the connectivity between auditory brain areas and the hippocampus, a region responsible for memory and social emotion consolidation. Given that musical preferences are uniquely individualized phenomena and that music can vary in acoustic complexity and the presence or absence of lyrics, the consistency of these results was contrary to previous neuroscientific understanding. These findings may explain why comparable emotional and mental states can be experienced by people listening to music that differs as widely as Beethoven and Eminem. The neurobiological and neurorehabilitation implications of these results are discussed

To investigate power of brain activity using EEG comparison between creative and non-creative design task

In recent times, neurophysiological measurement methods such as EEG and fMRI are widely used in an Engineering field to study designer’s brain activity during creative thinking. In literature, many researchers reported the synchronization and desynchronization of EEG activity in specific brain cortex during creative thinking. However, we do not find many studies associated to comparison of designer’s brain activity during creativity/non-creativity related task demands. The chief objective of present thesis is to investigate the power of brain activity using EEG comparison between creative and non-creative design task. For psychometric measures of creative thinking, Torrance Test of Creative Thinking (TTCT) (Torrance, 1966) is widely used. In present thesis, we use modified TTCT according to our experiment requirement. The test was decomposed between creative and non-creative design task. In creative design task, designers were instructed to think creatively whereas in non-creative design task they were required to think intuitively. When designers were performing these design tasks, their EEG recordings were obtained to investigate brain activity during design tasks. The EEG powers are calculated through spectral analysis. We aggregate electrode positions to identify distribution of EEG powers among brain regions during cognitive task performance. In order to compare EEG between creative and non-creative design task using cortical area to area approach, we perform repeated measure ANOVA for within-subject factors such as design task and brain areas. However, we found non-significant interaction effect between creative/non-creative design task and cortical areas

2018 - The Twenty-third Annual Symposium of Student Scholars

The full program book from the Twenty-third Annual Symposium of Student Scholars, held on April 19, 2018. Includes abstracts from the presentations and posters.https://digitalcommons.kennesaw.edu/sssprograms/1020/thumbnail.jp

Bihemispheric reorganization of neuronal activity during hand movements after unilateral inactivation of the primary motor cortex

Le cortex moteur primaire (M1) est souvent endommagé lors des lésions cérébrales telles que les accidents vasculaires cérébraux. Ceci entraîne des déficits moteurs tels qu'une perte de contrôle des membres controlatéraux. La récupération des lésions M1 s'accompagne d'une réorganisation hémodynamique dans les zones motrices intactes des deux hémisphères. Cette réorganisation est plus prononcée dans les premiers jours et semaines qui suivent la lésion. Toutefois, nous avons une compréhension limitée de la réorganisation neuronale rapide qui se produit dans ce réseau moteur cortical complexe. Ces changements neuronaux nous informent sur l’évolution possible de la plasticité subaiguë impliquée dans la récupération motrice. Par conséquent il était grand temps qu’une caractérisation de la réorganisation rapide de l'activité neuronale dans les régions motrices des deux hémisphères soit entreprise. Dans cette thèse nous avons exploré l'impact d'une lésion corticale localisée, unilatérale et réversible dans M1 sur l'activité neuronale des zones motrices des hémisphères ipsi et contralésionnel lorsque des primates non humains ont effectués des mouvements d’atteinte et de saisie. Notre modèle d'inactivation nous a permis d'enregistrer en continu des neurones isolés avant et après l'apparition des déficits moteurs. Dans une première étude, la réorganisation rapide qui se produit dans le cortex prémoteur ventral (PMv) des deux hémisphères a été étudiée (Chapitre 2). Le PMv est une zone connue pour être impliquée dans le contrôle moteur de la main et la récupération des lésions M1. Dans une seconde étude, la réorganisation rapide du M1 contralésionnel (cM1) a été étudiée et comparée à celles se produisant dans les PMv bilatérales (Chapitre 3). Le cM1 joue un rôle complexe dans la récupération des mouvements de précision de la main suite à une blessure à son homologue. Nous révélons une réorganisation neuronale importante et beaucoup plus complexe que prévu dans les deux hémisphères lors de l’apparition initiale des déficiences motrices. Nos données démontrent que les changements neuronaux survenant quelques minutes après une lésion cérébrale sont hétérogènes à la fois dans et entre les zones du réseau moteur cortical. Ils se produisent dans les deux hémisphères lors des mouvements des bras parétiques et non parétiques, et ils varient au cours des différentes phases du mouvement. Ces découvertes constituent une première étape nécessaire pour démêler les corrélats neuronaux complexes de la réorganisation au travers du réseau moteur des deux hémisphères à la suite d’une lésion cérébrale.After brain injuries such as stroke, the primary motor cortex (M1) is often damaged leading to motor deficits that include a loss of fine motor skills of the contralateral limbs. Recovery from M1 lesions is accompanied by hemodynamic reorganization in motor areas distal to the site of injury in both hemispheres that are most pronounced early after injury. However, we have limited understanding of the rapid neuronal reorganization that occurs in this complex and distributed cortical motor network. As these neural changes reflect the landscape on which subacute plasticity involved in motor recovery will take place, an exploration of the rapid reorganization in neural activity that occurs in motor regions of both hemispheres is long overdue. In the current thesis, we set out to explore the impact of a localized, unilateral and reversible cortical injury to the M1 hand area on neuronal activity in motor-related areas of both the ipsi and contralesional hemispheres as non-human primates performed a reach and grasp task. Our inactivation model allowed us to continuously record isolated neurons before and after the onset of motor deficits. In a first study, the rapid reorganization taking place in the ventral premotor cortex (PMv) of both hemispheres was investigated (Chapter 2). The PMv is an area well-known to be critically involved in hand motor control and recovery from M1 lesions. In a second study, the rapid reorganization taking place in the contralesional M1 (cM1) was studied and compared to those occurring in bilateral PMv (Chapter 3). The cM1 has a complex role in recovery of dexterous hand movements following injury to its homologue. We reveal extensive, and much more complex than expected, neuronal reorganization in both hemispheres at the very onset of motor impairments. Our data demonstrate that neuronal changes occurring within minutes after brain injury are heterogenous both within and across areas of the cortical motor network. They occur in the two hemispheres during movements of both the paretic and non-paretic arms, and they vary during different phases of movement. These findings constitute a first step in a much needed and timely effort to unravel the complex neuronal correlates of the reorganization that takes place across the distributed motor network after brain injury

The Effect of Social Interaction on the Neural Correlates of Language Processing and Mentalizing

Recent behavioral and neuroscience evidence suggests that studying the social brain in detached and offline contexts (e.g., listening to prerecorded stories about characters) may not capture real-world social processes. Few studies, however, have directly compared neural activation during live interaction to conventional recorded paradigms. The current study used a novel fMRI paradigm to investigate whether real-time social interaction modulates the neural correlates of language processing and mentalizing. Regions associated with social engagement (i.e., dorsal medial prefrontal cortex) were more active during live interaction. Processing live versus recorded language increased activation in regions associated with narrative processing and mentalizing (i.e., temporal parietal junction). Regions associated with intentionality understanding (i.e., posterior superior temporal sulcus) were more active when mentalizing about a live partner. These results have implications for quantifying and understanding the neural correlates of real-world social behavior in typical adults, in developmental populations, and in individuals with social disabilities such as autism

Salience Coding in the Basal Forebrain and the Heterogeneous Underpinnings Underlying Novelty Computations

Humans and animals are consistently learning from the environment by interacting with it and getting feedback from their actions. In the environment, some objects are more important than others, because they are associated with reward, uncertainty, surprise, or novelty etc. These objects are salient to the animal. Salient objects attract attention and orientation, increase arousal, facilitate learning and memory, and affect reinforcement learning and credit assignment. However, the neural basis to support these effects is still not fully understood.We first studied how the basal forebrain, one of the principal sources of modulation of the neocortex, encodes salience events. We found two types of neurons that process salient events in distinct manners: one with phasic burst activity to cues predicting salient events and one with ramping activity anticipating such events. Bursting neurons respond to reward itself and cues that predict the magnitude, probability, and timing of reward. However, they do not have a selective response to reward omission. Thus, bursting neurons signal surprise associated with external events, which is different from the reward prediction error signaled by the midbrain dopamine neurons. Furthermore, they discriminate fully expected novel visual objects from familiar objects and respond to object-sequence violations. In contrast, ramping neurons predict the timing of many salient, novel, and surprising events. Their ramping activity is highly sensitive to the subjects\u27 confidence in event timing and on average encodes the subjects\u27 surprise after unexpected events occur. These data suggest that the primate BF contains mechanisms to anticipate the timing of a diverse set of salient external events (via tonic ramping activity) and to rapidly deploy cognitive resources when these events occur (via phasic bursting activity). Then we sailed out to study one special salience signal – Novelty. The basal forebrain responds to novelty, but the neuronal mechanisms of novelty detection remain unclear. Prominent theories propose that novelty is either derived from the computation of recency or is a form of sensory surprise. Here, we used high-channel electrophysiology in primates to show that, in many prefrontal, temporal, and subcortical brain areas, object novelty sensitivity is related to both computations of recency (the sensitivity to how long ago a stimulus was experienced) and sensory surprise (violation of predictions about incoming sensory information). Also, we studied neuronal novelty-to-familiarity transformations during learning across many days and found a diversity of timescales in neurons\u27 learning rates and between-session forgetting rates within and across brain regions that is well suited to support flexible behavior and learning in response to novelty. These findings show that novelty sensitivity arises on multiple timescales across single neurons due to diverse related computations of sensory surprise and recency, and shed light on the logic and computational underpinnings of novelty detection in the primate brain

Proceedings of the 1st European conference on disability, virtual reality and associated technologies (ECDVRAT 1996)

The proceedings of the conferenc