Real-Time Decoding of Brain Responses to Visuospatial Attention Using 7T fMRI

Brain-Computer interface technologies mean to create new communication channels between our mind and our environment, independent of the motor system, by detecting and classifying self regulation of local brain activity. BCIs can provide patients with severe paralysis a means to communicate and to live more independent lives. There has been a growing interest in using invasive recordings for BCI to improve the signal quality. This also potentially gives access to new control strategies previously inaccessible by non-invasive methods. However, before surgery, the best implantation site needs to be determined. The blood-oxygen-level dependent signal changes measured with fMRI have been shown to agree well spatially with those found with invasive electrodes, and are the best option for pre-surgical localization. We show, using real-time fMRI at 7T, that eye movement-independent visuospatial attention can be used as a reliable control strategy for BCIs. At this field strength even subtle signal changes can be detected in single trials thanks to the high contrast-to-noise ratio. A group of healthy subjects were instructed to move their attention between three (two peripheral and one central) spatial target regions while keeping their gaze fixated at the center. The activated regions were first located and thereafter the subjects were given real-time feedback based on the activity in these regions. All subjects managed to regulate local brain areas without training, which suggests that visuospatial attention is a promising new target for intracranial BCI. ECoG data recorded from one epilepsy patient showed that local changes in gamma-power can be used to separate the three classes

Real-time decoding of covert attention in higher-order visual areas

Brain-computer-interfaces (BCI) provide a means of using human brain activations to control devices for communication. Until now this has only been demonstrated in primary motor and sensory brain regions, using surgical implants or non-invasive neuroimaging techniques. Here, we provide proof-of-principle for the use of higher-order brain regions involved in complex cognitive processes such as attention. Using realtime fMRI, we implemented an online ‘winner-takes-all approach’ with quadrant-specific parameter estimates, to achieve single-block classification of brain activations. These were linked to the covert allocation of attention to real-world images presented at 4-quadrant locations. Accuracies in three target regions were significantly above chance, with individual decoding accuracies reaching upto 70%. By utilising higher order mental processes, ‘cognitive BCIs’ access varied and therefore more versatile information, potentially providing a platform for communication in patients who are unable to speak or move due to brain injury

Visual imagery during real-time fMRI neurofeedback from occipital and superior parietal cortex

Abstract Visual imagery has been suggested to recruit occipital cortex via feedback projections from fronto-parietal regions, suggesting that these feedback projections might be exploited to boost recruitment of occipital cortex by means of real-time neurofeedback. To test this prediction, we instructed a group of healthy participants to perform peripheral visual imagery while they received real-time auditory feedback based on the BOLD signal from either early visual cortex or the medial superior parietal lobe. We examined the amplitude and temporal aspects of the BOLD response in the two regions. Moreover, we compared the impact of self-rated mental focus and vividness of visual imagery on the BOLD responses in these two areas. We found that both early visual cortex and the medial superior parietal cortex are susceptible to auditory neurofeedback within a single feedback session per region. However, the signal in parietal cortex was sustained for a longer time compared to the signal in occipital cortex. Moreover, the BOLD signal in the medial superior parietal lobe was more affected by focus and vividness of the visual imagery than early visual cortex. Our results thus demonstrate that (a) participants can learn to self-regulate the BOLD signal in early visual and parietal cortex within a single session, (b) that different nodes in the visual imagery network respond differently to neurofeedback, and that (c) responses in parietal, but not in occipital cortex are susceptible to self-rated vividness of mental imagery. Together, these results suggest that medial superior parietal cortex might be a suitable candidate to provide real-time feedback to patients suffering from visual field defects

Applications of realtime fMRI for non-invasive brain computer interface-decoding and neurofeedback

Non-invasive brain-computer interfaces (BCIs) seek to enable or restore brain function by using neuroimaging e.g. functional magnetic resonance imaging (fMRI), to engage brain activations without the need for explicit behavioural output or surgical implants. Brain activations are converted into output signals, for use in communication interfaces, motor prosthetics, or to directly shape brain function via a feedback loop. The aim of this thesis was to develop cognitive BCIs using realtime fMRI (rt-fMRI), with the potential for use as a communication interface, or for initiating neural plasticity to facilitate neurorehabilitation. Rt-fMRI enables brain activation to be manipulated directly to produce changes in function, such as perception. Univariate and multivariate classification approaches were used to decode brain activations produced by the deployment of covert spatial attention to simple visual stimuli. Primary and higher order visual areas were examined, as well as potential control regions. The classification platform was then developed to include the use of real-world visual stimuli, exploiting the use of category-specific visual areas, and demonstrating real-world applicability as a communications interface. Online univariate classification of spatial attention was successfully achieved, with individual classification accuracies for 4-quadrant spatial attention reaching 70%. Further, a novel implementation of m-sequences enabled the use of the timing of stimuli presentation to enhance signal characterisation. An established rt-fMRI analysis loop was then used for neurofeedback-led manipulation of category-specific visual brain regions, modulating their functioning, and, as a result, biasing visual perception during binocular rivalry. These changes were linked with functional and effective connectivity changes in trained regions, as well as in a putative top-down control region. The work presented provides proof-of-principle for non-invasive BCIs using rt-fMRI, with the potential for translation into the clinical environment. Decoding and 4 neurofeedback applied to non-invasive and implantable BCIs form an evolving continuum of options for enabling and restoring brain function

The brain as image processor and generator:towards function-restoring brain-computer-interfaces

As neuroscientists are slowly unraveling the mysteries of the brain, neurotechnology like brain-computer-interfaces (BCIs) might become a new standard for medical applications in those with brain injuries. BCIs allow for direct communication between the brain and a device, and could potentially restore links that are broken due to brain damage. In addition, a better understanding of the human mind and its mechanisms could greatly boost the success of these devices. This dissertation features (high-field) functional magnetic resonance imaging (fMRI) to study human cognitive functioning, as fMRI allows for studying the brain of living humans in great spatial detail. Firstly, the dissertation describes how well brain regions that are important for visual perception can be located between individuals. Some of these regions are in part responsible for recognizing objects like faces, bodies, places and motion. Secondly, differences in functional organization of the brain were explored between individuals by simulating the placement of a visual cortical prosthesis. Such a prosthesis can bypass the (broken) connections between the eye and brain in blind people, and potentially restore a rudimentary form of vision. Finally, new techniques were presented that show that visual perception and mental imagery are closely related, and allow for reading letter shapes directly from the mind. Together, this dissertation adds new foundations for the development of neurotechnological applications

a methodological approach

In natural environments, visual and auditory stimulation elicit responses across a large set of brain regions in a fraction of a second, yielding representations of the multimodal scene and its properties. The rapid and complex neural dynamics underlying visual and auditory information processing pose major challenges to human cognitive neuroscience. Brain signals measured non-invasively are inherently noisy, the format of neural representations is unknown, and transformations between representations are complex and often nonlinear. Further, no single non-invasive brain measurement technique provides a spatio-temporally integrated view. In this opinion piece, we argue that progress can be made by a concerted effort based on three pillars of recent methodological development: (i) sensitive analysis techniques such as decoding and cross-classification, (ii) complex computational modelling using models such as deep neural networks, and (iii) integration across imaging methods (magnetoencephalography/electroencephalography, functional magnetic resonance imaging) and models, e.g. using representational similarity analysis. We showcase two recent efforts that have been undertaken in this spirit and provide novel results about visual and auditory scene analysis. Finally, we discuss the limits of this perspective and sketch a concrete roadmap for future research

Content representation in the human medial temporal lobe

The transformation of sensory inputs into complex memory representations is fundamental to human experience; yet, little is known about how this crucial process is achieved. When you meet your friend at the new cafe in town, what part of the brain encodes this novel scene into long term memory? What part encoded your friend’s favorite t-shirt, so that the sight of it gives you a feeling of familiarity rather than surprise? It is well-established that the medial temporal lobe (MTL) is crucial to both processes, but the MTL is not a single homogeneous region. In fact, it is composed of several anatomically distinct subregions including hippocampus, perirhinal cortex (PRC) and parahippocampal cortex (PHC). However, the computations performed by each subregion to encode individual events is still unclear. The present research tests the central hypothesis that different forms of event content are transformed into memory by distinct subregions within the MTL. A critical barrier in the study of content representation thus far has been its focus on comparing univariate peak activations in a region to different stimulus materials. To go beyond this limited approach, we employed multivariate statistical analyses that takes into account how event content is represented by distributed activity in MTL subregions. First, we examine the content-specific contributions of MTL subregions to episodic encoding and retrieval. Then, we demonstrate how these distributed representations support memory-based prediction to resolve ambiguities in our environment.Neuroscienc

A Multifaceted Approach to Covert Attention Brain-Computer Interfaces

Over the last years, brain-computer interfaces (BCIs) have shown their value for assistive technology and neurorehabilitation. Recently, a BCI-approach for the rehabilitation of hemispatial neglect has been proposed on the basis of covert visuospatial attention (CVSA). CVSA is an internal action which can be described as shifting one's attention to the visual periphery without moving the actual point of gaze. Such attention shifts induce a lateralization in parietooccipital blood flow and oscillations in the so-called alpha band (8-14 Hz), which can be detected via electroencephalography (EEG), magnetoencephalography (MEG) or functional magnetic resonance imaging (fMRI). Previous studies have proven the technical feasibility of using CVSA as a control signal for BCIs, but unfortunately, these BCIs could not provide every subject with sufficient control. The aim of this thesis was to investigate the possibility of amplifying the weak lateralization patterns in the alpha band - the main reason behind insufficient CVSA BCI performance. To this end, I have explored three different approaches that could lead to better performing and more inclusive CVSA BCI systems. The first approach illuminated the changes in the behavior and brain patterns by closing the loop between subject and system with continuous real-time feedback at the instructed locus of attention. I could observe that even short (20 minutes) stretches of real-time feedback have an effect on behavioral correlates of attention, even when the changes observed in the EEG remained less conclusive. The second approach attempted to complement the information extracted fromthe EEG signal with another sensing modality that could provide additional information about the state of CVSA. For this reason, I firstly combined functional functional near-infrared spectroscopy (fNIRS) with EEG measurements. The results showed that, while the EEG was able to pick up the expected lateralization in the alpha band, the fNIRS was not able to reliably image changes in blood circulation in the parietooccipital cortex. Secondly, I successfully combined data from the EEG with measures of pupil size changes, induced by a high illumination contrast between the covertly attended target regions, which resulted in an improved BCI decoding performance. The third approach examined the option of using noninvasive electrical brain stimulation to boost the power of the alpha band oscillations and therefore render the lateralization pattern in the alpha band more visible compared to the background activity. However, I could not observe any impact of the stimulation on the ongoing alpha band power, and thus results of the subsequent effect on the lateralization remain inconclusive. Overall, these studies helped to further understand CVSA and lay out a useful basis for further exploration of the connection between behavior and alpha power oscillations in CVSA tasks, as well as for potential directions to improve CVSA-based BCIs

Static and dynamic resting-state brain activity patterns of table tennis players in 7-Tesla MRI

Table tennis involves quick and accurate motor responses during training and competition. Multiple studies have reported considerably faster visuomotor responses and expertise-related intrinsic brain activity changes among table tennis players compared with matched controls. However, the underlying neural mechanisms remain unclear. Herein, we performed static and dynamic resting-state functional magnetic resonance imaging (rs-fMRI) analyses of 20 table tennis players and 21 control subjects using 7T ultra-high field imaging. We calculated the static and dynamic amplitude of low-frequency fluctuations (ALFF) of the two groups. The results revealed that table tennis players exhibited decreased static ALFF in the left inferior temporal gyrus (lITG) compared with the control group. Voxel-wised static functional connectivity (sFC) and dynamic functional connectivity (dFC) analyses using lITG as the seed region afforded complementary and overlapping results. The table tennis players exhibited decreased sFC in the right middle temporal gyrus and left inferior parietal gyrus. Conversely, they displayed increased dFC from the lITG to prefrontal cortex, particularly the left middle frontal gyrus, left superior frontal gyrus-medial, and left superior frontal gyrus-dorsolateral. These findings suggest that table tennis players demonstrate altered visuomotor transformation and executive function pathways. Both pathways involve the lITG, which is a vital node in the ventral visual stream. These static and dynamic analyses provide complementary and overlapping results, which may help us better understand the neural mechanisms underlying the changes in intrinsic brain activity and network organization induced by long-term table tennis skill training