A simulation study on the effects of neuronal ensemble properties on decoding algorithms for intracortical brain-machine interfaces

Background: Intracortical brain-machine interfaces (BMIs) harness movement information by sensing neuronal activities using chronic microelectrode implants to restore lost functions to patients with paralysis. However, neuronal signals often vary over time, even within a day, forcing one to rebuild a BMI every time they operate it. The term "rebuild" means overall procedures for operating a BMI, such as decoder selection, decoder training, and decoder testing. It gives rise to a practical issue of what decoder should be built for a given neuronal ensemble. This study aims to address it by exploring how decoders' performance varies with the neuronal properties. To extensively explore a range of neuronal properties, we conduct a simulation study. Methods: Focusing on movement direction, we examine several basic neuronal properties, including the signal-to-noise ratio of neurons, the proportion of well-tuned neurons, the uniformity of their preferred directions (PDs), and the non-stationarity of PDs. We investigate the performance of three popular BMI decoders: Kalman filter, optimal linear estimator, and population vector algorithm. Results: Our simulation results showed that decoding performance of all the decoders was affected more by the proportion of well-tuned neurons that their uniformity. Conclusions: Our study suggests a simulated scenario of how to choose a decoder for intracortical BMIs in various neuronal conditions

Low-dimensional representations of neural time-series data with applications to peripheral nerve decoding

Bioelectronic medicines, implanted devices that influence physiological states by peripheral neuromodulation, have promise as a new way of treating diverse conditions from rheumatism to diabetes. We here explore ways of creating nerve-based feedback for the implanted systems to act in a dynamically adapting closed loop. In a first empirical component, we carried out decoding studies on in vivo recordings of cat and rat bladder afferents. In a low-resolution data-set, we selected informative frequency bands of the neural activity using information theory to then relate to bladder pressure. In a second high-resolution dataset, we analysed the population code for bladder pressure, again using information theory, and proposed an informed decoding approach that promises enhanced robustness and automatic re-calibration by creating a low-dimensional population vector. Coming from a different direction of more general time-series analysis, we embedded a set of peripheral nerve recordings in a space of main firing characteristics by dimensionality reduction in a high-dimensional feature-space and automatically proposed single efficiently implementable estimators for each identified characteristic. For bioelectronic medicines, this feature-based pre-processing method enables an online signal characterisation of low-resolution data where spike sorting is impossible but simple power-measures discard informative structure. Analyses were based on surrogate data from a self-developed and flexibly adaptable computer model that we made publicly available. The wider utility of two feature-based analysis methods developed in this work was demonstrated on a variety of datasets from across science and industry. (1) Our feature-based generation of interpretable low-dimensional embeddings for unknown time-series datasets answers a need for simplifying and harvesting the growing body of sequential data that characterises modern science. (2) We propose an additional, supervised pipeline to tailor feature subsets to collections of classification problems. On a literature standard library of time-series classification tasks, we distilled 22 generically useful estimators and made them easily accessible.Open Acces

Learning and adaptation in brain machine interfaces

Balancing subject learning and decoder adaptation is central to increasing brain machine interface (BMI) performance. We addressed these complementary aspects in two studies: (1) a learning study, in which mice modulated “beta” band activity to control a 1D auditory cursor, and (2) an adaptive decoding study, in which a simple recurrent artificial neural network (RNN) decoded intended saccade targets of monkeys. In the learning study, three mice successfully increased beta band power following trial initiations, and specifically increased beta burst durations from 157 ms to 182 ms, likely contributing to performance. Though the task did not explicitly require specific movements, all three mice appeared to modulate beta activity via active motor control and had consistent vibrissal motor cortex multiunit activity and local field potential relationships with contralateral whisker pad electromyograms. The increased burst durations may therefore by a direct result of increased motor activity. These findings suggest that only a subset of beta rhythm phenomenology can be volitionally modulated (e.g. the tonic “hold” beta), therefore limiting the possible set of successful beta neuromodulation strategies. In the adaptive decoding study, RNNs decoded delay period activity in oculomotor and working memory regions while monkeys performed a delayed saccade task. Adaptive decoding sessions began with brain-controlled trials using pre-trained RNN models, in contrast to static decoding sessions in which 300-500 initial eye-controlled training trials were performed. Closed loop RNN decoding performance was lower than predicted by offline simulations. More consistent delay period activity and saccade paths across trials were associated with higher decoding performance. Despite the advantage of consistency, one monkey’s delay period activity patterns changed over the first week of adaptive decoding, and the other monkey’s saccades were more erratic during adaptive decoding than during static decoding sessions. It is possible that the altered session paradigm eliminating eye-controlled training trials led to either frustration or exploratory learning, causing the neural and behavioral changes. Considering neural control and decoder adaptation of BMIs in these studies, future work should improve the “two-learner” subject-decoder system by better modeling the interaction between underlying brain states (and possibly their modulation) and the neural signatures representing desired outcomes

Low-frequency local field potentials in primate motor cortex and their application to neural interfaces

PhD ThesisFor patients with spinal cord injury and paralysis, there are currently very limited options for clinical therapy. Brain-machine interfaces (BMIs) are neuroprosthetic devices that are being developed to record from the motor cortex in such patients, bypass the spinal lesion, and use decoded signals to control an effector, such as a prosthetic limb. The ideal BMI would be durable, reliable, totally predictable, fully-implantable, and have generous battery life. Current, state-of-the-art BMIs are limited in all of these domains; partly because the typical signals used—neuronal action potentials, or ‘spikes’—are very susceptible to micro-movement of recording electrodes. Recording spikes from the same neurons over many months is therefore difficult, and decoder behaviour may be unpredictable from day-today. Spikes also need to be digitized at high frequencies (~104 Hz) and heavily processed. As a result, devices are energy-hungry and difficult to miniaturise. Low-frequency local field potentials (lf-LFPs; < 5 Hz) are an alternative cortical signal. They are more stable and can be captured and processed at much lower frequencies (~101 Hz). Here we investigate rhythmical lf-LFP activity, related to the firing of local cortical neurons, during isometric wrist movements in Rhesus macaques. Multichannel spike-related slow potentials (SRSPs) can be used to accurately decode the firing rates of individual motor cortical neurons, and subjects can control a BMI task using this synthetic signal, as if they were controlling the actual firing rate. Lf-LFP–based firing rate estimates are stable over time – even once actual spike recordings have been lost. Furthermore, the dynamics of lf-LFPs are distinctive enough, that an unsupervised approach can be used to train a decoder to extract movement-related features for use in biofeedback BMIs. Novel electrode designs may help us optimise the recording of these signals, and facilitate progress towards a new generation of robust, implantable BMIs for patients.Research Studentship from the MRC, and Andy Jackson’s laboratory (hence this work) is supported by the Wellcome Trust

Developing robust movement decoders for local field potentials

textBrain Computer Interfaces (BCI) are devices that translate acquired neural signals to command and control signals. Applications of BCI include neural rehabilitation and neural prosthesis (thought controlled wheelchair, thought controlled speller etc.) to aid patients with disabilities and to augment human computer interaction. A successful practical BCI requires a faithful acquisition modality to record high quality neural signals; a signal processing system to construct appropriate features from these signals; and an algorithm to translate these features to appropriate outputs. Intracortical recordings like local field potentials provide reliable high SNR signals over long periods and suit BCI applications well. However, the non-stationarity of neural signals poses a challenge in robust decoding of subject behavior. Most BCI research focuses either on developing daily re-calibrated decoders that require exhaustive training sessions; or on providing cross-validation results. Such results ignore the variation of signal characteristics over different sessions and provide an optimistic estimate of BCI performance. Specifically, traditional BCI algorithms fail to perform at the same level on chronological data recordings. Neural signals are susceptible to variations in signal characteristics due to changes in subject behavior and learning, and variability in electrode characteristics due to tissue interactions. While training day-specific BCI overcomes signal variability, BCI re-training causes user frustration and exhaustion. This dissertation presents contributions to solve these challenges in BCI research. Specifically, we developed decoders trained on a single recording session and applied them on subsequently recorded sessions. This strategy evaluates BCI in a practical scenario with a potential to alleviate BCI user frustration without compromising performance. The initial part of the dissertation investigates extracting features that remain robust to changes in neural signal over several days of recordings. It presents a qualitative feature extraction technique based on ranking the instantaneous power of multichannel data. These qualitative features remain robust to outliers and changes in the baseline of neural recordings, while extracting discriminative information. These features form the foundation in developing robust decoders. Next, this dissertation presents a novel algorithm based on the hypothesis that multiple neural spatial patterns describe the variation in behavior. The presented algorithm outperforms the traditional methods in decoding over chronological recordings. Adapting such a decoder over multiple recording sessions (over 6 weeks) provided > 90% accuracy in decoding eight movement directions. In comparison, performance of traditional algorithms like Common Spatial Patterns deteriorates to 16% over the same time. Over time, adaptation reinforces some spatial patterns while diminishing others. Characterizing these spatial patterns reduces model complexity without user input, while retaining the same accuracy levels. Lastly, this dissertation provides an algorithm that overcomes the variation in recording quality. Chronic electrode implantation causes changes in signal-to-noise ratio (SNR) of neural signals. Thus, some signals and their corresponding features available during training become unavailable during testing and vice-versa. The proposed algorithm uses prior knowledge on spatial pattern evolution to estimate unknown neural features. This algorithm overcomes SNR variations and provides up to 93% decoding of eight movement directions over 6 weeks. Since model training requires only one session, this strategy reduces user frustration. In a practical closed-loop BCI, the user learns to produce stable spatial patterns, which improves performance of the proposed algorithms.Electrical and Computer Engineerin

Developing an oculomotor brain-computer interface and charactering its dynamic functional network

To date, invasive brain-computer interface (BCI) research has largely focused on replacing lost limb functions using signals from hand/arm areas of motor cortex. However, the oculomotor system may be better suited to BCI applications involving rapid serial selection from spatial targets, such as choosing from a set of possible words displayed on a computer screen in an augmentative and alternative communication application. First, we develop an intracortical oculomotor BCI based on the delayed saccade paradigm and demonstrate its feasibility to decode intended saccadic eye movement direction in primates. Using activity from three frontal cortical areas implicated in oculomotor production – dorsolateral prefrontal cortex, supplementary eye field, and frontal eye field – we could decode intended saccade direction in real time with high accuracy, particularly at contralateral locations. In a number of analyses in the decoding context, we investigated the amount of saccade-related information contained in different implant regions and in different neural measures. A novel neural measure using power in the 80-500 Hz band is proposed as the optimal signal for this BCI purpose. In the second part of this thesis, we characterize the interactions between the neural signals recorded from electrodes in these three implant areas. We employ a number of techniques to quantify the spectrotemporal dynamics in this complex network, and we describe the resulting functional connectivity patterns between the three implant regions in the context of eye-movement production. In addition, we compare and contrast the amount of saccade-related information present in the coupling strengths in the network, on both an electrode-to-electrode scale and an area-to-area scale. Different frequency bands stand out during different epochs of the task, and their information contents are distinct between implant regions. For example, the 13-30 Hz band stands out during the delay epoch, and the 8-12 Hz band is relevant during target and response epochs. This work extends the boundary of BCI research into the oculomotor domain, and invites potential applications by showing its feasibility. Furthermore, it elucidates the complex dynamics of the functional coupling underlying oculomotor production across multiple areas of frontal cortex

Developing implant technologies and evaluating brain-machine interfaces using information theory

Brain-machine interfaces (BMIs) hold promise for restoring motor functions in severely paralyzed individuals. Invasive BMIs are capable of recording signals from individual neurons and typically provide the highest signal-to-noise ratio. Despite many efforts in the scientific community, BMI technology is still not reliable enough for widespread clinical application. The most prominent challenges include biocompatibility, stability, longevity, and lack of good models for informed signal processing and BMI comparison. To address the problem of low signal quality of chronic probes, in the first part of the thesis one such design, the Neurotrophic Electrode, was modified by increasing its channel capacity to form a Neurotrophic Array (NA). Specifically, single wires were replaced with stereotrodes and the total number of recording wires was increased. This new array design was tested in a rhesus macaque performing a delayed saccade task. The NA recorded little single unit spiking activity, and its local field potentials (LFPs) correlated with presented visual stimuli and saccade locations better than did extracted spikes. The second part of the thesis compares the NA to the Utah Array (UA), the only other micro-array approved for chronic implantation in a human brain. The UA recorded significantly more spiking units, which had larger amplitudes than NA spikes. This was likely due to differences in the array geometry and construction. LFPs on the NA electrodes were more correlated with each other than those on the UA. These correlations negatively impacted the NA's information capacity when considering more than one recording site. The final part of this dissertation applies information theory to develop objective measures of BMI performance. Currently, decoder information transfer rate (ITR) is the most popular BMI information performance metric. However, it is limited by the selected decoding algorithm and does not represent the full task information embedded in the recorded neural signal. A review of existing methods to estimate ITR is presented, and these methods are interpreted within a BMI context. A novel Gaussian mixture Monte Carlo method is developed to produce good ITR estimates with a low number of trials and high number of dimensions, as is typical for BMI applications

POPULATION CODES AND THEIR CORRELATES IN DECISION MAKING

This dissertation was organized in two parts: in part 1, we discussed Neural Correlates of Perceptual Accuracy” and in part 2 we discussed “Strategy encoding in Prefrontal Cortex”. Abstract of part 1_The accurate transmission of electrical signals within neocortex is central to sensory perception and cognition. Theoretical studies have long proposed that the temporal coordination of cortical spiking activity controls signal transmission and cognitive function. In reality, whether and how the precise temporal coordination in neuronal populations during wakefulness influences perception remains a mystery. Here, we simultaneously recorded populations of neurons in early and mid-level visual cortex (areas V1 and V4) to discover that the precise temporal coordination between the spikes of three or more neurons carries information about perceptual reports in the absence of firing rate modulation. Perceptual accuracy was correlated with higher-order spiking coordination within V4, but not V1, and with the feedforward coordination between V1 and V4 activity. Our results indicate that while stimulus encoding is related to the discharge rates of neurons, perceptual accuracy is correlated with the precise spiking coordination within visual cortical populations. Abstract of part 2_Foraging animals explore the environment to earn valuable resources, at the lowest cost. Previous experiments on various species suggest that they simply maximize the current flow of rewards without predicting the future outcome of their actions. We found that monkeys are able to predict the reward outcomes to plan ahead, when allowed to forage freely in an interactive environment. In uncertain environments, the prediction of outcome requires access to a model of the reward structure. We recorded the activity of a population of individual neurons in dorso-lateral prefrontal cortex (dlPFC) and found the representation of an internal reward model in this area. We singled out the component in dlPFC activity that represents the reward model and showed that this component predicts the next action. We think our naturalistic experimental setting as well as multi-dimensional analysis of the neural activity was the key to this finding and perhaps can shift the paradigm in studying neural correlates of complex behaviors