Long-Term Asynchronous Decoding of Arm Motion Using Electrocorticographic Signals in Monkeys

Brain–machine interfaces (BMIs) employ the electrical activity generated by cortical neurons directly for controlling external devices and have been conceived as a means for restoring human cognitive or sensory-motor functions. The dominant approach in BMI research has been to decode motor variables based on single-unit activity (SUA). Unfortunately, this approach suffers from poor long-term stability and daily recalibration is normally required to maintain reliable performance. A possible alternative is BMIs based on electrocorticograms (ECoGs), which measure population activity and may provide more durable and stable recording. However, the level of long-term stability that ECoG-based decoding can offer remains unclear. Here we propose a novel ECoG-based decoding paradigm and show that we have successfully decoded hand positions and arm joint angles during an asynchronous food-reaching task in monkeys when explicit cues prompting the onset of movement were not required. Performance using our ECoG-based decoder was comparable to existing SUA-based systems while evincing far superior stability and durability. In addition, the same decoder could be used for months without any drift in accuracy or recalibration. These results were achieved by incorporating the spatio-spectro-temporal integration of activity across multiple cortical areas to compensate for the lower fidelity of ECoG signals. These results show the feasibility of high-performance, chronic and versatile ECoG-based neuroprosthetic devices for real-life applications. This new method provides a stable platform for investigating cortical correlates for understanding motor control, sensory perception, and high-level cognitive processes

Effects of Random External Background Stimulation on Network Synaptic Stability After Tetanization: A Modeling Study

We constructed a simulated spiking neural network model to investigate the effects of random background stimulation on the dynamics of network activity patterns and tetanus induced network plasticity. The simulated model was a “leaky integrate-and-fire” (LIF) neural model with spike-timing-dependent plasticity (STDP) and frequency-dependent synaptic depression. Spontaneous and evoked activity patterns were compared with those of living neuronal networks cultured on multielectrode arrays. To help visualize activity patterns and plasticity in our simulated model, we introduced new population measures called Center of Activity (CA) and Center of Weights (CW) to describe the spatio-temporal dynamics of network-wide firing activity and network-wide synaptic strength, respectively. Without random background stimulation, the network synaptic weights were unstable and often drifted after tetanization. In contrast, with random background stimulation, the network synaptic weights remained close to their values immediately after tetanization. The simulation suggests that the effects of tetanization on network synaptic weights were difficult to control because of ongoing synchronized spontaneous bursts of action potentials, or “barrages.” Random background stimulation helped maintain network synaptic stability after tetanization by reducing the number and thus the influence of spontaneous barrages. We used our simulated network to model the interaction between ongoing neural activity, external stimulation and plasticity, and to guide our choice of sensory-motor mappings for adaptive behavior in hybrid neural-robotic systems or “hybrots.

MEART: The Semi-Living Artist

Here, we and others describe an unusual neurorobotic project, a merging of art and science called MEART, the semi-living artist. We built a pneumatically actuated robotic arm to create drawings, as controlled by a living network of neurons from rat cortex grown on a multi-electrode array (MEA). Such embodied cultured networks formed a real-time closed-loop system which could now behave and receive electrical stimulation as feedback on its behavior. We used MEART and simulated embodiments, or animats, to study the network mechanisms that produce adaptive, goal-directed behavior. This approach to neural interfacing will help instruct the design of other hybrid neural-robotic systems we call hybrots. The interfacing technologies and algorithms developed have potential applications in responsive deep brain stimulation systems and for motor prosthetics using sensory components. In a broader context, MEART educates the public about neuroscience, neural interfaces, and robotics. It has paved the way for critical discussions on the future of bio-art and of biotechnology

Higher-Order Partial Least Squares (HOPLS) : a generalized multi-linear regression method

A new generalized multilinear regression model, termed the Higher-Order Partial Least Squares (HOPLS), is introduced with the aim to predict a tensor (multiway array) Y from a tensor X through projecting the data onto the latent space and performing regression on the corresponding latent variables. HOPLS differs substantially from other regression models in that it explains the data by a sum of orthogonal Tucker tensors, while the number of orthogonal loadings serves as a parameter to control model complexity and prevent overfitting. The low dimensional latent space is optimized sequentially via a deflation operation, yielding the best joint subspace approximation for both X and Y. Instead of decomposing X and Y individually, higher order singular value decomposition on a newly defined generalized cross-covariance tensor is employed to optimize the orthogonal loadings. A systematic comparison on both synthetic data and real-world decoding of 3D movement trajectories from electrocorticogram (ECoG) signals demonstrate the advantages of HOPLS over the existing methods in terms of better predictive ability, suitability to handle small sample sizes, and robustness to noise.Fil: Zhao, Qibin . RIKEN Brain Science Institute; JapónFil: Caiafa, Cesar Federico. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico La Plata. Instituto Argentino de Radioastronomia (i); ArgentinaFil: Mandic, Danilo P. . Imperial College Of Science And Technology; Reino UnidoFil: Chao, Zenas C. . RIKEN Brain Science Institute; JapónFil: Nagasaka, Yasuo . RIKEN Brain Science Institute; JapónFil: Fujii, Naotaka. RIKEN Brain Science Institute; JapónFil: Zhang, Liqing. Shanghai Jiao Tong University; ChinaFil: Cichocki, Andrzej. RIKEN Brain Science Institute; Japó

Shaping Embodied Neural Networks for Adaptive Goal-directed Behavior

The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves

Long-Term Activity-Dependent Plasticity of Action Potential Propagation Delay and Amplitude in Cortical Networks

Background: The precise temporal control of neuronal action potentials is essential for regulating many brain functions. From the viewpoint of a neuron, the specific timings of afferent input from the action potentials of its synaptic partners determines whether or not and when that neuron will fire its own action potential. Tuning such input would provide a powerful mechanism to adjust neuron function and in turn, that of the brain. However, axonal plasticity of action potential timing is counter to conventional notions of stable propagation and to the dominant theories of activity-dependent plasticity focusing on synaptic efficacies. Methodology/Principal Findings: Here we show the occurrence of activity-dependent plasticity of action potentia

Toward the neurocomputer: goal-directed learning in embodied cultured networks

Brains display very high-level parallel computation, fault-tolerance, and adaptability, all of which are what we struggle to recreate in engineered systems. The neurocomputer (an organic computer built from living neurons) seems possible and may lead to a new generation of computing device that can operate in a brain-like manner. Cultured neuronal networks on multi-electrode arrays (MEAs) are one of the best candidates for the neurocomputer for their controllability, accessibility, flexibility, and the ability to self-organize. I explored the possibility of the neurocomputer by studying whether we can show goal-directed learning, one of the most fascinating behavior of brains, in cultured networks. Inspired by the brain, which needs to be embodied in some way and interact with its surroundings in order to give a purpose to its activities, we have developed tools for closing the sensory-motor loop between a cultured network and a robot or an artificial animal (an animat), termed a ¡§hybrot¡¨. In order to efficiently find an effective closed-loop design among infinite potential options, I constructed a biologically-inspired simulated network. By using this simulated network, I designed: (1) a statistic that can effectively and efficiently decode network functional plasticity, and (2) feedback stimulations and an adaptive training algorithm to encode sensory information and to direct network plasticity. By closing the sensory-motor loop with these decoding and encoding designs, we successfully demonstrated a simple adaptive goal-directed behavior: learning to move in a user-defined direction, and further showed that multiple tasks could be learned simultaneously. These results suggest that even though a cultured network lacks the 3-D structure of the brain, it still can be functionally shaped and show meaningful behavior. To our knowledge, this is the first demonstration of goal-directed learning in embodied cultured networks. Extending from these findings, I further proposed a research plan to optimize closed-loop designs for evaluating the maximal learning capacity (or even true intelligence) of the cultured network. Knowledge gained from effective closed-loop designs provides insights about learning and memory in the nervous system, which could influence the design of neurocomputers, future artificial neural networks, and more effective neuroprosthetics.Ph.D.Committee Chair: Potter, Steve; Committee Member: Butera, Robert; Committee Member: DeMarse, Thomas; Committee Member: Jaeger, Dieter; Committee Member: Lee, Rober

Fatigue acts in spatially distinguishable subpopulations.

A: Response size RA as a function of distance from the center of the stimulation site, evaluated across all data sets in 200 bins between 0 mm and 6.5 mm, and normalized by the response size in recovered networks (initial trial). Each of the 200 data points represents the mean response across all relevant trials of all neurons in the corresponding distance bin. Left, spatial illustration, using all data with a fictive stimulus location. Right, the same data plotted as curves. The vertical line indicates the chosen local/global divider. B: As in panel A, but showing TA without normalization. C: As in panel A, but showing DA without normalization. D: Mean TA in neurons A-as-std and A-as-dev in the local subset, and between A-in-con and A-as-dev in the global subset. E: Corresponding mean DA in local and global portions of the network.</p

Deviance detection under model ablation.

A: Sample network responses to target stimulation, with the deviance detection index in each instance noted in the plot titles. Lines show the average (trial mean +- SEM, left axis) network-wide spike counts in target trials in each sequence. Boxes summarize the spike count totals per trial (right axis), the means of which form the basis for index calculations (see panel B). Note the different axis scaling on the top row (without TA) and the bottom row (with TA). B: Deviance detection indices across data sets. Asterisks indicate data greater than 0, established with a one-tailed Wilcoxon signed-rank test at a significance level 0.05. See main text for detailed statistics.</p