Improving Brain–Machine Interface Performance by Decoding Intended Future Movements

Objective. A brain–machine interface (BMI) records neural signals in real time from a subject\u27s brain, interprets them as motor commands, and reroutes them to a device such as a robotic arm, so as to restore lost motor function. Our objective here is to improve BMI performance by minimizing the deleterious effects of delay in the BMI control loop. We mitigate the effects of delay by decoding the subject\u27s intended movements a short time lead in the future. Approach. We use the decoded, intended future movements of the subject as the control signal that drives the movement of our BMI. This should allow the user\u27s intended trajectory to be implemented more quickly by the BMI, reducing the amount of delay in the system. In our experiment, a monkey (Macaca mulatta) uses a future prediction BMI to control a simulated arm to hit targets on a screen. Main Results. Results from experiments with BMIs possessing different system delays (100, 200 and 300 ms) show that the monkey can make significantly straighter, faster and smoother movements when the decoder predicts the user\u27s future intent. We also characterize how BMI performance changes as a function of delay, and explore offline how the accuracy of future prediction decoders varies at different time leads. Significance. This study is the first to characterize the effects of control delays in a BMI and to show that decoding the user\u27s future intent can compensate for the negative effect of control delay on BMI performance

Learning and adaptation in brain machine interfaces

Balancing subject learning and decoder adaptation is central to increasing brain machine interface (BMI) performance. We addressed these complementary aspects in two studies: (1) a learning study, in which mice modulated “beta” band activity to control a 1D auditory cursor, and (2) an adaptive decoding study, in which a simple recurrent artificial neural network (RNN) decoded intended saccade targets of monkeys. In the learning study, three mice successfully increased beta band power following trial initiations, and specifically increased beta burst durations from 157 ms to 182 ms, likely contributing to performance. Though the task did not explicitly require specific movements, all three mice appeared to modulate beta activity via active motor control and had consistent vibrissal motor cortex multiunit activity and local field potential relationships with contralateral whisker pad electromyograms. The increased burst durations may therefore by a direct result of increased motor activity. These findings suggest that only a subset of beta rhythm phenomenology can be volitionally modulated (e.g. the tonic “hold” beta), therefore limiting the possible set of successful beta neuromodulation strategies. In the adaptive decoding study, RNNs decoded delay period activity in oculomotor and working memory regions while monkeys performed a delayed saccade task. Adaptive decoding sessions began with brain-controlled trials using pre-trained RNN models, in contrast to static decoding sessions in which 300-500 initial eye-controlled training trials were performed. Closed loop RNN decoding performance was lower than predicted by offline simulations. More consistent delay period activity and saccade paths across trials were associated with higher decoding performance. Despite the advantage of consistency, one monkey’s delay period activity patterns changed over the first week of adaptive decoding, and the other monkey’s saccades were more erratic during adaptive decoding than during static decoding sessions. It is possible that the altered session paradigm eliminating eye-controlled training trials led to either frustration or exploratory learning, causing the neural and behavioral changes. Considering neural control and decoder adaptation of BMIs in these studies, future work should improve the “two-learner” subject-decoder system by better modeling the interaction between underlying brain states (and possibly their modulation) and the neural signatures representing desired outcomes

Toward More Versatile and Intuitive Cortical Brain–Machine Interfaces

Brain–machine interfaces have great potential for the development of neuroprosthetic applications to assist patients suffering from brain injury or neurodegenerative disease. One type of brain–machine interface is a cortical motor prosthetic, which is used to assist paralyzed subjects. Motor prosthetics to date have typically used the motor cortex as a source of neural signals for controlling external devices. The review will focus on several new topics in the arena of cortical prosthetics. These include using: recordings from cortical areas outside motor cortex; local field potentials as a source of recorded signals; somatosensory feedback for more dexterous control of robotics; and new decoding methods that work in concert to form an ecology of decode algorithms. These new advances promise to greatly accelerate the applicability and ease of operation of motor prosthetics

Cortical motor prosthetics: the development and use for paralysis

The emerging research field of Brain Computer Interfaces (BCIs) has created an invasive type of BCI, the Cortical Motor Prosthetic (CMP) or invasive BCI (iBCI). The goal is to restore lost motor function via prosthetic control signals to individuals who have long-term paralysis. The development of the CMP consists of two major entities: the implantable, chronic microelectrode array (MEA) and the data acquisition hardware (DAQ) specifically the decoder. The iBCI's function is to record primary motor cortex (M1) neural signals via chronic MEA and translate into a motor command via decoder extraction algorithms that can control a prosthetic to perform the intended movement. The ultimate goal is to use the iBCI as a clinical tool for individuals with long-term paralysis to regain lost motor functioning. Thus, the iBCI is a beacon of hope that could enable individuals to independently perform daily activities and interact once again with their environment. This review seeks to accomplish two major goals. First, elaborate upon the development of the iBCI and focus on the advancements and efforts to create a viable system. Second, illustrate the exciting improvements in the iBCI's use for reaching and grasping actions and in human clinical trials. The ultimate goal is to use the iBCI as a clinical tool for individuals with long-term paralysis to regain movement control. Despite the promise in the iBCI, many challenges, which are described in this review, persist and must be overcome before the iBCI can be a viable tool for individuals with long-term. iBCI future endeavors aim to overcome the challenges and develop an efficient system enhancing the lives of many living with paralysis. Standard terms: Intracortical Brain Computer Interface (iBCI), Intracortical Brain Machine Interface (iBMI), Cortical Motor Prosthetic (CMP), Neuromotor Prostheses (NMP), Intracortical Neural Prosthetics, Invasive Neural Prosthetic all terms used interchangeabl

Brain-Machine Interfaces: A Tale of Two Learners

Brain-machine interface (BMI) technology has rapidly matured over the last two decades, mainly thanks to the introduction of artificial intelligence (AI) methods, in particular, machine-learning algorithms. Yet, the need for subjects to learn to modulate their brain activity is a key component of successful BMI control. Blending machine and subject learning, or mutual learning, is widely acknowledged in the BMI field. Nevertheless, we posit that current research trends are heavily biased toward the machine-learning side of BMI training. In this article, we take a critical view of the relevant literature, and our own previous work, to identify the key issues for more effective mutual-learning schemes in translational BMIs that are specifically tailored to promote subject learning. We identify the main caveats in the literature on subject learning in BMI, in particular, the lack of longitudinal studies involving end users and shortcomings in quantifying subject learning, and pinpoint critical improvements for future experimental designs

Efficient human-machine control with asymmetric marginal reliability input devices

Input devices such as motor-imagery brain-computer interfaces (BCIs) are often unreliable. In theory, channel coding can be used in the human-machine loop to robustly encapsulate intention through noisy input devices but standard feedforward error correction codes cannot be practically applied. We present a practical and general probabilistic user interface for binary input devices with very high noise levels. Our approach allows any level of robustness to be achieved, regardless of noise level, where reliable feedback such as a visual display is available. In particular, we show efficient zooming interfaces based on feedback channel codes for two-class binary problems with noise levels characteristic of modalities such as motor-imagery based BCI, with accuracy <75%. We outline general principles based on separating channel, line and source coding in human-machine loop design. We develop a novel selection mechanism which can achieve arbitrarily reliable selection with a noisy two-state button. We show automatic online adaptation to changing channel statistics, and operation without precise calibration of error rates. A range of visualisations are used to construct user interfaces which implicitly code for these channels in a way that it is transparent to users. We validate our approach with a set of Monte Carlo simulations, and empirical results from a human-in-the-loop experiment showing the approach operates effectively at 50-70% of the theoretical optimum across a range of channel conditions

A New Frontier: The Convergence of Nanotechnology, Brain Machine Interfaces, and Artificial Intelligence

A confluence of technological capabilities is creating an opportunity for machine learning and artificial intelligence (AI) to enable “smart” nanoengineered brain machine interfaces (BMI). This new generation of technologies will be able to communicate with the brain in ways that support contextual learning and adaptation to changing functional requirements. This applies to both invasive technologies aimed at restoring neurological function, as in the case of neural prosthesis, as well as non-invasive technologies enabled by signals such as electroencephalograph (EEG). Advances in computation, hardware, and algorithms that learn and adapt in a contextually dependent way will be able to leverage the capabilities that nanoengineering offers the design and functionality of BMI. We explore the enabling capabilities that these devices may exhibit, why they matter, and the state of the technologies necessary to build them. We also discuss a number of open technical challenges and problems that will need to be solved in order to achieve this

Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography

One of the hottest topics in rehabilitation robotics is that of proper control of prosthetic devices. Despite decades of research, the state of the art is dramatically behind the expectations. To shed light on this issue, in June, 2013 the first international workshop on Present and future of non-invasive PNS-Machine Interfaces was convened, hosted by the International Conference on Rehabilitation Robotics. The keyword PNS-Machine Interface (PMI) has been selected to denote human-machine interfaces targeted at the limb-deficient, mainly upper-limb amputees, dealing with signals gathered from the peripheral nervous system (PNS) in a non-invasive way, that is, from the surface of the residuum. The workshop was intended to provide an overview of the state of the art and future perspectives of such interfaces; this paper represents is a collection of opinions expressed by each and every researcher/group involved in it

Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography

abstract: One of the hottest topics in rehabilitation robotics is that of proper control of prosthetic devices. Despite decades of research, the state of the art is dramatically behind the expectations. To shed light on this issue, in June, 2013 the first international workshop on Present and future of non-invasive peripheral nervous system (PNS)–Machine Interfaces (MI; PMI) was convened, hosted by the International Conference on Rehabilitation Robotics. The keyword PMI has been selected to denote human–machine interfaces targeted at the limb-deficient, mainly upper-limb amputees, dealing with signals gathered from the PNS in a non-invasive way, that is, from the surface of the residuum. The workshop was intended to provide an overview of the state of the art and future perspectives of such interfaces; this paper represents is a collection of opinions expressed by each and every researcher/group involved in it

New perspectives on learning, inference, and control in brains and machines

The work presented in this thesis provides new perspectives and approaches for problems that arise in the analysis of neural data. Particular emphasis is placed on parameter fitting and automated analysis problems that would arise naturally in closed-loop experiments. Part one focuses on two brain-computer interface problems. First, we provide a framework for understanding co-adaptation, the setting in which decoder updating and user learning occur simultaneously. We also provide a new perspective on intention-based parameter fitting and tools to extend this approach to higher dimensional decoders. Part two focuses on event inference, which refers to the decomposition of observed timeseries data into interpretable events. We present application of event inference methods on voltage-clamp recordings as well as calcium imaging, and describe extensions to allow for combining data across modalities or trials