A Synergetic Brain-Machine Interfacing Paradigm for Multi-DOF Robot Control

This paper proposes a novel brain-machine interfacing (BMI) paradigm for control of a multijoint redundant robot system. Here, the user would determine the direction of end-point movement of a 3-degrees of freedom (DOF) robot arm using motor imagery electroencephalography signal with co-adaptive decoder (adaptivity between the user and the decoder) while a synergetic motor learning algorithm manages a peripheral redundancy in multi-DOF joints toward energy optimality through tacit learning. As in human motor control, torque control paradigm is employed for a robot to be adaptive to the given physical environment. The dynamic condition of the robot arm is taken into consideration by the learning algorithm. Thus, the user needs to only think about the end-point movement of the robot arm, which allows simultaneous multijoints control by BMI. The support vector machine-based decoder designed in this paper is adaptive to the changing mental state of the user. Online experiments reveals that the users successfully reach their targets with an average decoder accuracy of over 75% in different end-point load conditions

Empowering and assisting natural human mobility: The simbiosis walker

This paper presents the complete development of the Simbiosis Smart Walker. The device is equipped with a set of sensor subsystems to acquire user-machine interaction forces and the temporal evolution of user's feet during gait. The authors present an adaptive filtering technique used for the identification and separation of different components found on the human-machine interaction forces. This technique allowed isolating the components related with the navigational commands and developing a Fuzzy logic controller to guide the device. The Smart Walker was clinically validated at the Spinal Cord Injury Hospital of Toledo - Spain, presenting great acceptability by spinal chord injury patients and clinical staf

Autonomy Infused Teleoperation with Application to BCI Manipulation

Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments

Development of a hybrid robotic system based on an adaptive and associative assistance for rehabilitation of reaching movement after stroke

Stroke causes irreversible neurological damage. Depending on the location and the size of this brain injury, different body functions could result affected. One of the most common consequences is motor impairments. The level of motor impairment affectation varies between post-stroke subjects, but often, it hampers the execution of most activities of daily living. Consequently, the quality of life of the stroke population is severely decreased. The rehabilitation of the upper-limb motor functions has gained special attention in the scientific community due the poor reported prognosis of post-stroke patients for recovering normal upper-extremity function after standard rehabilitation therapy. Driven by the advance of technology and the design of new rehabilitation methods, the use of robot devices, functional electrical stimulation and brain-computer interfaces as a neuromodulation system is proposed as a novel and promising rehabilitation tools. Although the uses of these technologies present potential benefits with respect to standard rehabilitation methods, there still are some milestones to be addressed for the consolidation of these methods and techniques in clinical settings. Mentioned evidences reflect the motivation for this dissertation. This thesis presents the development and validation of a hybrid robotic system based on an adaptive and associative assistance for rehabilitation of reaching movements in post-stroke subjects. The hybrid concept refers the combined use of robotic devices with functional electrical stimulation. Adaptive feature states a tailored assistance according to the users’ motor residual capabilities, while the associative term denotes a precise pairing between the users’ motor intent and the peripheral hybrid assistance. The development of the hybrid platform comprised the following tasks: 1. The identification of the current challenges for hybrid robotic system, considering twofold perspectives: technological and clinical. The hybrid systems submitted in literature were critically reviewed for such purpose. These identified features will lead the subsequent development and method framed in this work. 2. The development and validation of a hybrid robotic system, combining a mechanical exoskeleton with functional electrical stimulation to assist the execution of functional reaching movements. Several subsystems are integrated within the hybrid platform, which interact each other to cooperatively complement the rehabilitation task. Complementary, the implementation of a controller based on functional electrical stimulation to dynamically adjust the level of assistance is addressed. The controller is conceived to tackle one of the main limitations when using electrical stimulation, i.e. the highly nonlinear and time-varying muscle response. An experimental procedure was conducted with healthy and post-stroke patients to corroborate the technical feasibility and the usability evaluation of the system. 3. The implementation of an associative strategy within the hybrid platform. Three different strategies based on electroencephalography and electromyography signals were analytically compared. The main idea is to provide a precise temporal association between the hybrid assistance delivered at the periphery (arm muscles) and the users’ own intention to move and to configure a feasible clinical setup to be use in real rehabilitation scenarios. 4. Carry out a comprehensive pilot clinical intervention considering a small cohort of patient with post-stroke patients to evaluate the different proposed concepts and assess the feasibility of using the hybrid system in rehabilitation settings. In summary, the works here presented prove the feasibility of using the hybrid robotic system as a rehabilitative tool with post-stroke subjects. Moreover, it is demonstrated the adaptive controller is able to adjust the level of assistance to achieve successful tracking movement with the affected arm. Remarkably, the accurate association in time between motor cortex activation, represented through the motor-related cortical potential measured with electroencephalography, and the supplied hybrid assistance during the execution of functional (multidegree of freedom) reaching movement facilitate distributed cortical plasticity. These results encourage the validation of the overall hybrid concept in a large clinical trial including an increased number of patients with a control group, in order to achieve more robust clinical results and confirm the presented herein.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Ramón Ceres Ruiz.- Secretario: Luis Enrique Moreno Lorente.- Vocal: Antonio Olivier

Toward a model-based predictive controller design in brain-computer interfaces

A first step in designing a robust and optimal model-based predictive controller (MPC) for brain-computer interface (BCI) applications is presented in this article. An MPC has the potential to achieve improved BCI performance compared to the performance achieved by current ad hoc, nonmodel-based filter applications. The parameters in designing the controller were extracted as model-based features from motor imagery task-related human scalp electroencephalography. Although the parameters can be generated from any model-linear or non-linear, we here adopted a simple autoregressive model that has well-established applications in BCI task discriminations. It was shown that the parameters generated for the controller design can as well be used for motor imagery task discriminations with performance (with 8-23% task discrimination errors) comparable to the discrimination performance of the commonly used features such as frequency specific band powers and the AR model parameters directly used. An optimal MPC has significant implications for high performance BCI applications.Grants K25NS061001 (MK) and K02MH01493 (SJS) from the National Institute of Neurological Disorders And Stroke (NINDS) and the National Institute of Mental Health (NIMH), the Portuguese Foundation for Science and Technology (FCT) Grant SFRH/BD/21529/2005 (NSD), the Pennsylvania Department of Community and Economic Development Keystone Innovation Zone Program Fund (SJS), and the Pennsylvania Department of Health using Tobacco Settlement Fund (SJS)

Tracking Control for Non-Minimum Phase System and Brain Computer Interface

For generations, humans dreamed about the ability to communicate and interact with machines through thought alone or to create devices that can peer into a person’s mind and thoughts. Researchers have developed new technologies to create brain computer interfaces (BCIs), communication systems that do not depend on the brain’s normal output pathways of peripheral nerves and muscles. The objective of the first part of this thesis is to develop a new BCI based on electroencephalography (EEG) to move a computer cursor over a short training period in real time. The work motivations of this part are to increase: speed and accuracy, as in BCI settings, subject has a few seconds to make a selection with a relatively high accuracy. Recently, improvements have been developed to make EEG more accurate by increasing the spatial resolution. One such improvement is the application of the surface Laplacian to the EEG, the second spatial derivative. Tripolar concentric ring electrodes (TCREs) automatically perform the Laplacian on the surface potentials and provide better spatial selectivity and signal-to-noise ratio than conventional EEG that is recorded with conventional disc electrodes. Another important feature using TCRE is the capability to record the EEG and the TCRE EEG (tEEG) signals concurrently from the same location on the scalp for the same electrical activity coming from the brain. In this part we also demonstrate that tEEG signals can enable users to control a computer cursor rapidly in different directions with significantly higher accuracy during their first session of training for 1D and 2D cursor control. Output tracking control of non-minimum phase systems is a highly challenging problem encountered in many practical engineering applications. Classical inversion techniques provide exact output tracking but lead to internal instability, whereas modern inversion methods provide stable asymptotic tracking but produce large transient errors. Both methods provide an approximation of feedback control, which leads to non robust systems, very sensitive to noise, considerable tracking errors and a significant singularity problem. Aiming at the problem of system inversion to the true system, the objective of the second part of this thesis is to develop a new method based on true inversion for minimum phase system and approximate inversion for non-minimum phase systems. The proposed algorithm is automatic and has minimal computational complexities which make it suitable for real-time control. The process to develop the proposed algorithm is partitioned into (1) minimum phase feedforward inverse filter, and (2) non-minimum phase inversion. In a minimum phase inversion, we consider the design of a feedforward controller to invert the response of a feedback loop that has stable zero locations. The complete control system consists of a feedforward controller cascaded with a closed-loop system. The outputs of the resulting inverse filter are delayed versions of the corresponding reference input signals, and delays are given by the vector relative degree of the closed-loop

Bio-signal based control in assistive robots: a survey

Recently, bio-signal based control has been gradually deployed in biomedical devices and assistive robots for improving the quality of life of disabled and elderly people, among which electromyography (EMG) and electroencephalography (EEG) bio-signals are being used widely. This paper reviews the deployment of these bio-signals in the state of art of control systems. The main aim of this paper is to describe the techniques used for (i) collecting EMG and EEG signals and diving these signals into segments (data acquisition and data segmentation stage), (ii) dividing the important data and removing redundant data from the EMG and EEG segments (feature extraction stage), and (iii) identifying categories from the relevant data obtained in the previous stage (classification stage). Furthermore, this paper presents a summary of applications controlled through these two bio-signals and some research challenges in the creation of these control systems. Finally, a brief conclusion is summarized

Closed-Loop Brain-Computer Interfaces for Memory Restoration Using Deep Brain Stimulation

The past two decades have witnessed the rapid growth of therapeutic brain-computer interfaces (BCI) targeting a diversity of brain dysfunctions. Among many neurosurgical procedures, deep brain stimulation (DBS) with neuromodulation technique has emerged as a fruitful treatment for neurodegenerative disorders such as epilepsy, Parkinson\u27s disease, post-traumatic amnesia, and Alzheimer\u27s disease, as well as neuropsychiatric disorders such as depression, obsessive-compulsive disorder, and schizophrenia. In parallel to the open-loop neuromodulation strategies for neuromotor disorders, recent investigations have demonstrated the superior performance of closed-loop neuromodulation systems for memory-relevant disorders due to the more sophisticated underlying brain circuitry during cognitive processes. Our efforts are focused on discovering unique neurophysiological patterns associated with episodic memories then applying control theoretical principles to achieve closed-loop neuromodulation of such memory-relevant oscillatory activity, especially, theta and gamma oscillations. First, we use a unique dataset with intracranial electrodes inserted simultaneously into the hippocampus and seven cortical regions across 40 human subjects to test for the presence of a pattern that the phase of hippocampal theta oscillation modulates gamma oscillations in the cortex, termed cross-regional phase-amplitude coupling (xPAC), representing a key neurophysiological mechanism that promotes the temporal organization of interregional oscillatory activities, which has not previously been observed in human subjects. We then establish that the magnitude of xPAC predicts memory encoding success along with other properties of xPAC. We find that strong functional xPAC occurs principally between the hippocampus and other mesial temporal structures, namely entorhinal and parahippocampal cortices, and that xPAC is overall stronger for posterior hippocampal connections. Next, we focus on hippocampal gamma power as a `biomarker\u27 and use a novel dataset in which open-loop DBS was applied to the posterior cingulate cortex (PCC) during the encoding of episodic memories. We evaluate the feasibility of modulating hippocampal power by a precise control of stimulation via a linear quadratic integral (LQI) controller based on autoregressive with exogenous input (ARX) modeling for in-vivo use. In the simulation framework, we demonstrate proposed BCI system achieves effective control of hippocampal gamma power in 15 out of 17 human subjects and we show our DBS pattern is physiologically safe with realistic time scales. Last, we further develop the PCC-applied binary-noise (BN) DBS paradigm targeting the neuromodulation of both hippocampal theta and gamma oscillatory power in 12 human subjects. We utilize a novel nonlinear autoregressive with exogenous input neural network (NARXNN) as the plant paired with a proportional–integral–derivative (PID) controller (NARXNN-PID) for delivering a precise stimulation pattern to achieve desired oscillatory power level. Compared to a benchmark consisted of a linear state-space model (LSSM) with a PID controller, we not only demonstrate that the superior performance of our NARXNN plant model but also show the greater capacity of NARXNN-PID architecture in controlling both hippocampal theta and gamma power. We outline further experimentation to test our BCI system and compare our findings to emerging closed-loop neuromodulation strategies

Intrinsic and Extrinsic Biomechanical Factors in a Co-adaptive ECoG-based Brain Computer Interface

Paralysis, due to spinal cord injury, amyotrophic lateral sclerosis (ALS), or stroke, is the result of severed communication between the brain and the motor periphery. Brain computer interfaces (BCIs) are neuroprosthetic devices that create novel communication pathways by measuring and transforming neural activity into operational commands. State of the art BCI systems measure brain activity using penetrating electrode arrays able to record from hundreds of individual cortical neurons simultaneously. Unfortunately, these systems are highly susceptible to signal degradation which limits their efficacy to 1-2 years. However, electrocorticography (ECoG) signals recorded from the surface of the brain deliver a more competitive balance between surgical risk, long-term stability, signal bandwidth, and signal-to-noise ratio when compared to both the aforementioned intracortical systems and the more common non-invasive electroencephalography (EEG) technologies. Historically, neural signals for controlling a computer cursor or robotic arm have been mapped to extrinsic, kinematic (i.e. position or velocity) variables. Although this strategy is adequate for use in simple environments, it may not be ideal for control of real-world prosthetic devices that are subject to external and unexpected forces. When reaching for an object, the trajectory of the hand through space can be defined in either extrinsic (e.g. Cartesian) or intrinsic (e.g. joint angles, muscle forces) frames of reference. During this movement, the brain has to perform a series of sensorimotor transformations that involve solving a complex, 2nd order differential equation (i.e. musculoskeletal biomechanics) in order to determine the appropriate muscle activations. Functional neuromuscular stimulation (FNS) is a desirable BCI application because it attempts to restore motor function to paralyzed limbs through electrical excitation of muscles. Rather than applying the conventional extrinsic kinematic control signals to such a system, it may be more appropriate to map neural activity to muscle activation directly and allow the brain to develop its own transfer function. This dissertation examines the application of intrinsic decoding schemes to control an upper limb using ECoG in non-human primates. ECoG electrode arrays were chronically implanted in rhesus monkeys over sensorimotor cortex. A novel multi-joint reaching task was developed to train the subjects to control a virtual arm simulating muscle and inertial forces. Utilizing a co-adaptive algorithm (where both the brain adapts via biofeedback and the decoding algorithm adapts to improve performance), new decoding models were initially built over the course of the first 3-5 minutes of each daily experimental session and then continually adapted throughout the day. Three subjects performed the task using neural control signals mapped to 1) joint angular velocity, 2) joint torque, and 3) muscle forces of the virtual arm. Performance exceeded 97%, 93%, and 89% accuracy for the three control paradigms respectively. Neural control features in the upper gamma frequency bands (70-115 and 130-175 Hz) were found to be directionally tuned in an ordered fashion, with preferred directions varying topographically in the mediolateral direction without distinction between motor and sensory areas. Long-term stability was demonstrated by all three monkeys, which maintained performance at 42, 55, and 57 months post-implantation. These results provide insights into the capabilities of sensorimotor cortex for control of non-linear multi-joint reaching dynamics and present a first step toward design of intrinsic, force-based BCI systems suitable for long-term FNS applications