A Novel Framework of Online, Task-Independent Cognitive State Transition Detection and Its Applications

Complex reach, grasp, and object manipulation tasks require sequential, temporal coordination of a movement plan by neurons in the brain. Detecting cognitive state transitions associated with motor tasks from sequential neural data is pivotal in rehabilitation engineering. The cognitive state detectors proposed thus far rely on task-dependent (TD) models, i.e., the detection strategy exploits a priori knowledge of the movement goals to determine the actual states, regardless of whether these cognitive states actually depend on the movement tasks or not. This approach, however, is not viable when the tasks are not known a priori (e.g., the subject performs many different tasks) or when there is a paucity of neural data for each task. Moreover, some cognitive states (e.g., holding) are invariant to the tasks performs. I first develop an offline, task-dependent cognitive state transition detector and a kinematics decoder to show the feasibility of distinguishing between cognitive states based on their inherent features extracted via a hidden Markov model (HMM) based detection framework. The proposed framework is designed to decode both cognitive states and kinematics from ensemble neural activity. The proposed decoding framework is able to a) automatically differentiate between baseline, plan, and movement, and b) determine novel holding epochs of neural activity and also estimate the epoch-dependent kinematics. Specifically, the framework is mainly composed of a hidden Markov model (HMM) state decoder and a switching linear system (S-LDS) kinematics decoder. I take a supervised approach and use a generative framework of neural activity and kinematics. I demonstrate the decoding framework using neural recordings from ventral premotor (PMv) and dorsal premotor (PMd) neurons of a non-human primate executing four complex reach-to-grasp tasks along with the corresponding kinematics recording. Using the HMM state decoder, I demonstrate that the transitions between neighboring epochs of neural activity, regardless of the existence of any external kinematics changes, can be detected with high accuracy (>85%) and short latencies (<150 ms). I further show that the joint angle kinematics can be estimated reliably with high accuracy (mean = 88%) using a S-LDS kinematics decoder. In addition, I demonstrate that the use of multiple latent state variables to model the within-epoch neural activity variability can improve the decoder performance. This unified decoding framework combining a HMM state decoder and a S-LDS may be useful in neural decoding of cognitive states and complex movements of prosthetic limbs in practical brain-computer interface implementations. I then develop a real-time (online) task-independent (TI) framework to detect cognitive state transitions from spike trains and kinematic measurements. I applied this framework to 226 single-unit recordings collected via multi-electrode arrays in the premotor dorsal and ventral (PMd and PMv) regions of the cortex of two non-human primates performing 3D multi-object reach-to-grasp tasks, and I used the detection latency and accuracy of state transitions to measure the performance. I found that, in both online and offline detection modes, (i) TI models have significantly better performance than TD models when using neuronal data alone, however (ii) during movements, the addition of the kinematics history to the TI models further improves detection performance. These findings suggest that TI models may be able to more accurately detect cognitive state transitions than TD under certain circumstances. The proposed framework could pave the way for a TI control of prosthesis from cortical neurons, a beneficial outcome when the choice of tasks is vast, but despite that the basic movement cognitive states need to be decoded. Based on the online cognitive state transition detector, I further construct an online task-independent kinematics decoder. I constructed our framework using single-unit recordings from 452 neurons and synchronized kinematics recordings from two non-human primates performing 3D multi-object reach-to-grasp tasks. I find that (i) the proposed TI framework performs significantly better than current frameworks that rely on TD models (p = 0.03); and (ii) modeling cognitive state information further improves decoding performance. These findings suggest that TI models with cognitive-state-dependent parameters may more accurately decode kinematics and could pave the way for more clinically viable neural prosthetics

Cortical motor prosthetics: the development and use for paralysis

The emerging research field of Brain Computer Interfaces (BCIs) has created an invasive type of BCI, the Cortical Motor Prosthetic (CMP) or invasive BCI (iBCI). The goal is to restore lost motor function via prosthetic control signals to individuals who have long-term paralysis. The development of the CMP consists of two major entities: the implantable, chronic microelectrode array (MEA) and the data acquisition hardware (DAQ) specifically the decoder. The iBCI's function is to record primary motor cortex (M1) neural signals via chronic MEA and translate into a motor command via decoder extraction algorithms that can control a prosthetic to perform the intended movement. The ultimate goal is to use the iBCI as a clinical tool for individuals with long-term paralysis to regain lost motor functioning. Thus, the iBCI is a beacon of hope that could enable individuals to independently perform daily activities and interact once again with their environment. This review seeks to accomplish two major goals. First, elaborate upon the development of the iBCI and focus on the advancements and efforts to create a viable system. Second, illustrate the exciting improvements in the iBCI's use for reaching and grasping actions and in human clinical trials. The ultimate goal is to use the iBCI as a clinical tool for individuals with long-term paralysis to regain movement control. Despite the promise in the iBCI, many challenges, which are described in this review, persist and must be overcome before the iBCI can be a viable tool for individuals with long-term. iBCI future endeavors aim to overcome the challenges and develop an efficient system enhancing the lives of many living with paralysis. Standard terms: Intracortical Brain Computer Interface (iBCI), Intracortical Brain Machine Interface (iBMI), Cortical Motor Prosthetic (CMP), Neuromotor Prostheses (NMP), Intracortical Neural Prosthetics, Invasive Neural Prosthetic all terms used interchangeabl

Encoding of Motor Behaviors by Cortical Neuronal Networks

Performance of motor behavior requires complex coordination of neural activity across diverse regions of cortex at multiples scales. At the level of coordination across large areas of cortex, this activity is thought to be related to similarly broad concepts of movement from goal identification to motor planning to generation of motor commands. At smaller scales on the level of local populations of individual neurons in motor and premotor cortex, we observe complex non-stationary firing patterns that appear to be related to the movement itself. Our understanding of the details of this relationship are incomplete, however. Earlier work by the community largely focused on the analysis of individual units in isolation. Technological advances and changes in experimental paradigms have led to the simultaneous recording of hundreds of neurons simultaneously. From an analysis standpoint, we are observing a similar shift in focus from the individual neuron to the population as a whole. This dissertation investigates encoding and decoding techniques that handle time-varying neuronal activity from within the context of a reach-to-grasp task. The first part of this work investigates the dynamics of neural coding of reach and grasp through a series of temporally localized classifiers. The second part of this thesis proposes a semi-supervised learning approach to identifying task relevant neurons for classification purposes and for identifying communities of neurons that co-modulate their activity in correlation to a common external variable. The third part of this work proposes and demonstrates an approach to modeling the firing of individual neurons as a weighted combination of other neurons with weighting dependent on the task being performed

De animais a máquinas : humanos tecnicamente melhores nos imaginários de futuro da convergência tecnológica

Dissertação (mestrado)—Universidade de Brasília, Instituto de Ciências Sociais, Departamento de Sociologia, 2020.O tema desta investigação é discutir os imaginários sociais de ciência e tecnologia que emergem a partir da área da neuroengenharia, em sua relação com a Convergência Tecnológica de quatro disciplinas: Nanotecnologia, Biotecnologia, tecnologias da Informação e tecnologias Cognitivas - neurociências- (CT-NBIC). Estas áreas desenvolvem-se e são articuladas por meio de discursos que ressaltam o aprimoramento das capacidades físicas e cognitivas dos seres humanos, com o intuito de construir uma sociedade melhor por meio do progresso científico e tecnológico, nos limites das agendas de pesquisa e desenvolvimento (P&D). Objetivos: Os objetivos nesse cenário, são discutir as implicações éticas, econômicas, políticas e sociais deste modelo de sistema sociotécnico. Nos referimos, tanto as aplicações tecnológicas, quanto as consequências das mesmas na formação dos imaginários sociais, que tipo de relações se estabelecem e como são criadas dentro desse contexto. Conclusão: Concluímos na busca por refletir criticamente sobre as propostas de aprimoramento humano mediado pela tecnologia, que surgem enquanto parte da agenda da Convergência Tecnológica NBIC. No entanto, as propostas de melhoramento humano vão muito além de uma agenda de investigação. Há todo um quadro de referências filosóficas e políticas que defendem o aprimoramento da espécie, vertentes estas que se aliam a movimentos trans-humanistas e pós- humanistas, posições que são ao mesmo tempo éticas, políticas e econômicas. A partir de nossa análise, entendemos que ciência, tecnologia e política estão articuladas, em coprodução, em relação às expectativas de futuros que são esperados ou desejados. Ainda assim, acreditamos que há um espaço de diálogo possível, a partir do qual buscamos abrir propostas para o debate público sobre questões de ciência e tecnologia relacionadas ao aprimoramento da espécie humana.Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)The subject of this research is to discuss the social imaginaries of science and technology that emerge from the area of neuroengineering in relation with the Technological Convergence of four disciplines: Nanotechnology, Biotechnology, Information technologies and Cognitive technologies -neurosciences- (CT-NBIC). These areas are developed and articulated through discourses that emphasize the enhancement of human physical and cognitive capacities, the intuition it is to build a better society, through the scientific and technological progress, at the limits of the research and development (R&D) agendas. Objectives: The objective in this scenery, is to discuss the ethic, economic, politic and social implications of this model of sociotechnical system. We refer about the technological applications and the consequences of them in the formation of social imaginaries as well as the kind of social relations that are created and established in this context. Conclusion: We conclude looking for critical reflections about the proposals of human enhancement mediated by the technology. That appear as a part of the NBIC technologies agenda. Even so, the proposals of human enhancement go beyond boundaries that an investigation agenda. There is a frame of philosophical and political references that defend the enhancement of the human beings. These currents that ally to the transhumanism and posthumanism movements, positions that are ethic, politic and economic at the same time. From our analysis, we understand that science, technology and politics are articulated, are in co-production, regarding the expected and desired futures. Even so, we believe that there is a space of possible dialog, from which we look to open proposals for the public discussion on questions of science and technology related to enhancement of human beings

Network Modeling of Motor Pathways from Neural Recordings

During cued motor tasks, for both speech and limb movement, information propagates from primary sensory areas, to association areas, to primary and supplementary motor and language areas. Through the recent advent of high density recordings at multiple scales, it has become possible to simultaneously observe activity occurring from these disparate regions at varying resolution. Models of brain activity generally used in brain-computer interface (BCI) control do not take into account the global differences in recording site function, or the interactions between them. Through the use of connectivity measures, however, it has been made possible to determine the contribution of individual recording sites to the global activity, as they vary with task progression. This dissertation extends those connectivity models to provide summary information about the importance of individual sites. This is achieved through the application of network measures on the adjacency structure determined by connectivity measures. Similarly, by analyzing the coordinated activity of all of the electrode sites simultaneously during task performance, it is possible to elucidate discrete functional units through clustering analysis of the electrode recordings. In this dissertation, I first describe a BCI system using simple motor movement imagination at single recording sites. I then incorporate connectivity through the use of TV-DBN modeling on higher resolution electrode recordings, specifically electrocorticography (ECoG). I show that PageRank centrality reveals information about task progression and regional specificity which was obscured by direct application of the connectivity measures, due to the combinatorial increase in feature dimensionality. I then show that clustering of ECoG recordings using a method to determine the inherent cluster count algorithmically provides insight into how network involvement in task execution evolves, though in a manner dependent on grid coverage. Finally, I extend clustering analysis to show how individual neurons in motor cortex form distinct functional communities. These communities are shown to be task-specific, suggesting that neurons can form functional units with distinct neural populations across multiple recording sites in a context dependent impermanent manner. This work demonstrates that network measures of connectivity models of neurophysiological recordings are a rich source of information relevant to the field of neuroscience, as well as offering the promise of improved degree-of-freedom and naturalness possible through direct BCI control. These models are shown to be useful at multiple recording scales, from cortical-area level ECoG, to highly localized single unit microelectrode recordings

Brain-Machine Interface for Reaching: Accounting for Target Size, Multiple Motor Plans, and Bimanual Coordination

<p>Brain-machine interfaces (BMIs) offer the potential to assist millions of people worldwide suffering from immobility due to loss of limbs, paralysis, and neurodegenerative diseases. BMIs function by decoding neural activity from intact cortical brain regions in order to control external devices in real-time. While there has been exciting progress in the field over the past 15 years, the vast majority of the work has focused on restoring of motor function of a single limb. In the work presented in this thesis, I first investigate the expanded role of primary sensory (S1) and motor (M1) cortex during reaching movements. By varying target size during reaching movements, I discovered the cortical correlates of the speed-accuracy tradeoff known as Fitts' law. Similarly, I analyzed cortical motor processing during tasks where the motor plan is quickly reprogrammed. In each study, I found that parameters relevant to the reach, such as target size or alternative movement plans, could be extracted by neural decoders in addition to simple kinematic parameters such as velocity and position. As such, future BMI functionality could expand to account for relevant sensory information and reliably decode intended reach trajectories, even amidst transiently considered alternatives.</p><p> The second portion of my thesis work was the successful development of the first bimanual brain-machine interface. To reach this goal, I expanded the neural recordings system to enable bilateral, multi-site recordings from approximately 500 neurons simultaneously. In addition, I upgraded the experiment to feature a realistic virtual reality end effector, customized primate chair, and eye tracking system. Thirdly, I modified the tuning function of the unscented Kalman filter (UKF) to conjointly represent both arms in a single 4D model. As a result of widespread cortical plasticity in M1, S1, supplementary motor area (SMA), and posterior parietal cortex (PPC), the bimanual BMI enabled rhesus monkeys to simultaneously control two virtual limbs without any movement of their own body. I demonstrate the efficacy of the bimanual BMI in both a subject with prior task training using joysticks and a subject naïve to the task altogether, which simulates a common clinical scenario. The neural decoding algorithm was selected as a result of a methodical comparison between various neural decoders and decoder settings. I lastly introduce a two-stage switching model with a classify step and predict step which was designed and tested to generalize decoding strategies to include both unimanual and bimanual movements.</p>Dissertatio

Modeling and Analysis of Neural Spike Trains

open

Evaluation and Advancement of Electrocorticographic Brain-Machine Interfaces for Individuals with Upper-Limb Paralysis

Brain-machine interface (BMI) technology aims to provide individuals with movement paralysis a natural and intuitive means for the restoration of function. Electrocorticography (ECoG), in which disc electrodes are placed on either the surface of the dura or the cortex to record field potential activity, has been proposed as a viable neural recording modality for BMI systems, potentially providing stable, long-term recordings of cortical activity with high spatial and temporal resolution. Previous demonstrations of BMI control using ECoG have consisted of short-term periods of control by able-bodied subjects utilizing basic processing and decoding techniques. This dissertation presents work seeking to advance the current state of ECoG BMIs through an assessment of the ability of individuals with movement paralysis to control an ECoG BMI, an investigation into adaptation during BMI skill acquisition, an evaluation of chronic implantation of an ECoG electrode grid, and improved extraction of BMI command signals from ECoG recordings. Two individuals with upper-limb paralysis were implanted with high-density ECoG electrode grids over sensorimotor cortical areas for up to 30 days, with both subjects found to be capable of voluntarily modulating their cortical activity to control movement of a computer cursor with up to three degrees of freedom. Analysis of control signal angular error and the tuning characteristics of ECoG spectral features during the acquisition of brain control revealed that both decoder calibration and fixed-decoder training could facilitate performance improvements. In addition, to better understand the capability of ECoG to provide robust, long-term recordings, work was conducted assessing the effects of chronic implantation of an ECoG electrode grid in a non-human primate, demonstrating that movement-related modulation could be recorded from electrode nearly two years post-implantation despite the presence of substantial fibrotic encapsulation. Finally, it was found that the extraction of command signals from ECoG recordings could be improved through the use of a decoding method incorporating weight-space priors accounting for the expected correlation structure of electrical field potentials. Combined, this work both demonstrates the feasibility of ECoG-based BMI systems as well as addresses some of key challenges that must be overcome before such systems are translated to the clinical realm

Neurophysiological mechanisms of sensorimotor recovery from stroke

Ischemic stroke often results in the devastating loss of nervous tissue in the cerebral cortex, leading to profound motor deficits when motor territory is lost, and ultimately resulting in a substantial reduction in quality of life for the stroke survivor. The International Classification of Functioning, Disability and Health (ICF) was developed in 2002 by the World Health Organization (WHO) and provides a framework for clinically defining impairment after stroke. While the reduction of burdens due to neurological disease is stated as a mission objective of the National Institute of Neurological Disorders and Stroke (NINDS), recent clinical trials have been unsuccessful in translating preclinical research breakthroughs into actionable therapeutic treatment strategies with meaningful progress towards this goal. This means that research expanding another NINDS mission is now more important than ever: improving fundamental knowledge about the brain and nervous system in order to illuminate the way forward. Past work in the monkey model of ischemic stroke has suggested there may be a relationship between motor improvements after injury and the ability of the animal to reintegrate sensory and motor information during behavior. This relationship may be subserved by sprouting cortical axonal processes that originate in the spared premotor cortex after motor cortical injury in squirrel monkeys. The axons were observed to grow for relatively long distances (millimeters), significantly changing direction so that it appears that they specifically navigate around the injury site and reorient toward the spared sensory cortex. Critically, it remains unknown whether such processes ever form functional synapses, and if they do, whether such synapses perform meaningful calculations or other functions during behavior. The intent of this dissertation was to study this phenomenon in both intact rats and rats with a focal ischemia in primary motor cortex (M1) contralateral to the preferred forelimb during a pellet retrieval task. As this proved to be a challenging and resource-intensive endeavor, a primary objective of the dissertation became to provide the tools to facilitate such a project to begin with. This includes the creation of software, hardware, and novel training and behavioral paradigms for the rat model. At the same time, analysis of previous experimental data suggested that plasticity in the neural activity of the bilateral motor cortices of rats performing pellet retrievals after focal M1 ischemia may exhibit its most salient changes with respect to functional changes in behavior via mechanisms that were different than initially hypothesized. Specifically, a major finding of this dissertation is the finding that evidence of plasticity in the unit activity of bilateral motor cortical areas of the reaching rat is much stronger at the level of population features. These features exhibit changes in dynamics that suggest a shift in network fixed points, which may relate to the stability of filtering performed during behavior. It is therefore predicted that in order to define recovery by comparison to restitution, a specific type of fixed point dynamics must be present in the cortical population state. A final suggestion is that the stability or presence of these dynamics is related to the reintegration of sensory information to the cortex, which may relate to the positive impact of physical therapy during rehabilitation in the postacute window. Although many more rats will be needed to state any of these findings as a definitive fact, this line of inquiry appears to be productive for identifying targets related to sensorimotor integration which may enhance the efficacy of future therapeutic strategies