Are We the Robots? : Man-Machine Integration

We experience and interact with the world through our body. The founding father of computer science, Alan Turing, correctly realized that one of the most important features of the human being is the interaction between mind and body. Since the original demonstration that electrical activity of the cortical neurons can be employed to directly control a robotic device, the research on the so-called Brain-Machine Interfaces (BMIs) has impressively grown. For example, current BMIs dedicated to both experimental and clinical studies can translate raw neuronal signals into computational commands to reproduce reaching or grasping in artificial actuators. These developments hold promise for the restoration of limb mobility in paralyzed individuals. However, as the authors review in this chapter, before this goal can be achieved, several hurdles have to be overcome, including developments in real-time computational algorithms and in designing fully implantable and biocompatible devices. Future investigations will have to address the best solutions for restoring sensation to the prosthetic limb, which still remains a major challenge to full integration of the limb into the user's self-image

Workshops of the Sixth International Brain–Computer Interface Meeting: brain–computer interfaces past, present, and future

Brain–computer interfaces (BCI) (also referred to as brain–machine interfaces; BMI) are, by definition, an interface between the human brain and a technological application. Brain activity for interpretation by the BCI can be acquired with either invasive or non-invasive methods. The key point is that the signals that are interpreted come directly from the brain, bypassing sensorimotor output channels that may or may not have impaired function. This paper provides a concise glimpse of the breadth of BCI research and development topics covered by the workshops of the 6th International Brain–Computer Interface Meeting

De animais a máquinas : humanos tecnicamente melhores nos imaginários de futuro da convergência tecnológica

Dissertação (mestrado)—Universidade de Brasília, Instituto de Ciências Sociais, Departamento de Sociologia, 2020.O tema desta investigação é discutir os imaginários sociais de ciência e tecnologia que emergem a partir da área da neuroengenharia, em sua relação com a Convergência Tecnológica de quatro disciplinas: Nanotecnologia, Biotecnologia, tecnologias da Informação e tecnologias Cognitivas - neurociências- (CT-NBIC). Estas áreas desenvolvem-se e são articuladas por meio de discursos que ressaltam o aprimoramento das capacidades físicas e cognitivas dos seres humanos, com o intuito de construir uma sociedade melhor por meio do progresso científico e tecnológico, nos limites das agendas de pesquisa e desenvolvimento (P&D). Objetivos: Os objetivos nesse cenário, são discutir as implicações éticas, econômicas, políticas e sociais deste modelo de sistema sociotécnico. Nos referimos, tanto as aplicações tecnológicas, quanto as consequências das mesmas na formação dos imaginários sociais, que tipo de relações se estabelecem e como são criadas dentro desse contexto. Conclusão: Concluímos na busca por refletir criticamente sobre as propostas de aprimoramento humano mediado pela tecnologia, que surgem enquanto parte da agenda da Convergência Tecnológica NBIC. No entanto, as propostas de melhoramento humano vão muito além de uma agenda de investigação. Há todo um quadro de referências filosóficas e políticas que defendem o aprimoramento da espécie, vertentes estas que se aliam a movimentos trans-humanistas e pós- humanistas, posições que são ao mesmo tempo éticas, políticas e econômicas. A partir de nossa análise, entendemos que ciência, tecnologia e política estão articuladas, em coprodução, em relação às expectativas de futuros que são esperados ou desejados. Ainda assim, acreditamos que há um espaço de diálogo possível, a partir do qual buscamos abrir propostas para o debate público sobre questões de ciência e tecnologia relacionadas ao aprimoramento da espécie humana.Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)The subject of this research is to discuss the social imaginaries of science and technology that emerge from the area of neuroengineering in relation with the Technological Convergence of four disciplines: Nanotechnology, Biotechnology, Information technologies and Cognitive technologies -neurosciences- (CT-NBIC). These areas are developed and articulated through discourses that emphasize the enhancement of human physical and cognitive capacities, the intuition it is to build a better society, through the scientific and technological progress, at the limits of the research and development (R&D) agendas. Objectives: The objective in this scenery, is to discuss the ethic, economic, politic and social implications of this model of sociotechnical system. We refer about the technological applications and the consequences of them in the formation of social imaginaries as well as the kind of social relations that are created and established in this context. Conclusion: We conclude looking for critical reflections about the proposals of human enhancement mediated by the technology. That appear as a part of the NBIC technologies agenda. Even so, the proposals of human enhancement go beyond boundaries that an investigation agenda. There is a frame of philosophical and political references that defend the enhancement of the human beings. These currents that ally to the transhumanism and posthumanism movements, positions that are ethic, politic and economic at the same time. From our analysis, we understand that science, technology and politics are articulated, are in co-production, regarding the expected and desired futures. Even so, we believe that there is a space of possible dialog, from which we look to open proposals for the public discussion on questions of science and technology related to enhancement of human beings

Understanding and Decoding Imagined Speech using Electrocorticographic Recordings in Humans

Certain brain disorders, resulting from brainstem infarcts, traumatic brain injury, stroke and amyotrophic lateral sclerosis, limit verbal communication despite the patient being fully aware. People that cannot communicate due to neurological disorders would benefit from a system that can infer internal speech directly from brain signals. Investigating how the human cortex encodes imagined speech remains a difficult challenge, due to the lack of behavioral and observable measures. As a consequence, the fine temporal properties of speech cannot be synchronized precisely with brain signals during internal subjective experiences, like imagined speech. This thesis aims at understanding and decoding the neural correlates of imagined speech (also called internal speech or covert speech), for targeting speech neuroprostheses. In this exploratory work, various imagined speech features, such as acoustic sound features, phonetic representations, and individual words were investigated and decoded from electrocorticographic signals recorded in epileptic patients in three different studies. This recording technique provides high spatiotemporal resolution, via electrodes placed beneath the skull, but without penetrating the cortex In the first study, we reconstructed continuous spectrotemporal acoustic features from brain signals recorded during imagined speech using cross-condition linear regression. Using this technique, we showed that significant acoustic features of imagined speech could be reconstructed in seven patients. In the second study, we decoded continuous phoneme sequences from brain signals recorded during imagined speech using hidden Markov models. This technique allowed incorporating a language model that defined phoneme transitions probabilities. In this preliminary study, decoding accuracy was significant across eight phonemes in one patients. In the third study, we classified individual words from brain signals recorded during an imagined speech word repetition task, using support-vector machines. To account for temporal irregularities during speech production, we introduced a non-linear time alignment into the classification framework. Classification accuracy was significant across five patients. In order to compare speech representations across conditions and integrate imagined speech into the general speech network, we investigated imagined speech in parallel with overt speech production and/or speech perception. Results shared across the three studies showed partial overlapping between imagined speech and speech perception/production in speech areas, such as superior temporal lobe, anterior frontal gyrus and sensorimotor cortex. In an attempt to understanding higher-level cognitive processing of auditory processes, we also investigated the neural encoding of acoustic features during music imagery using linear regression. Despite this study was not directly related to speech representations, it provided a unique opportunity to quantitatively study features of inner subjective experiences, similar to speech imagery. These studies demonstrated the potential of using predictive models for basic decoding of speech features. Despite low performance, results show the feasibility for direct decoding of natural speech. In this respect, we highlighted numerous challenges that were encountered, and suggested new avenues to improve performances

Somatosensory Involvement in the Conceptual Representation of Objects

The involvement of the sensorimotor system in visual object processing is at the forefront of cognitive neuroscience research. Since the discovery of the mirror neuron system, a plethora of research has been dedicated to understanding how action influences cognition. Of particular interest to the current work is the way in which two-dimensional objects are represented in the human brain. Embodied cognition theories assert that the sensorimotor system plays a large (if not entire) role in the conceptual representation of objects. Interestingly, however, although somatosensation provides the first means of acquiring information from our environments and thus is integral to the development of conceptual representation, research has generally focused on motor system contributions to object processing. Therefore, this series of experiments will focus on unravelling the relationship between the somatosensory system and object processing. To do this, we employed two different priming paradigms, one in which vibratory stimulation served as a prime and an object picture as the target (Experiments 1 to 4), and the other where the object was the prime and the vibration the target (reverse priming task; Experiments 5 and 6). In Experiments 1 to 3, the participant was required to indicate how they would interact with the presented object (i.e., a semantic generation task). Results from Experiments 1 and 2 showed that object processing of graspable objects could be facilitated by a vibratory hand prime, compared to non-graspable objects (Experiment 1) and objects with foot related action affordances (Experiment 2), both of which showed no priming effects. Experiment 3 used a vibratory foot prime to investigate whether the priming effects in Experiments 1 and 2 were due semantic matching effects, such that drawing attention to a modality serves to enhance processing of objects related to that modality, and found no evidence to support this account. Experiment 4 assessed the degree to which sensorimotor representations are automatically activated using an object-naming paradigm, which showed no somatosensory priming effects, and thus no evidence for automatic somatosensory involvement. Experiment 5 utilized the reverse priming task (described above), and found evidence for faster somatosensory detection when primed with a hand object, providing converging evidence of a reciprocal relationship between the somatosensory system and object processing. Finally, Experiment 6 examined whether the results from Experiment 5 were due to matching effects (similar to Experiment 3), and found no evidence for this account. Taken together, our research provides corroborative, converging evidence that semantic knowledge about how one interacts with manipulable objects involves sensorimotor representations in the somatosensory system. This supports theories of embodied cognition and the mirror neuron system, and extends them from the motor domain to accommodate somatosensory influences, opening a new window into exploration of how touch may be incorporated into these theories. Implications for models of the mirror neuron system, and future directions for localizing these effects using neuroimaging are discussed

State-dependent modulation of cortico-spinal networks

Beta-band rhythm (13-30 Hz) is a dominant oscillatory activity in the sensorimotor system. Numerous studies reported on links between motor performance and the cortical and cortico-spinal beta rhythm. However, these studies report divergent beta-band frequencies and are, additionally, based on differently performed motor-tasks (e.g., motor imagination, muscle contraction, reach, grasp, and attention). This diversity blurs the role of beta in the sensorimotor system. It consequently challenges the development of beta-band activity-dependent stimulation protocols in the sensorimotor system. In this vein, we studied the functional role of beta-band cortico-cortical and cortico-spinal networks during a motor learning task. We studied how the contribution of cortical and spinal beta changes in the course of learning, and how this modulation is affected by afferent feedback to the sensorimotor system. We furthermore researched the relationship to motor performance. Consider that we made our study in the absence of any residual movement to allow our findings to be translated into rehabilitation programs for severely affected stroke patients. This thesis, at first, investigates evoked responses after transcranial magnetic stimulation (TMS). This revealed two different beta-band networks, i.e., in the low and high beta-band reflecting cortical and cortico-spinal activity. We, then, used a broader frequency range in the beta-band to trigger passive opening of the hand (peripheral feedback) or cortical stimulation (cortical feedback). While a unilateral hemispheric increase in cortico-spinal synchronization was observed in the group with peripheral feedback, a bilateral hemispheric increase in cortico-cortical and cortico-spinal synchronization was observed for the group with cortical feedback. An improvement in motor performance was found in the peripheral group only. Additionally, an enhancement in the directed cortico-spinal synchronization from cortex to periphery was observed for the peripheral group. Similar neurophysiological and behavioral changes were observed for stroke patients receiving peripheral feedback. The results 6 suggest two different mechanisms for beta-band activity-dependent protocols depending on the feedback modality. While the peripheral feedback appears to increase the synchronization among neural groups, cortical stimulation appears to recruit dormant neurons and to extend the involved motor network. These findings may provide insights regarding the mechanism behind novel activity-dependent protocols. It also highlights the importance of afferent feedback for motor restoration in beta-band activity-dependent rehabilitation programs

Integrating Cortical Sensorimotor Representations Across Spatial Scales and Task Contexts

Our understanding of how brains function is stratified between two very different scales: mesoscale (what function a given cortical area performs), measured with tools like fMRI; and microscale (what a given neuron does), measured with implanted microelectrodes. While extensive research has been done to characterize brain activity at both of these spatial scales, describing relationships between these two domains has proven difficult. Identifying ways to integrate findings between these scales is valuable for both research and clinical applications, but is particularly important for intracortical brain-computer interfaces (BCIs), which aim to restore motor function after paralysis or amputation. In humans, the brain is much larger than the available microelectrode arrays, so determining where to place the arrays is a critical aspect of ensuring optimal performance. BCIs preferentially target primary motor and somatosensory cortices, due to their direct relationship to motor control and critical role in skilled and dexterous movements. However, despite these areas displaying a relatively ordered spatial organization, it is difficult to accurately predict the behavior of neurons recorded from a given area for several reasons. Mesoscale activity is overlapping, with activity relating to multiple different movements observed in a single area. Additionally, neurons have flexible behavior, displaying different “tuning” to similar behavior under different contexts. Here I present my research integrating neuroimaging-based cortical mapping with directly-recorded neural activity in human sensorimotor cortex. First, I examine how the large-scale organization of sensorimotor representations measured with fMRI is affected by contextual sensory information. I then examine how spatially separate neural populations recorded with intracortical microelectrode arrays encode different types of movement. Finally, I examine whether how population encoding changes to reflect contextual sensory information using the same task as in the fMRI study. Together, these results provide a foundation for reconciling neural activity across spatial scales and task contexts, and will inform the design and placement of more capable BCI systems

How the brain controls hand actions: TMS, fMRI and behavioural studies

This thesis focused on testing the predictions made in Milner and Goodale’s model and reports finding from experiments investigating how inputs from both the dorsal and the ventral streams are required when we perform hand actions with objects (Chapter 2) and tools (Chapter 3 & 4) using different paradigms such as real and pantomimed grasping and techniques such as transcranial magnetic stimulation, motion-tracking of hand movements and cutting-edge fMRI multivoxel pattern analysis. The primary aim was to gain a new insight on the role of the dorsal and the ventral visual streams in real grasping and pantomiming and to understand what specific aspects of objects and movements associated with them are represented within the two streams. The first experiment (Chapter 2) examined the causal role of the anterior intraparietal and the lateral occipital in object’s real and pantomimed grasping using TMS. The results showed that real object grasping and pantomime actions without the objects in hand require the left dorsal stream but that information from the ventral stream is additionally required for pantomiming. The experiments in Chapter 3 and 4 investigated how tools and tool related actions are represented within the dorsal and the ventral stream (Chapter 3) and whether different action end-goals affected early grasping kinematics (Chapter 4). Using MVPA we showed that both dorsal and ventral stream regions represent information about functional and structural manipulation knowledge of tools. Moreover, we showed that both streams represent tool identity, which seems in line with our behavioural findings that tool identity affects grasping kinematics. The current work provided a detailed understanding of how the dorsal and the ventral streams interact in tool processing and propose a more sophisticated view of the distributed representations across the two streams. These findings open up a number of research avenues as well as help understanding how actions are disrupted in brain-damaged patients and advance the development of neural prosthetics