"Wink to grasp" – comparing eye, voice & EMG gesture control of grasp with soft-robotic gloves

The ability of robotic rehabilitation devices to support paralysed end-users is ultimately limited by the degree to which human-machine-interaction is designed to be effective and efficient in translating user intention into robotic action. Specifically, we evaluate the novel possibility of binocular eye-tracking technology to detect voluntary winks from involuntary blink commands, to establish winks as a novel low-latency control signal to trigger robotic action. By wearing binocular eye-tracking glasses we enable users to directly observe their environment or the actuator and trigger movement actions, without having to interact with a visual display unit or user interface. We compare our novel approach to two conventional approaches for controlling robotic devices based on electromyo-graphy (EMG) and speech-based human-computer interaction technology. We present an integrated software framework based on ROS that allows transparent integration of these multiple modalities with a robotic system. We use a soft-robotic SEM glove (Bioservo Technologies AB, Sweden) to evaluate how the 3 modalities support the performance and subjective experience of the end-user when movement assisted. All 3 modalities are evaluated in streaming, closed-loop control operation for grasping physical objects. We find that wink control shows the lowest error rate mean with lowest standard deviation of (0.23 ± 0.07, mean ± SEM) followed by speech control (0.35 ± 0. 13) and EMG gesture control (using the Myo armband by Thalamic Labs), with the highest mean and standard deviation (0.46 ± 0.16). We conclude that with our novel own developed eye-tracking based approach to control assistive technologies is a well suited alternative to conventional approaches, especially when combined with 3D eye-tracking based robotic end-point control

Proceedings of the 6th international conference on disability, virtual reality and associated technologies (ICDVRAT 2006)

The proceedings of the conferenc

Brain-Machine Interface for Reaching: Accounting for Target Size, Multiple Motor Plans, and Bimanual Coordination

<p>Brain-machine interfaces (BMIs) offer the potential to assist millions of people worldwide suffering from immobility due to loss of limbs, paralysis, and neurodegenerative diseases. BMIs function by decoding neural activity from intact cortical brain regions in order to control external devices in real-time. While there has been exciting progress in the field over the past 15 years, the vast majority of the work has focused on restoring of motor function of a single limb. In the work presented in this thesis, I first investigate the expanded role of primary sensory (S1) and motor (M1) cortex during reaching movements. By varying target size during reaching movements, I discovered the cortical correlates of the speed-accuracy tradeoff known as Fitts' law. Similarly, I analyzed cortical motor processing during tasks where the motor plan is quickly reprogrammed. In each study, I found that parameters relevant to the reach, such as target size or alternative movement plans, could be extracted by neural decoders in addition to simple kinematic parameters such as velocity and position. As such, future BMI functionality could expand to account for relevant sensory information and reliably decode intended reach trajectories, even amidst transiently considered alternatives.</p><p> The second portion of my thesis work was the successful development of the first bimanual brain-machine interface. To reach this goal, I expanded the neural recordings system to enable bilateral, multi-site recordings from approximately 500 neurons simultaneously. In addition, I upgraded the experiment to feature a realistic virtual reality end effector, customized primate chair, and eye tracking system. Thirdly, I modified the tuning function of the unscented Kalman filter (UKF) to conjointly represent both arms in a single 4D model. As a result of widespread cortical plasticity in M1, S1, supplementary motor area (SMA), and posterior parietal cortex (PPC), the bimanual BMI enabled rhesus monkeys to simultaneously control two virtual limbs without any movement of their own body. I demonstrate the efficacy of the bimanual BMI in both a subject with prior task training using joysticks and a subject naïve to the task altogether, which simulates a common clinical scenario. The neural decoding algorithm was selected as a result of a methodical comparison between various neural decoders and decoder settings. I lastly introduce a two-stage switching model with a classify step and predict step which was designed and tested to generalize decoding strategies to include both unimanual and bimanual movements.</p>Dissertatio

Proceedings of the 4th international conference on disability, virtual reality and associated technologies (ICDVRAT 2002)

The proceedings of the conferenc

Neural Prosthetic Advancement: identification of circuitry in the Posterior Parietal Cortex

There are limited options for rehabilitation following an established Spinal Cord Injury (SCI) resulting in paralysis. For most of the individuals affected, SCI means a lifetime of confinement to a wheelchair and overall reduced independence. Brain-Computer and Brain-Machine Interface (BCI and BMI) techniques may be of aid when used for assistive purposes. However, these techniques are still far from being implemented in daily rehabilitative practice. Existing literature on the use of BCI and BMI techniques in SCI is limited and focuses on the extraction of motor control signals from the primary motor cortex (M1). However, evidence suggests that in long-term established SCI the functional activation of motor and premotor areas tends to decrease over time. In the present project, we explore the possibility of successful implementation of assistive BCI and BMI systems using posterior parietal areas as extraction sites of motor control activity. Firstly, we will investigate the representation of space in the posterior parietal cortex (PPC) and whether evidence of body-centered reference frames can be found in healthy individuals. We will then proceed to extract information regarding the residual level of motor imagery activity in individuals suffering from long-term and high-level SCI. Our aim is to ascertain whether functional activation of motor and posterior areas is comparable to that of matched controls. Finally, we will present work that was done in collaboration with the Netherlands Organisation for Applied Scientific Research that can offer an example of successful application of a BCI technique for rehabilitation purposes

Proceedings of the 3rd international conference on disability, virtual reality and associated technologies (ICDVRAT 2000)

The proceedings of the conferenc

Augmenting user capabilities through an adaptive assistive manipulator

Mención Internacional en el título de doctorAssistive robot manipulators have the potential to increase the independence of disabled persons in activities of daily living. The current designs are mainly limited to pure teleoperation by the user, given the need for keeping the user in the control loop, and the complexity of the tasks and environments in which they operate. This thesis aims to augment the user’s capabilities for performing such tasks by adapting the robot, and its level of assistance, to the user. Methodologies for modeling and benchmarking the complete human-robot system were established, which helped drive the development of different approaches to adaptation. This included a task-oriented optimization of the robot physical structure, approaches for low-level adaptive shared control, and work on interactive learning of, and assistance on completing, simple object manipulation tasks. Three experimental platforms were used: The ASIBOT manipulator of Universidad Carlos III de Madrid (UC3M), the AMOR manipulator of Exact Dynamics, and the iCub humanoid robot.Los manipuladores asistenciales tienen el potencial de incrementar la independencia de personas discapacitadas en sus actividades de la vida diaria. Los diseños actuales se limitan principalmente a una pura teleoperación, pues dada la complejidad de las tareas y del entorno, se necesita mantener al usuario en el lazo de control. Esta tesis pretende mejorar las capacidades del usuario para realizar estas tareas, adaptando el robot y su nivel de asistencia a las necesidades del usuario. Se han establecido metodologías para el modelado y evaluación del comportamiento del sistema formado por humano y robot, lo que ha permitido el desarrollo de diferentes aproximaciones a la adaptación. Esto incluye desde la optimización de la estructura del robot atendiendo a las tareas, la evaluación de diversas aproximaciones al control compartido adaptativo a bajo nivel, al aprendizaje interactivo y el desarrollo de asistencias para completar tareas sencillas de manipulación. Se ha hecho uso de tres plataformas experimentales: el manipulador ASIBOT de la Universidad Carlos III de Madrid (UC3M), el manipulador AMOR de Exact Dynamics y el humanoide iCub.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Alberto Sanfeliú.- Secretario: Concepción Alicia Monje Micharet.- Vocal: Yiannis Demiri

Augmentation of Brain Function: Facts, Fiction and Controversy. Volume III: From Clinical Applications to Ethical Issues and Futuristic Ideas

The final volume in this tripartite series on Brain Augmentation is entitled “From Clinical Applications to Ethical Issues and Futuristic Ideas”. Many of the articles within this volume deal with translational efforts taking the results of experiments on laboratory animals and applying them to humans. In many cases, these interventions are intended to help people with disabilities in such a way so as to either restore or extend brain function. Traditionally, therapies in brain augmentation have included electrical and pharmacological techniques. In contrast, some of the techniques discussed in this volume add specificity by targeting select neural populations. This approach opens the door to where and how to promote the best interventions. Along the way, results have empowered the medical profession by expanding their understanding of brain function. Articles in this volume relate novel clinical solutions for a host of neurological and psychiatric conditions such as stroke, Parkinson’s disease, Huntington’s disease, epilepsy, dementia, Alzheimer’s disease, autism spectrum disorders (ASD), traumatic brain injury, and disorders of consciousness. In disease, symptoms and signs denote a departure from normal function. Brain augmentation has now been used to target both the core symptoms that provide specificity in the diagnosis of a disease, as well as other constitutional symptoms that may greatly handicap the individual. The volume provides a report on the use of repetitive transcranial magnetic stimulation (rTMS) in ASD with reported improvements of core deficits (i.e., executive functions). TMS in this regard departs from the present-day trend towards symptomatic treatment that leaves unaltered the root cause of the condition. In diseases, such as schizophrenia, brain augmentation approaches hold promise to avoid lengthy pharmacological interventions that are usually riddled with side effects or those with limiting returns as in the case of Parkinson’s disease. Brain stimulation can also be used to treat auditory verbal hallucination, visuospatial (hemispatial) neglect, and pain in patients suffering from multiple sclerosis. The brain acts as a telecommunication transceiver wherein different bandwidth of frequencies (brainwave oscillations) transmit information. Their baseline levels correlate with certain behavioral states. The proper integration of brain oscillations provides for the phenomenon of binding and central coherence. Brain augmentation may foster the normalization of brain oscillations in nervous system disorders. These techniques hold the promise of being applied remotely (under the supervision of medical personnel), thus overcoming the obstacle of travel in order to obtain healthcare. At present, traditional thinking would argue the possibility of synergism among different modalities of brain augmentation as a way of increasing their overall effectiveness and improving therapeutic selectivity. Thinking outside of the box would also provide for the implementation of brain-to-brain interfaces where techniques, proper to artificial intelligence, could allow us to surpass the limits of natural selection or enable communications between several individual brains sharing memories, or even a global brain capable of self-organization. Not all brains are created equal. Brain stimulation studies suggest large individual variability in response that may affect overall recovery/treatment, or modify desired effects of a given intervention. The subject’s age, gender, hormonal levels may affect an individual’s cortical excitability. In addition, this volume discusses the role of social interactions in the operations of augmenting technologies. Finally, augmenting methods could be applied to modulate consciousness, even though its neural mechanisms are poorly understood. Finally, this volume should be taken as a debate on social, moral and ethical issues on neurotechnologies. Brain enhancement may transform the individual into someone or something else. These techniques bypass the usual routes of accommodation to environmental exigencies that exalted our personal fortitude: learning, exercising, and diet. This will allow humans to preselect desired characteristics and realize consequent rewards without having to overcome adversity through more laborious means. The concern is that humans may be playing God, and the possibility of an expanding gap in social equity where brain enhancements may be selectively available to the wealthier individuals. These issues are discussed by a number of articles in this volume. Also discussed are the relationship between the diminishment and enhancement following the application of brain-augmenting technologies, the problem of “mind control” with BMI technologies, free will the duty to use cognitive enhancers in high-responsibility professions, determining the population of people in need of brain enhancement, informed public policy, cognitive biases, and the hype caused by the development of brain- augmenting approaches

Brain Waves, A Cultural History: Oscillations of Neuroscience, Technology, Telepathy, and Transcendence

This project proceeds from a narrow question: What, if anything, is a brain wave? Beguiling in its simplicity, this question prompts a cultural-historical investigation that spans over 150 years of science, technology, and society. Proposed in 1869, the original theory of brain waves cites etheric undulations to explain reports of apparent thought transference. Though most modern thinkers no longer believe in outright telepathy, I argue that dreams of thought transmission and other mental miracles subtly persist—not in obscure and occult circles, but at the forefront of technoscience. A hybrid of science and fiction, brain waves represent an ideal subject through which to explore the ways in which technical language shrouds spiritual dreams. Today, the phrase “brain waves” often function as shorthand for electrical changes in the brain, particularly in the context of technologies that purport to “read” some aspect of mental function, or to transmit neural data to a digital device. While such technologies appear uniquely modern, the history of brain waves reveals that they are merely the millennial incarnation of a much older hope—a hope for transmission and transcendence via the brain’s emanations

Proceedings of the 9th international conference on disability, virtual reality and associated technologies (ICDVRAT 2012)

The proceedings of the conferenc