Robot Learning and Control Using Error-Related Cognitive Brain Signals

Durante los últimos años, el campo de los interfaces cerebro-máquina (BMIs en inglés) ha demostrado cómo humanos y animales son capaces de controlar dispositivos neuroprotésicos directamente de la modulación voluntaria de sus señales cerebrales, tanto en aproximaciones invasivas como no invasivas. Todos estos BMIs comparten un paradigma común, donde el usuario trasmite información relacionada con el control de la neuroprótesis. Esta información se recoge de la actividad cerebral del usuario, para luego ser traducida en comandos de control para el dispositivo. Cuando el dispositivo recibe y ejecuta la orden, el usuario recibe una retroalimentación del rendimiento del sistema, cerrando de esta manera el bucle entre usuario y dispositivo. La mayoría de los BMIs decodifican parámetros de control de áreas corticales para generar la secuencia de movimientos para la neuroprótesis. Esta aproximación simula al control motor típico, dado que enlaza la actividad neural con el comportamiento o la ejecución motora. La ejecución motora, sin embargo, es el resultado de la actividad combinada del córtex cerebral, áreas subcorticales y la médula espinal. De hecho, numerosos movimientos complejos, desde la manipulación a andar, se tratan principalmente al nivel de la médula espinal, mientras que las áreas corticales simplemente proveen el punto del espacio a alcanzar y el momento de inicio del movimiento. Esta tesis propone un paradigma BMI alternativo que trata de emular el rol de los niveles subcorticales durante el control motor. El paradigma se basa en señales cerebrales que transportan información cognitiva asociada con procesos de toma de decisiones en movimientos orientados a un objetivo, y cuya implementación de bajo nivel se maneja en niveles subcorticales. A lo largo de la tesis, se presenta el primer paso hacia el desarrollo de este paradigma centrándose en una señal cognitiva específica relacionada con el procesamiento de errores humano: los potenciales de error (ErrPs) medibles mediante electroencefalograma (EEG). En esta propuesta de paradigma, la neuroprótesis ejecuta activamente una tarea de alcance mientras el usuario simplemente monitoriza el rendimiento del dispositivo mediante la evaluación de la calidad de las acciones ejecutadas por el dispositivo. Estas evaluaciones se traducen (gracias a los ErrPs) en retroalimentación para el dispositivo, el cual las usa en un contexto de aprendizaje por refuerzo para mejorar su comportamiento. Esta tesis demuestra por primera vez este paradigma BMI de enseñanza con doce sujetos en tres experimentos en bucle cerrado concluyendo con la operación de un manipulador robótico real. Como la mayoría de BMIs, el paradigma propuesto requiere una etapa de calibración específica para cada sujeto y tarea. Esta fase, un proceso que requiere mucho tiempo y extenuante para el usuario, dificulta la distribución de los BMIs a aplicaciones fuera del laboratorio. En el caso particular del paradigma propuesto, una fase de calibración para cada tarea es altamente impráctico ya que el tiempo necesario para esta fase se suma al tiempo de aprendizaje de la tarea, retrasando sustancialmente el control final del dispositivo. Así, sería conveniente poder entrenar clasificadores capaces de funcionar independientemente de la tarea de aprendizaje que se esté ejecutando. Esta tesis analiza desde un punto de vista electrofisiológico cómo los potenciales se ven afectados por diferentes tareas ejecutadas por el dispositivo, mostrando cambios principalmente en la latencia la señal; y estudia cómo transferir el clasificador entre tareas de dos maneras: primero, aplicando clasificadores adaptativos del estado del arte, y segundo corrigiendo la latencia entre las señales de dos tareas para poder generalizar entre ambas. Otro reto importante bajo este paradigma viene del tiempo necesario para aprender la tarea. Debido al bajo ratio de información transferida por minuto del BMI, el sistema tiene una pobre escalabilidad: el tiempo de aprendizaje crece exponencialmente con el tamaño del espacio de aprendizaje, y por tanto resulta impráctico obtener el comportamiento motor óptimo mediante aprendizaje por refuerzo. Sin embargo, este problema puede resolverse explotando la estructura de la tarea de aprendizaje. Por ejemplo, si el número de posiciones a alcanzar es discreto se puede pre-calcular la política óptima para cada posible posición. En esta tesis, se muestra cómo se puede usar la estructura de la tarea dentro del paradigma propuesto para reducir enormemente el tiempo de aprendizaje de la tarea (de diez minutos a apenas medio minuto), mejorando enormemente así la escalabilidad del sistema. Finalmente, esta tesis muestra cómo, gracias a las lecciones aprendidas en los descubrimientos anteriores, es posible eliminar completamente la etapa de calibración del paradigma propuesto mediante el aprendizaje no supervisado del clasificador al mismo tiempo que se está ejecutando la tarea. La idea fundamental es calcular un conjunto de clasificadores que sigan las restricciones de la tarea anteriormente usadas, para a continuación seleccionar el mejor clasificador del conjunto. De esta manera, esta tesis presenta un BMI plug-and-play que sigue el paradigma propuesto, aprende la tarea y el clasificador y finalmente alcanza la posición del espacio deseada por el usuario

Human aware robot navigation

Abstract. Human aware robot navigation refers to the navigation of a robot in an environment shared with humans in such a way that the humans should feel comfortable, and natural with the presence of the robot. On top of that, the robot navigation should comply with the social norms of the environment. The robot can interact with humans in the environment, such as avoiding them, approaching them, or following them. In this thesis, we specifically focus on the approach behavior of the robot, keeping the other use cases still in mind. Studying and analyzing how humans move around other humans gives us the idea about the kind of navigation behaviors that we expect the robots to exhibit. Most of the previous research does not focus much on understanding such behavioral aspects while approaching people. On top of that, a straightforward mathematical modeling of complex human behaviors is very difficult. So, in this thesis, we proposed an Inverse Reinforcement Learning (IRL) framework based on Guided Cost Learning (GCL) to learn these behaviors from demonstration. After analyzing the CongreG8 dataset, we found that the incoming human tends to make an O-space (circle) with the rest of the group. Also, the approaching velocity slows down when the approaching human gets closer to the group. We utilized these findings in our framework that can learn the optimal reward and policy from the example demonstrations and imitate similar human motion

Advances in Robot Navigation

Robot navigation includes different interrelated activities such as perception - obtaining and interpreting sensory information; exploration - the strategy that guides the robot to select the next direction to go; mapping - the construction of a spatial representation by using the sensory information perceived; localization - the strategy to estimate the robot position within the spatial map; path planning - the strategy to find a path towards a goal location being optimal or not; and path execution, where motor actions are determined and adapted to environmental changes. This book integrates results from the research work of authors all over the world, addressing the abovementioned activities and analyzing the critical implications of dealing with dynamic environments. Different solutions providing adaptive navigation are taken from nature inspiration, and diverse applications are described in the context of an important field of study: social robotics

Rehabilitation of gait after stroke: a review towards a top-down approach

This document provides a review of the techniques and therapies used in gait rehabilitation after stroke. It also examines the possible benefits of including assistive robotic devices and brain-computer interfaces in this field, according to a top-down approach, in which rehabilitation is driven by neural plasticity

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 376)

This bibliography lists 265 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during Jun. 1993. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance

Variable autonomy assignment algorithms for human-robot interactions.

As robotic agents become increasingly present in human environments, task completion rates during human-robot interaction has grown into an increasingly important topic of research. Safe collaborative robots executing tasks under human supervision often augment their perception and planning capabilities through traded or shared control schemes. However, such systems are often proscribed only at the most abstract level, with the meticulous details of implementation left to the designer\u27s prerogative. Without a rigorous structure for implementing controls, the work of design is frequently left to ad hoc mechanism with only bespoke guarantees of systematic efficacy, if any such proof is forthcoming at all. Herein, I present two quantitatively defined models for implementing sliding-scale variable autonomy, in which levels of autonomy are determined by the relative efficacy of autonomous subroutines. I experimentally test the resulting Variable Autonomy Planning (VAP) algorithm and against a traditional traded control scheme in a pick-and-place task, and apply the Variable Autonomy Tasking algorithm to the implementation of a robot performing a complex sanitation task in real-world environs. Results show that prioritizing autonomy levels with higher success rates, as encoded into VAP, allows users to effectively and intuitively select optimal autonomy levels for efficient task completion. Further, the Pareto optimal design structure of the VAP+ algorithm allows for significant performance improvements to be made through intervention planning based on systematic input determining failure probabilities through sensorized measurements. This thesis describes the design, analysis, and implementation of these two algorithms, with a particular focus on the VAP+ algorithm. The core conceit is that they are methods for rigorously defining locally optimal plans for traded control being shared between a human and one or more autonomous processes. It is derived from an earlier algorithmic model, the VAP algorithm, developed to address the issue of rigorous, repeatable assignment of autonomy levels based on system data which provides guarantees on basis of the failure-rate sorting of paired autonomous and manual subtask achievement systems. Using only probability ranking to define levels of autonomy, the VAP algorithm is able to sort modules into optimizable ordered sets, but is limited to only solving sequential task assignments. By constructing a joint cost metric for the entire plan, and by implementing a back-to-front calculation scheme for this metric, it is possible for the VAP+ algorithm to generate optimal planning solutions which minimize the expected cost, as amortized over time, funds, accuracy, or any metric combination thereof. The algorithm is additionally very efficient, and able to perform on-line assessments of environmental changes to the conditional probabilities associated with plan choices, should a suitable model for determining these probabilities be present. This system, as a paired set of two algorithms and a design augmentation, form the VAP+ algorithm in full

Medical Robotics

The first generation of surgical robots are already being installed in a number of operating rooms around the world. Robotics is being introduced to medicine because it allows for unprecedented control and precision of surgical instruments in minimally invasive procedures. So far, robots have been used to position an endoscope, perform gallbladder surgery and correct gastroesophogeal reflux and heartburn. The ultimate goal of the robotic surgery field is to design a robot that can be used to perform closed-chest, beating-heart surgery. The use of robotics in surgery will expand over the next decades without any doubt. Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. The aim of this book is to provide an overview of the state-of-art, to present new ideas, original results and practical experiences in this expanding area. Nevertheless, many chapters in the book concern advanced research on this growing area. The book provides critical analysis of clinical trials, assessment of the benefits and risks of the application of these technologies. This book is certainly a small sample of the research activity on Medical Robotics going on around the globe as you read it, but it surely covers a good deal of what has been done in the field recently, and as such it works as a valuable source for researchers interested in the involved subjects, whether they are currently “medical roboticists” or not

Towards Closed-loop, Robot Assisted Percutaneous Interventions under MRI Guidance

Image guided therapy procedures under MRI guidance has been a focused research area over past decade. Also, over the last decade, various MRI guided robotic devices have been developed and used clinically for percutaneous interventions, such as prostate biopsy, brachytherapy, and tissue ablation. Though MRI provides better soft tissue contrast compared to Computed Tomography and Ultrasound, it poses various challenges like constrained space, less ergonomic patient access and limited material choices due to its high magnetic field. Even after, advancements in MRI compatible actuation methods and robotic devices using them, most MRI guided interventions are still open-loop in nature and relies on preoperative or intraoperative images. In this thesis, an intraoperative MRI guided robotic system for prostate biopsy comprising of an MRI compatible 4-DOF robotic manipulator, robot controller and control application with Clinical User Interface (CUI) and surgical planning applications (3DSlicer and RadVision) is presented. This system utilizes intraoperative images acquired after each full or partial needle insertion for needle tip localization. Presented system was approved by Institutional Review Board at Brigham and Women\u27s Hospital(BWH) and has been used in 30 patient trials. Successful translation of such a system utilizing intraoperative MR images motivated towards the development of a system architecture for close-loop, real-time MRI guided percutaneous interventions. Robot assisted, close-loop intervention could help in accurate positioning and localization of the therapy delivery instrument, improve physician and patient comfort and allow real-time therapy monitoring. Also, utilizing real-time MR images could allow correction of surgical instrument trajectory and controlled therapy delivery. Two of the applications validating the presented architecture; closed-loop needle steering and MRI guided brain tumor ablation are demonstrated under real-time MRI guidance

Progress and Prospects of the Human-Robot Collaboration

International audienceRecent technological advances in hardware designof the robotic platforms enabled the implementationof various control modalities for improved interactions withhumans and unstructured environments. An important applicationarea for the integration of robots with such advancedinteraction capabilities is human-robot collaboration. Thisaspect represents high socio-economic impacts and maintainsthe sense of purpose of the involved people, as the robotsdo not completely replace the humans from the workprocess. The research community’s recent surge of interestin this area has been devoted to the implementation of variousmethodologies to achieve intuitive and seamless humanrobot-environment interactions by incorporating the collaborativepartners’ superior capabilities, e.g. human’s cognitiveand robot’s physical power generation capacity. In fact,the main purpose of this paper is to review the state-of-thearton intermediate human-robot interfaces (bi-directional),robot control modalities, system stability, benchmarking andrelevant use cases, and to extend views on the required futuredevelopments in the realm of human-robot collaboration

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues