Intuitive Human-Machine Interfaces for Non-Anthropomorphic Robotic Hands

As robots become more prevalent in our everyday lives, both in our workplaces and in our homes, it becomes increasingly likely that people who are not experts in robotics will be asked to interface with robotic devices. It is therefore important to develop robotic controls that are intuitive and easy for novices to use. Robotic hands, in particular, are very useful, but their high dimensionality makes creating intuitive human-machine interfaces for them complex. In this dissertation, we study the control of non-anthropomorphic robotic hands by non-roboticists in two contexts: collaborative manipulation and assistive robotics. In the field of collaborative manipulation, the human and the robot work side by side as independent agents. Teleoperation allows the human to assist the robot when autonomous grasping is not able to deal sufficiently well with corner cases or cannot operate fast enough. Using the teleoperator’s hand as an input device can provide an intuitive control method, but finding a mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands’ dissimilar kinematics. In this dissertation, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to enable effective real-time teleoperation, even for novice users. We propose a low-dimensional and continuous teleoperation subspace which can be used as an intermediary for mapping between different hand pose spaces. We first propose the general concept of the subspace, its properties and the variables needed to map from the human hand to a robot hand. We then propose three ways to populate the teleoperation subspace mapping. Two of our mappings use a dataglove to harvest information about the user's hand. We define the mapping between joint space and teleoperation subspace with an empirical definition, which requires a person to define hand motions in an intuitive, hand-specific way, and with an algorithmic definition, which is kinematically independent, and uses objects to define the subspace. Our third mapping for the teleoperation subspace uses forearm electromyography (EMG) as a control input. Assistive orthotics is another area of robotics where human-machine interfaces are critical, since, in this field, the robot is attached to the hand of the human user. In this case, the goal is for the robot to assist the human with movements they would not otherwise be able to achieve. Orthotics can improve the quality of life of people who do not have full use of their hands. Human-machine interfaces for assistive hand orthotics that use EMG signals from the affected forearm as input are intuitive and repeated use can strengthen the muscles of the user's affected arm. In this dissertation, we seek to create an EMG based control for an orthotic device used by people who have had a stroke. We would like our control to enable functional motions when used in conjunction with a orthosis and to be robust to changes in the input signal. We propose a control for a wearable hand orthosis which uses an easy to don, commodity forearm EMG band. We develop an supervised algorithm to detect a user’s intent to open and close their hand, and pair this algorithm with a training protocol which makes our intent detection robust to changes in the input signal. We show that this algorithm, when used in conjunction with an orthosis over several weeks, can improve distal function in users. Additionally, we propose two semi-supervised intent detection algorithms designed to keep our control robust to changes in the input data while reducing the length and frequency of our training protocol

Down-Conditioning of Soleus Reflex Activity using Mechanical Stimuli and EMG Biofeedback

Spasticity is a common syndrome caused by various brain and neural injuries, which can severely impair walking ability and functional independence. To improve functional independence, conditioning protocols are available aimed at reducing spasticity by facilitating spinal neuroplasticity. This down-conditioning can be performed using different types of stimuli, electrical or mechanical, and reflex activity measures, EMG or impedance, used as biofeedback variable. Still, current results on effectiveness of these conditioning protocols are incomplete, making comparisons difficult. We aimed to show the within-session task- dependent and across-session long-term adaptation of a conditioning protocol based on mechanical stimuli and EMG biofeedback. However, in contrast to literature, preliminary results show that subjects were unable to successfully obtain task-dependent modulation of their soleus short-latency stretch reflex magnitude

Multi-user Touch Surfaces to Promote Social Participation and Self-efficacy in Upper-limb Stroke Rehabilitation

This thesis addresses the growing incidence of stroke and its impact on long-term disability, emphasizing the importance of post-stroke rehabilitation. Predicting a 34% increase in stroke cases by 2025, this work focuses on factors affecting post-stroke motor rehabilitation, such as depressive symptoms, while recognizing the role of social participation and self-efficacy in modulating these outcomes. To tackle these challenges, we propose the use of a multi-user interactive table for upper limb rehabilitation. This innovative approach involves detecting objects of various sizes and shapes as interfaces for serious games, fostering collective therapeutic activities targeting motor rehabilitation, depressive symptoms, social participation, and self-efficacy. Four exploratory studies informed the development of the interactive table. The studies determined game mechanics, object types, and game features. Three game modes (competitive, co-active, and collaborative) were explored, with the collaborative mode showing higher social engagement and more positive outcomes. Subsequent studies involving stroke survivors confirmed the effectiveness of collaborative gameplay. Feasibility, engagement, and usability were tested using objects of different sizes and shapes, revealing insights into task performance, grasping, and task complexity adjustments. The developed interactive table accommodates up to four participants, using top down object shape tracking and a multi-touch panel. Four serious games were designed to target specific upper limb skills, played in collaborative mode with incorporated features to enhance the overall experience. A pilot study with 12 stroke survivors over four weeks demonstrated significant improvements in motor outcomes, including range of motion, dexterity, strength, and coordination. The participants reported high enjoyment and interest in the system, with good usability scores. The study suggests that a group-based holistic motor rehabilitation approach, as presented, holds potential for enhancing motor outcomes by promoting social interaction and self-efficacy. These results indicate the viability of the proposed system as a promising solution for stroke rehabilitation.Esta tese aborda a crescente incidência do AVC e o seu impacto na funcionalidade a longo prazo, realçando a importância da reabilitação pós-AVC. Prevendo-se um aumento de 34% nos casos de AVC até 2025, este trabalho foca-se em fatores que afetam a reabilitação motora pós-AVC, como os sintomas depressivos, reconhecendo o papel da participação social e do sentimento de auto-eficácia na nestes sintomas. Para enfrentar esses desafios, propomos o uso de uma mesa interativa multi utilizador para reabilitação dos membros superiores. Esta abordagem inovadora envolve a deteção de objetos de vários tamanhos e formas como interfaces para os jogos sérios, permitindo atividades terapêuticas em grupo direcionadas à reabilitação motora, sintomas depressivos, participação social e autoeficácia. Quatro estudos exploratórios suportaram o desenvolvimento da mesa interativa, determinando mecânicas de jogo, tipos de objetos e recursos do jogo. Três modos de jogo (competitivo, co-ativo e colaborativo) foram investigados, com o modo colaborativo mostrando maior envolvimento social e resultados mais positivos. Estudos subsequentes envolvendo sobreviventes de AVC confirmaram a eficácia do jogo colaborativo. Viabilidade, envolvimento e usabilidade foram testados usando objetos de diferentes tamanhos e formas, revelando insights importantes relativos ao desempenho nas tarefas, preensões e ajustes de complexidade da tarefa. A mesa interativa desenvolvida acomoda até quatro participantes, e permite a deteção de objectos para serem usados como interfaces e um painel multi-toque. Quatro jogos sérios foram projetados para reabilitar competências específicas dos membros superiores, desenhados para serem jogados em modo colaborativo e com recursos incorporados para potenciar o processo de reabilitação e a experiência do utilizador. Um estudo piloto com 12 sobreviventes de AVC ao longo de quatro semanas, demonstrou melhorias significativas nos resultados motores, nomeadamente amplitude de movimento, destreza, força e coordenação. Os participantes reportaram níveis altos de prazer e interesse no sistema, e pontuações de usabilidade promissoras. Este trabalho sugere que uma abordagem de reabilitação motora holística e em grupo, como apresentada, tem potencial para melhorar os resultados motores promovendo a interação social e o sentimento de auto-eficácia. Estes resultados sugerem a viabilidade do sistema proposto como uma solução promissora para a reabilitação motora no AVC

Bioinspired robotic rehabilitation tool for lower limb motor learning after stroke

Mención Internacional en el título de doctorEsta tesis doctoral presenta, tras repasar la marcha humana, las principales patologíıas y condiciones que la afectan, y los distintos enfoques de rehabilitación con la correspondiente implicación neurofisiológica, el camino de investigación que desemboca en la herramienta robótica de rehabilitación y las terapias que se han desarrollado en el marco de los proyectos europeos BioMot: Smart Wearable Robots with Bioinspired Sensory-Motor Skills y HANK: European advanced exoskeleton for rehabilitation of Acquired Brain Damage (ABD) and/or spinal cord injury’s patients, y probado bajo el paraguas del proyecto europeo ASTONISH: Advancing Smart Optical Imaging and Sensing for Health y el proyecto nacional ASSOCIATE: A comprehensive and wearable robotics based approach to the rehabilitation and assistance to people with stroke and spinal cord injury.This doctoral thesis presents, after reviewing human gait, the main pathologies and conditions that affect it, and the different rehabilitation approaches with the corresponding neurophysiological implications, the research journey that leads to the development of the rehabilitation robotic tool, and the therapies that have been designed, within the framework of the European projects BioMot: Smart Wearable Robots with Bioinspired Sensory-Motor Skills and HANK: European advanced exoskeleton for rehabilitation of Acquired Brain Damage (ABD) and/or spinal cord injury’s patients and tested under the umbrella of the European project ASTONISH: Advancing Smart Optical Imaging and Sensing for Health and the national project ASSOCIATE: A comprehensive and wearable robotics based approach to the rehabilitation and assistance to people with stroke and spinal cord injury.This work has been carried out at the Neural Rehabilitation Group (NRG), Cajal Institute, Spanish National Research Council (CSIC). The research presented in this thesis has been funded by the Commission of the European Union under the BioMot project - Smart Wearable Robots with Bioinspired Sensory-Motor Skills (Grant Agreement number IFP7-ICT - 611695); under HANK Project - European advanced exoskeleton for rehabilitation of Acquired Brain Damage (ABD) and/or spinal cord injury’s patients (Grant Agreements number H2020-EU.2. - PRIORITY ’Industrial leadership’ and H2020-EU.3. - PRIORITY ’Societal challenges’ - 699796); also under the ASTONISH Project - Advancing Smart Optical Imaging and Sensing for Health (Grant Agreement number H2020-EU.2.1.1.7. - ECSEL - 692470); with financial support of Spanish Ministry of Economy and Competitiveness (MINECO) under the ASSOCIATE project - A comprehensive and wearable robotics based approach to the rehabilitation and assistance to people with stroke and spinal cord injury (Grant Agreement number 799158449-58449-45-514); and with grant RYC-2014-16613, also by Spanish Ministry of Economy and Competitiveness.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Fernando Javier Brunetti Fernández.- Secretario: Dorin Sabin Copaci.- Vocal: Antonio Olivier

Robotics in health care: Perspectives of robot-aided interventions in clinical practice for rehabilitation of upper limbs

This article belongs to the Special Issue Rehabilitation Robotics: Recent Advancements and New Perspectives about Training and Assessment of Sensorimotor Functions.Robot-aided systems to support the physical rehabilitation of individuals with neurological impairment is one of the fields that has been widely developed in the last few decades. However, the adoption of these systems in clinical practice remains limited. In order to better understanding the causes of this limitation, a systematic review of robot-based systems focused on upper extremity rehabilitation is presented in this paper. A systematic search and review of related articles in the literature were conducted. The chosen works were analyzed according to the type of device, the data analysis capability, the therapy method, the human–robot interaction, the safety strategies, and the focus of treatment. As a conclusion, self-adaptation for personalizing the treatments, safeguarding and enhancing of patient–robot interaction towards training essential factors of movement generation into the same paradigm, or the use of lifelike environments in fully-immersive virtual reality for increasing the assimilation of motor gains could be relevant factors to develop more accepted robot-aided systems in clinical practice.This work was supported in part by the Spanish Ministry of Economy and Competitiveness via the ROBOESPASproject (DPI2017-87562-C2-1-R) and in part by the RoboCity2030-DIH-CMMadrid Robotics Digital Innovation Hub ("Robótica aplicada a la mejora de la calidad de vida de los ciudadanos, Fase IV"; S2018/NMT-4331), which is funded by the Programas de Actividades I+DComunidad de Madrid and cofunded by the Structural Funds of the EU

Expert-in-the-Loop Multilateral Telerobotics for Haptics-Enabled Motor Function and Skills Development

Among medical robotics applications are Robotics-Assisted Mirror Rehabilitation Therapy (RAMRT) and Minimally-Invasive Surgical Training (RAMIST) that extensively rely on motor function development. Haptics-enabled expert-in-the-loop motor function development for such applications is made possible through multilateral telerobotic frameworks. While several studies have validated the benefits of haptic interaction with an expert in motor learning, contradictory results have also been reported. This emphasizes the need for further in-depth studies on the nature of human motor learning through haptic guidance and interaction. The objective of this study was to design and evaluate expert-in-the-loop multilateral telerobotic frameworks with stable and human-safe control loops that enable adaptive “hand-over-hand” haptic guidance for RAMRT and RAMIST. The first prerequisite for such frameworks is active involvement of the patient or trainee, which requires the closed-loop system to remain stable in the presence of an adaptable time-varying dominance factor. To this end, a wave-variable controller is proposed in this study for conventional trilateral teleoperation systems such that system stability is guaranteed in the presence of a time-varying dominance factor and communication delay. Similar to other wave-variable approaches, the controller is initially developed for the Velocity-force Domain (VD) based on the well-known passivity assumption on the human arm in VD. The controller can be applied straightforwardly to the Position-force Domain (PD), eliminating position-error accumulation and position drift, provided that passivity of the human arm in PD is addressed. However, the latter has been ignored in the literature. Therefore, in this study, passivity of the human arm in PD is investigated using mathematical analysis, experimentation as well as user studies involving 12 participants and 48 trials. The results, in conjunction with the proposed wave-variables, can be used to guarantee closed-loop PD stability of the supervised trilateral teleoperation system in its classical format. The classic dual-user teleoperation architecture does not, however, fully satisfy the requirements for properly imparting motor function (skills) in RAMRT (RAMIST). Consequently, the next part of this study focuses on designing novel supervised trilateral frameworks for providing motor learning in RAMRT and RAMIST, each customized according to the requirements of the application. The framework proposed for RAMRT includes the following features: a) therapist-in-the-loop mirror therapy; b) haptic feedback to the therapist from the patient side; c) assist-as-needed therapy realized through an adaptive Guidance Virtual Fixture (GVF); and d) real-time task-independent and patient-specific motor-function assessment. Closed-loop stability of the proposed framework is investigated using a combination of the Circle Criterion and the Small-Gain Theorem. The stability analysis addresses the instabilities caused by: a) communication delays between the therapist and the patient, facilitating haptics-enabled tele- or in-home rehabilitation; and b) the integration of the time-varying nonlinear GVF element into the delayed system. The platform is experimentally evaluated on a trilateral rehabilitation setup consisting of two Quanser rehabilitation robots and one Quanser HD2 robot. The framework proposed for RAMIST includes the following features: a) haptics-enabled expert-in-the-loop surgical training; b) adaptive expertise-oriented training, realized through a Fuzzy Interface System, which actively engages the trainees while providing them with appropriate skills-oriented levels of training; and c) task-independent skills assessment. Closed-loop stability of the architecture is analyzed using the Circle Criterion in the presence and absence of haptic feedback of tool-tissue interactions. In addition to the time-varying elements of the system, the stability analysis approach also addresses communication delays, facilitating tele-surgical training. The platform is implemented on a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA), integrated with the da Vinci Research Kit (dVRK) motor controllers, and the dV-Trainer master console (Mimic Technology Inc., Seattle, WA). In order to save on the expert\u27s (therapist\u27s) time, dual-console architectures can also be expanded to accommodate simultaneous training (rehabilitation) for multiple trainees (patients). As the first step in doing this, the last part of this thesis focuses on the development of a multi-master/single-slave telerobotic framework, along with controller design and closed-loop stability analysis in the presence of communication delays. Various parts of this study are supported with a number of experimental implementations and evaluations. The outcomes of this research include multilateral telerobotic testbeds for further studies on the nature of human motor learning and retention through haptic guidance and interaction. They also enable investigation of the impact of communication time delays on supervised haptics-enabled motor function improvement through tele-rehabilitation and mentoring

Control System and Graphical User Interface Design of an Upper-Extremity Rehabilitation Robot

Stroke is one of the leading causes of death, physical disability, and loss of brain functionality each year, especially amongst older adults. The ability to access good quality post-stroke rehabilitation exercises is essential for stroke survivors to maximize their potential to regain skills and physical abilities. Robot-assisted therapy is showing promise as a way to provide stroke survivors with engaging, challenging, and repetitive tasks while delivering measured therapy that is able to objectively evaluate patients’ progress. Among several challenges that are associated with the design of rehabilitation robots (e.g., the mechanical structure, the actuator types, the control strategies), the design of the control strategy is one of the most critical. Depending on the type of patient and the severity of the impairment of motor control, various control strategies could be applied for the recovery of the impaired limb in stroke survivors using robot-assisted therapy. Research is needed into the development of how best to control rehabilitation robots; this includes both the internal control algorithms and the User Interface (UI) for therapists. As such, the first objective of this research is to design and implement a motion controller and force-field controller for a 2-Degree of Freedom (DOF) manipulandum upper-extremity rehabilitation robot that is able to deliver planar rehabilitation exercises for stroke survivors while taking therapeutic rehabilitation goals into account. The motion control algorithm can precisely follow a prescribed time-dependent trajectory whereas the force-control method will only provide assistance (or even resistance to introduce extra challenge) to the patient to do the task rather than forcing the movement. For doing the simulation studies, a motor control model of post-stroke patients was proposed. The effectiveness of these controllers was explored in simulations and it was observed that the developed force-field algorithm had a positive effect on the motor control recovery for a simulated patient. The simulation results also indicated that the resistive mode of therapy would result in better outcomes after the therapy which aligns with experimental studies by other researchers. In addition, a novel adaptive algorithm was proposed for fine-tuning the proposed force-field parameters based on the performance of the patient during the therapy as a subject specific controller can help to achieve a desirable performance for each patient. While this approach is promising, the effectiveness of the adaptation rule has yet to be evaluated on real patients in the future. To enable effective access and use of the robot, the controller needs to be visualized through a Graphical User Interface (GUI) in a way that therapists can understand and use. The second goal of this thesis research was to work with therapists to collaboratively design an intuitive to use GUI for therapists to control the robot and provide objective information on patients’ performance. The identification of features and feedback on the intuitiveness of the GUI developed in this research highlights the value of collaborative design between engineers and therapists to create the interface that enables therapists to control the rehabilitation robot. This research also identifies the need for collaborative GUI design with patients as their needs and preference may be different from therapists. During the collaborative GUI design, it was observed that including obstacles and force-field method might be a possible useful method for supporting patients’ movement trajectory, not only because therapists can adjust the force strength to suit a specific patient, but also because they can use its numerical data for objective measurement of patients’ performance. Therapists who participated in this research stated that objective measurements (i.e., trajectory smoothness, speed, mobility range, and error) could be used to evaluate the patient performance. While rehabilitation robots are different in terms of mechanical structure, work-space, and the exercise that they can provide, similar methods could be used for supporting patients’ movement trajectory and performance evaluation. As the GUI is the first prototype, it needs to be used with and evaluated by therapists and patients to ascertain if the information presented in the GUI is intuitive and to explore if they can understand it or use it

Cerebral Palsy

Nowadays, cerebral palsy (CP) rehabilitation, along with medical and surgical interventions in children with CP, leads to better motor and postural control and can ensure ambulation and functional independence. In achieving these improvements, many modern practices may be used, such as comprehensive multidisciplinary assessment, clinical decision making, multilevel surgery, botulinum toxin applications, robotic ambulation applications, treadmill, and other walking aids to increase the quality and endurance of walking. Trainings are based on neurodevelopmental therapy, muscle training and strength applications, adaptive equipment and orthotics, communication, technological solves, and many others beyond the scope of this book. In the years of clinical and academic experiences, children with cerebral palsy have shown us that the world needs a book to give clinical knowledge to health professionals regarding these important issue. This book is an attempt to fulfill and to give “current steps” about CP. The book is intended for use by physicians, therapists, and allied health professionals who treat/rehabilitate children with CP. We focus on the recent concepts in the treatment of body and structure problems and describe the associated disability, providing suggestions for further reading. All authors presented the most frequently used and accepted treatment methods with scientifically proven efficacy and included references at the end of each chapter