Safe Haptics-enabled Patient-Robot Interaction for Robotic and Telerobotic Rehabilitation of Neuromuscular Disorders: Control Design and Analysis

Motivation: Current statistics show that the population of seniors and the incidence rate of age-related neuromuscular disorders are rapidly increasing worldwide. Improving medical care is likely to increase the survival rate but will result in even more patients in need of Assistive, Rehabilitation and Assessment (ARA) services for extended periods which will place a significant burden on the world\u27s healthcare systems. In many cases, the only alternative is limited and often delayed outpatient therapy. The situation will be worse for patients in remote areas. One potential solution is to develop technologies that provide efficient and safe means of in-hospital and in-home kinesthetic rehabilitation. In this regard, Haptics-enabled Interactive Robotic Neurorehabilitation (HIRN) systems have been developed. Existing Challenges: Although there are specific advantages with the use of HIRN technologies, there still exist several technical and control challenges, e.g., (a) absence of direct interactive physical interaction between therapists and patients; (b) questionable adaptability and flexibility considering the sensorimotor needs of patients; (c) limited accessibility in remote areas; and (d) guaranteeing patient-robot interaction safety while maximizing system transparency, especially when high control effort is needed for severely disabled patients, when the robot is to be used in a patient\u27s home or when the patient experiences involuntary movements. These challenges have provided the motivation for this research. Research Statement: In this project, a novel haptics-enabled telerobotic rehabilitation framework is designed, analyzed and implemented that can be used as a new paradigm for delivering motor therapy which gives therapists direct kinesthetic supervision over the robotic rehabilitation procedure. The system also allows for kinesthetic remote and ultimately in-home rehabilitation. To guarantee interaction safety while maximizing the performance of the system, a new framework for designing stabilizing controllers is developed initially based on small-gain theory and then completed using strong passivity theory. The proposed control framework takes into account knowledge about the variable biomechanical capabilities of the patient\u27s limb(s) in absorbing interaction forces and mechanical energy. The technique is generalized for use for classical rehabilitation robotic systems to realize patient-robot interaction safety while enhancing performance. In the next step, the proposed telerobotic system is studied as a modality of training for classical HIRN systems. The goal is to first model and then regenerate the prescribed kinesthetic supervision of an expert therapist. To broaden the population of patients who can use the technology and HIRN systems, a new control strategy is designed for patients experiencing involuntary movements. As the last step, the outcomes of the proposed theoretical and technological developments are translated to designing assistive mechatronic tools for patients with force and motion control deficits. This study shows that proper augmentation of haptic inputs can not only enhance the transparency and safety of robotic and telerobotic rehabilitation systems, but it can also assist patients with force and motion control deficiencies

Expert-in-the-Loop Multilateral Telerobotics for Haptics-Enabled Motor Function and Skills Development

Among medical robotics applications are Robotics-Assisted Mirror Rehabilitation Therapy (RAMRT) and Minimally-Invasive Surgical Training (RAMIST) that extensively rely on motor function development. Haptics-enabled expert-in-the-loop motor function development for such applications is made possible through multilateral telerobotic frameworks. While several studies have validated the benefits of haptic interaction with an expert in motor learning, contradictory results have also been reported. This emphasizes the need for further in-depth studies on the nature of human motor learning through haptic guidance and interaction. The objective of this study was to design and evaluate expert-in-the-loop multilateral telerobotic frameworks with stable and human-safe control loops that enable adaptive “hand-over-hand” haptic guidance for RAMRT and RAMIST. The first prerequisite for such frameworks is active involvement of the patient or trainee, which requires the closed-loop system to remain stable in the presence of an adaptable time-varying dominance factor. To this end, a wave-variable controller is proposed in this study for conventional trilateral teleoperation systems such that system stability is guaranteed in the presence of a time-varying dominance factor and communication delay. Similar to other wave-variable approaches, the controller is initially developed for the Velocity-force Domain (VD) based on the well-known passivity assumption on the human arm in VD. The controller can be applied straightforwardly to the Position-force Domain (PD), eliminating position-error accumulation and position drift, provided that passivity of the human arm in PD is addressed. However, the latter has been ignored in the literature. Therefore, in this study, passivity of the human arm in PD is investigated using mathematical analysis, experimentation as well as user studies involving 12 participants and 48 trials. The results, in conjunction with the proposed wave-variables, can be used to guarantee closed-loop PD stability of the supervised trilateral teleoperation system in its classical format. The classic dual-user teleoperation architecture does not, however, fully satisfy the requirements for properly imparting motor function (skills) in RAMRT (RAMIST). Consequently, the next part of this study focuses on designing novel supervised trilateral frameworks for providing motor learning in RAMRT and RAMIST, each customized according to the requirements of the application. The framework proposed for RAMRT includes the following features: a) therapist-in-the-loop mirror therapy; b) haptic feedback to the therapist from the patient side; c) assist-as-needed therapy realized through an adaptive Guidance Virtual Fixture (GVF); and d) real-time task-independent and patient-specific motor-function assessment. Closed-loop stability of the proposed framework is investigated using a combination of the Circle Criterion and the Small-Gain Theorem. The stability analysis addresses the instabilities caused by: a) communication delays between the therapist and the patient, facilitating haptics-enabled tele- or in-home rehabilitation; and b) the integration of the time-varying nonlinear GVF element into the delayed system. The platform is experimentally evaluated on a trilateral rehabilitation setup consisting of two Quanser rehabilitation robots and one Quanser HD2 robot. The framework proposed for RAMIST includes the following features: a) haptics-enabled expert-in-the-loop surgical training; b) adaptive expertise-oriented training, realized through a Fuzzy Interface System, which actively engages the trainees while providing them with appropriate skills-oriented levels of training; and c) task-independent skills assessment. Closed-loop stability of the architecture is analyzed using the Circle Criterion in the presence and absence of haptic feedback of tool-tissue interactions. In addition to the time-varying elements of the system, the stability analysis approach also addresses communication delays, facilitating tele-surgical training. The platform is implemented on a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA), integrated with the da Vinci Research Kit (dVRK) motor controllers, and the dV-Trainer master console (Mimic Technology Inc., Seattle, WA). In order to save on the expert\u27s (therapist\u27s) time, dual-console architectures can also be expanded to accommodate simultaneous training (rehabilitation) for multiple trainees (patients). As the first step in doing this, the last part of this thesis focuses on the development of a multi-master/single-slave telerobotic framework, along with controller design and closed-loop stability analysis in the presence of communication delays. Various parts of this study are supported with a number of experimental implementations and evaluations. The outcomes of this research include multilateral telerobotic testbeds for further studies on the nature of human motor learning and retention through haptic guidance and interaction. They also enable investigation of the impact of communication time delays on supervised haptics-enabled motor function improvement through tele-rehabilitation and mentoring

The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

Design of a Haptic Interface for Medical Applications using Magneto-Rheological Fluid based Actuators

This thesis reports on the design, construction, and evaluation of a prototype two degrees-of-freedom (DOF) haptic interface, which takes advantage of Magneto-Rheological Fluid (MRF) based clutches for actuation. Haptic information provides important cues in teleoperated systems and enables the user to feel the interaction with a remote or virtual environment during teleoperation. The two main objectives in designing a haptic interface are stability and transparency. Indeed, deficiencies in these factors in haptics-enabled telerobotic systems has the introduction of haptics in medical environments where safety and reliability are prime considerations. An actuator with poor dynamics, high inertia, large size, and heavy weight can significantly undermine the stability and transparency of a teleoperated system. In this work, the potential benefits of MRF-based actuators to the field of haptics in medical applications are studied. Devices developed with such fluids are known to possess superior mechanical characteristics over conventional servo systems. These characteristics significantly contribute to improved stability and transparency of haptic devices. This idea is evaluated and verified through both theoretical and experimental points of view. The design of a small-scale MRF-based clutch, suitable for a multi-DOF haptic interface, is discussed and its performance is compared with conventional servo systems. This design is developed into four prototype clutches. In addition, a closed-loop torque control strategy is presented. The feedback signal used in this control scheme comes from the magnetic field acquired from embedded Hall sensors in the clutch. The controller uses this feedback signal to compensate for the nonlinear behavior using an estimated model, based on Artificial Neural Networks. Such a control strategy eliminates the need for torque sensors for providing feedback signals. The performance of the developed design and the effectiveness of the proposed modeling and control techniques are experimentally validated. Next, a 2-DOF haptic interface based on a distributed antagonistic configuration of MRF-based clutches is constructed for a class of medical applications. This device is incorporated in a master-slave teleoperation setup that is used for applications involving needle insertion and soft-tissue palpation. Phantom and in vitro animal tissue were used to assess the performance of the haptic interface. The results show a great potential of MRF-based actuators for integration in haptic devices for medical interventions that require reliable, safe, accurate, highly transparent, and stable force reflection

A Review of Pneumatic Actuators Used for the Design of Medical Simulators and Medical Tools

International audienc

Design, implementation, control, and user evaluations of assiston-arm self-aligning upper-extremity exoskeleton

Physical rehabilitation therapy is indispensable for treating neurological disabilities. The use of robotic devices for rehabilitation holds high promise, since these devices can bear the physical burden of rehabilitation exercises during intense therapy sessions, while therapists are employed as decision makers. Robot-assisted rehabilitation devices are advantageous as they can be applied to patients with all levels of impairment, allow for easy tuning of the duration and intensity of therapies and enable customized, interactive treatment protocols. Moreover, since robotic devices are particularly good at repetitive tasks, rehabilitation robots can decrease the physical burden on therapists and enable a single therapist to supervise multiple patients simultaneously; hence, help to lower cost of therapies. While the intensity and quality of manually delivered therapies depend on the skill and fatigue level of therapists, high-intensity robotic therapies can always be delivered with high accuracy. Thanks to their integrated sensors, robotic devices can gather measurements throughout therapies, enable quantitative tracking of patient progress and development of evidence-based personalized rehabilitation programs. In this dissertation, we present the design, control, characterization and user evaluations of AssistOn-Arm, a powered, self-aligning exoskeleton for robotassisted upper-extremity rehabilitation. AssistOn-Arm is designed as a passive back-driveable impedance-type robot such that patients/therapists can move the device transparently, without much interference of the device dynamics on natural movements. Thanks to its novel kinematics and mechanically transparent design, AssistOn-Arm can passively self-align its joint axes to provide an ideal match between human joint axes and the exoskeleton axes, guaranteeing ergonomic movements and comfort throughout physical therapies. The self-aligning property of AssistOn-Arm not only increases the usable range of motion for robot-assisted upper-extremity exercises to cover almost the whole human arm workspace, but also enables the delivery of glenohumeral mobilization (scapular elevation/depression and protraction/retraction) and scapular stabilization exercises, extending the type of therapies that can be administered using upper-extremity exoskeletons. Furthermore, the self-alignment property of AssistOn-Arm signi cantly shortens the setup time required to attach a patient to the exoskeleton. As an impedance-type device with high passive back-driveability, AssistOn- Arm can be force controlled without the need of force sensors; hence, high delity interaction control performance can be achieved with open-loop impedance control. This control architecture not only simpli es implementation, but also enhances safety (coupled stability robustness), since open-loop force control does not su er from the fundamental bandwidth and stability limitations of force-feedback. Experimental characterizations and user studies with healthy volunteers con- rm the transparency, range of motion, and control performance of AssistOn- Ar

A review on manipulation skill acquisition through teleoperation-based learning from demonstration

Manipulation skill learning and generalization have gained increasing attention due to the wide applications of robot manipulators and the spurt of robot learning techniques. Especially, the learning from demonstration method has been exploited widely and successfully in the robotic community, and it is regarded as a promising direction to realize the manipulation skill learning and generalization. In addition to the learning techniques, the immersive teleoperation enables the human to operate a remote robot with an intuitive interface and achieve the telepresence. Thus, it is a promising way to transfer manipulation skills from humans to robots by combining the learning methods and the teleoperation, and adapting the learned skills to different tasks in new situations. This review, therefore, aims to provide an overview of immersive teleoperation for skill learning and generalization to deal with complex manipulation tasks. To this end, the key technologies, e.g. manipulation skill learning, multimodal interfacing for teleoperation and telerobotic control, are introduced. Then, an overview is given in terms of the most important applications of immersive teleoperation platform for robot skill learning. Finally, this survey discusses the remaining open challenges and promising research topics

A Service Robot for Navigation Assistance and Physical Rehabilitation of Seniors

The population of the advanced countries is ageing, with the direct consequence that an increasing number of people will have to live with sensitive, cognitive and physical disabilities. People with impaired physical ability are not confident to move alone, especially in crowded environment and for long journeys, highly reducing the quality of their life. We propose a new generation of robotic walking assistants whose mechanical and electronic components are conceived to optimize the collaboration between the robot and its users. We will apply these general ideas to investigate the interaction between older adults and a robotic walker, named FriWalk, exploiting it either as a navigational or as a rehabilitation aid. For the use of the FriWalk as a navigation assistance, the system guides the user securing high levels of safety, a perfect compliance with the social rules and non-intrusive interaction between human and machine. To this purpose, we developed several guidance systems ranging from completely passive strategies to active solutions exploiting either the rear or the front motors mounted on the robot. The common strategy at the basis of all the algorithms is that the responsibility of the locomotion belongs always to the user, both to increase the mobility of elder users and to enhance their perception of control over the robot. This way the robot intervenes only whenever it is strictly necessary not to mitigate the user safety. Moreover, the robotic walker has been endowed with a tablet and graphical user interface (GUI) which provides the user with the visual indications about the path to follow. Since the FriWalk was developed to suit the needs of users with different deficits, we conducted extensive human-robot interaction (HRI) experiments with elders, complemented with direct interviews of the participants. As concerns the use of the FriWalk as a rehabilitation aid, force sensing to estimate the torques applied by the user and change the user perceived inertia can be exploited by doctors to let the user feel the device heavier or lighter. Moreover, thanks to a new generation of sensors, the device can be exploited in a clinical context to track the performance of the users' rehabilitation exercises, in order to assist nurses and doctors during the hospitalization of older adults

Design and Control of Robotic Systems for Lower Limb Stroke Rehabilitation

Lower extremity stroke rehabilitation exhausts considerable health care resources, is labor intensive, and provides mostly qualitative metrics of patient recovery. To overcome these issues, robots can assist patients in physically manipulating their affected limb and measure the output motion. The robots that have been currently designed, however, provide assistance over a limited set of training motions, are not portable for in-home and in-clinic use, have high cost and may not provide sufficient safety or performance. This thesis proposes the idea of incorporating a mobile drive base into lower extremity rehabilitation robots to create a portable, inherently safe system that provides assistance over a wide range of training motions. A set of rehabilitative motion tasks were established and a six-degree-of-freedom (DOF) motion and force-sensing system was designed to meet high-power, large workspace, and affordability requirements. An admittance controller was implemented, and the feasibility of using this portable, low-cost system for movement assistance was shown through tests on a healthy individual. An improved version of the robot was then developed that added torque sensing and known joint elasticity for use in future clinical testing with a flexible-joint impedance controller

Homecare Robotic Systems for Healthcare 4.0: Visions and Enabling Technologies

Powered by the technologies that have originated from manufacturing, the fourth revolution of healthcare technologies is happening (Healthcare 4.0). As an example of such revolution, new generation homecare robotic systems (HRS) based on the cyber-physical systems (CPS) with higher speed and more intelligent execution are emerging. In this article, the new visions and features of the CPS-based HRS are proposed. The latest progress in related enabling technologies is reviewed, including artificial intelligence, sensing fundamentals, materials and machines, cloud computing and communication, as well as motion capture and mapping. Finally, the future perspectives of the CPS-based HRS and the technical challenges faced in each technical area are discussed