Prevalence of haptic feedback in robot-mediated surgery : a systematic review of literature

© 2017 Springer-Verlag. This is a post-peer-review, pre-copyedit version of an article published in Journal of Robotic Surgery. The final authenticated version is available online at: https://doi.org/10.1007/s11701-017-0763-4With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.Peer reviewe

Intuitive Human-Machine Interfaces for Non-Anthropomorphic Robotic Hands

As robots become more prevalent in our everyday lives, both in our workplaces and in our homes, it becomes increasingly likely that people who are not experts in robotics will be asked to interface with robotic devices. It is therefore important to develop robotic controls that are intuitive and easy for novices to use. Robotic hands, in particular, are very useful, but their high dimensionality makes creating intuitive human-machine interfaces for them complex. In this dissertation, we study the control of non-anthropomorphic robotic hands by non-roboticists in two contexts: collaborative manipulation and assistive robotics. In the field of collaborative manipulation, the human and the robot work side by side as independent agents. Teleoperation allows the human to assist the robot when autonomous grasping is not able to deal sufficiently well with corner cases or cannot operate fast enough. Using the teleoperator’s hand as an input device can provide an intuitive control method, but finding a mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands’ dissimilar kinematics. In this dissertation, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to enable effective real-time teleoperation, even for novice users. We propose a low-dimensional and continuous teleoperation subspace which can be used as an intermediary for mapping between different hand pose spaces. We first propose the general concept of the subspace, its properties and the variables needed to map from the human hand to a robot hand. We then propose three ways to populate the teleoperation subspace mapping. Two of our mappings use a dataglove to harvest information about the user's hand. We define the mapping between joint space and teleoperation subspace with an empirical definition, which requires a person to define hand motions in an intuitive, hand-specific way, and with an algorithmic definition, which is kinematically independent, and uses objects to define the subspace. Our third mapping for the teleoperation subspace uses forearm electromyography (EMG) as a control input. Assistive orthotics is another area of robotics where human-machine interfaces are critical, since, in this field, the robot is attached to the hand of the human user. In this case, the goal is for the robot to assist the human with movements they would not otherwise be able to achieve. Orthotics can improve the quality of life of people who do not have full use of their hands. Human-machine interfaces for assistive hand orthotics that use EMG signals from the affected forearm as input are intuitive and repeated use can strengthen the muscles of the user's affected arm. In this dissertation, we seek to create an EMG based control for an orthotic device used by people who have had a stroke. We would like our control to enable functional motions when used in conjunction with a orthosis and to be robust to changes in the input signal. We propose a control for a wearable hand orthosis which uses an easy to don, commodity forearm EMG band. We develop an supervised algorithm to detect a user’s intent to open and close their hand, and pair this algorithm with a training protocol which makes our intent detection robust to changes in the input signal. We show that this algorithm, when used in conjunction with an orthosis over several weeks, can improve distal function in users. Additionally, we propose two semi-supervised intent detection algorithms designed to keep our control robust to changes in the input data while reducing the length and frequency of our training protocol

On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation

Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas

A virtual hand assessment system for efficient outcome measures of hand rehabilitation

Previously held under moratorium from 1st December 2016 until 1st December 2021.Hand rehabilitation is an extremely complex and critical process in the medical rehabilitation field. This is mainly due to the high articulation of the hand functionality. Recent research has focused on employing new technologies, such as robotics and system control, in order to improve the precision and efficiency of the standard clinical methods used in hand rehabilitation. However, the designs of these devices were either oriented toward a particular hand injury or heavily dependent on subjective assessment techniques to evaluate the progress. These limitations reduce the efficiency of the hand rehabilitation devices by providing less effective results for restoring the lost functionalities of the dysfunctional hands. In this project, a novel technological solution and efficient hand assessment system is produced that can objectively measure the restoration outcome and, dynamically, evaluate its performance. The proposed system uses a data glove sensorial device to measure the multiple ranges of motion for the hand joints, and a Virtual Reality system to return an illustrative and safe visual assistance environment that can self-adjust with the subject’s performance. The system application implements an original finger performance measurement method for analysing the various hand functionalities. This is achieved by extracting the multiple features of the hand digits’ motions; such as speed, consistency of finger movements and stability during the hold positions. Furthermore, an advanced data glove calibration method was developed and implemented in order to accurately manipulate the virtual hand model and calculate the hand kinematic movements in compliance with the biomechanical structure of the hand. The experimental studies were performed on a controlled group of 10 healthy subjects (25 to 42 years age). The results showed intra-subject reliability between the trials (average of crosscorrelation ρ = 0.7), inter-subject repeatability across the subject’s performance (p > 0.01 for the session with real objects and with few departures in some of the virtual reality sessions). In addition, the finger performance values were found to be very efficient in detecting the multiple elements of the fingers’ performance including the load effect on the forearm. Moreover, the electromyography measurements, in the virtual reality sessions, showed high sensitivity in detecting the tremor effect (the mean power frequency difference on the right Vextensor digitorum muscle is 176 Hz). Also, the finger performance values for the virtual reality sessions have the same average distance as the real life sessions (RSQ =0.07). The system, besides offering an efficient and quantitative evaluation of hand performance, it was proven compatible with different hand rehabilitation techniques where it can outline the primarily affected parts in the hand dysfunction. It also can be easily adjusted to comply with the subject’s specifications and clinical hand assessment procedures to autonomously detect the classification task events and analyse them with high reliability. The developed system is also adaptable with different disciplines’ involvements, other than the hand rehabilitation, such as ergonomic studies, hand robot control, brain-computer interface and various fields involving hand control.Hand rehabilitation is an extremely complex and critical process in the medical rehabilitation field. This is mainly due to the high articulation of the hand functionality. Recent research has focused on employing new technologies, such as robotics and system control, in order to improve the precision and efficiency of the standard clinical methods used in hand rehabilitation. However, the designs of these devices were either oriented toward a particular hand injury or heavily dependent on subjective assessment techniques to evaluate the progress. These limitations reduce the efficiency of the hand rehabilitation devices by providing less effective results for restoring the lost functionalities of the dysfunctional hands. In this project, a novel technological solution and efficient hand assessment system is produced that can objectively measure the restoration outcome and, dynamically, evaluate its performance. The proposed system uses a data glove sensorial device to measure the multiple ranges of motion for the hand joints, and a Virtual Reality system to return an illustrative and safe visual assistance environment that can self-adjust with the subject’s performance. The system application implements an original finger performance measurement method for analysing the various hand functionalities. This is achieved by extracting the multiple features of the hand digits’ motions; such as speed, consistency of finger movements and stability during the hold positions. Furthermore, an advanced data glove calibration method was developed and implemented in order to accurately manipulate the virtual hand model and calculate the hand kinematic movements in compliance with the biomechanical structure of the hand. The experimental studies were performed on a controlled group of 10 healthy subjects (25 to 42 years age). The results showed intra-subject reliability between the trials (average of crosscorrelation ρ = 0.7), inter-subject repeatability across the subject’s performance (p > 0.01 for the session with real objects and with few departures in some of the virtual reality sessions). In addition, the finger performance values were found to be very efficient in detecting the multiple elements of the fingers’ performance including the load effect on the forearm. Moreover, the electromyography measurements, in the virtual reality sessions, showed high sensitivity in detecting the tremor effect (the mean power frequency difference on the right Vextensor digitorum muscle is 176 Hz). Also, the finger performance values for the virtual reality sessions have the same average distance as the real life sessions (RSQ =0.07). The system, besides offering an efficient and quantitative evaluation of hand performance, it was proven compatible with different hand rehabilitation techniques where it can outline the primarily affected parts in the hand dysfunction. It also can be easily adjusted to comply with the subject’s specifications and clinical hand assessment procedures to autonomously detect the classification task events and analyse them with high reliability. The developed system is also adaptable with different disciplines’ involvements, other than the hand rehabilitation, such as ergonomic studies, hand robot control, brain-computer interface and various fields involving hand control

Design and Prototyping of an Underactuated Hand Exoskeleton With Fingers Coupled by a Gear-Based Differential

Exoskeletons and more in general wearable mechatronic devices represent a promising opportunity for rehabilitation and assistance to people presenting with temporary and/or permanent diseases. However, there are still some limits in the diffusion of robotic technologies for neuro-rehabilitation, notwithstanding their technological developments and evidence of clinical effectiveness. One of the main bottlenecks that constrain the complexity, weight, and costs of exoskeletons is represented by the actuators. This problem is particularly evident in devices designed for the upper limb, and in particular for the hand, in which dimension limits and kinematics complexity are particularly challenging. This study presents the design and prototyping of a hand finger exoskeleton. In particular, we focus on the design of a gear-based differential mechanism aimed at coupling the motion of two adjacent fingers and limiting the complexity and costs of the system. The exoskeleton is able to actuate the flexion/extension motion of the fingers and apply bidirectional forces, that is, it is able to both open and close the fingers. The kinematic structure of the finger actuation system has the peculiarity to present three DoFs when the exoskeleton is not worn and one DoF when it is worn, allowing better adaptability and higher wearability. The design of the gear-based differential is inspired by the mechanism widely used in the automotive field; it allows actuating two fingers with one actuator only, keeping their movements independent

Kinematic Analysis of Multi-Fingered, Anthropomorphic Robotic Hands

The ability of stable grasping and fine manipulation with the multi-fingered robot hand with required precision and dexterity is playing an increasingly important role in the applications like service robots, rehabilitation, humanoid robots, entertainment robots, industries etc.. A number of multi-fingered robotic hands have been developed by various researchers in the past. The distinct advantages of a multi-fingered robot hand having structural similarity with human hand motivate the need for an anthropomorphic robot hand. Such a hand provides a promising base for supplanting human hand in execution of tedious, complicated and dangerous tasks, especially in situations such as manufacturing, space, undersea etc. These can also be used in orthopaedic rehabilitation of humans for improving the quality of the life of people having orthopedically and neurological disabilities. The developments so far are mostly driven by the application requirements. There are a number of bottlenecks with industrial grippers as regards to the stability of grasping objects of irregular geometries or complex manipulation operations. A multi-fingered robot hand can be made to mimic the movements of a human hand. The present piece of research work attempts to conceptualize and design a multi-fingered, anthropomorphic robot hand by structurally imitating the human hand. In the beginning, a brief idea about the history, types of robotic hands and application of multi-fingered hands in various fields are presented. A review of literature based on different aspects of the multi-fingered hand like structure, control, optimization, gasping etc. is made. Some of the important and more relevant literatures are elaborately discussed and a brief analysis is made on the outcomes and shortfalls with respect to multi-fingered hands. Based on the analysis of the review of literature, the research work aims at developing an improved anthropomorphic robot hand model in which apart from the four fingers and a thumb, the palm arch effect of human hand is also considered to increase its dexterity. A robotic hand with five anthropomorphic fingers including the thumb and palm arch effect having 25 degrees-of-freedom in all is investigated in the present work. Each individual finger is considered as an open loop kinematic chain and each finger segment is considered as a link of the manipulator. The wrist of the hand is considered as a fixed point. The kinematic analyses of the model for both forward kinematics and inverse kinematic are carried out. The trajectories of the tip positions of the thumb and the fingers with respect to local coordinate system are determined and plotted. This gives the extreme position of the fingertips which is obtained from the forward kinematic solution with the help of MATLAB. Similarly, varying all the joint iv angles of the thumb and fingers in their respective ranges, the reachable workspace of the hand model is obtained. Adaptive Neuro-Fuzzy Inference System (ANFIS) is used for solving the inverse kinematic problem of the fingers. Since the multi-fingered hand grasps the object mainly through its fingertips and the manipulation of the object is facilitated by the fingers due to their dexterity, the grasp is considered to be force-closure grasp. The grasping theory and different types of contacts between the fingertip and object are presented and the conditions for stable and equilibrium grasp are elaborately discussed. The proposed hand model is simulated to grasp five different shaped objects with equal base dimension and height. The forces applied on the fingertip during grasping are calculated. The hand model is also analysed using ANSYS to evaluate the stresses being developed at various points in the thumb and fingers. This analysis was made for the hand considering two different hand materials i.e. aluminium alloy and structural steel. The solution obtained from the forward kinematic analysis of the hand determines the maximum size for differently shaped objects while the solution to the inverse kinematic problem indicates the configurations of the thumb and the fingers inside the workspace of the hand. The solutions are predicted in which all joint angles are within their respective ranges. The results of the stress analysis of the hand model show that the structure of the fingers and the hand as a whole is capable of handling the selected objects. The robot hand under investigation can be realized and can be a very useful tool for many critical areas such as fine manipulation of objects, combating orthopaedic or neurological impediments, service robotics, entertainment robotics etc. The dissertation concludes with a summary of the contribution and the scope of further work

Wearable Hand Exoskeleton Systems for Virtual Reality and Rehabilitation

Department of Mechanical Engineeringthe aim is to overcome the limitations of conventional systems in terms of both wearability and portability. As the hand receives diverse physical information and manipulates different type of objects, conventional systems contain many sensors and actuators, and are both large and heavy. Thus, hand exoskeleton systems exhibiting high wearability and portability while measuring finger motions and delivering forces would be highly valuable. For VR hand exoskeleton systems, a wearable hand exoskeleton system with force-controllable actuator modules was developed to ensure free finger motion and force mode control. The linkage structure ensures motion with three degrees of freedom (DOF) and provides a large fingertip workspacethe finger postures assumed when interacting with objects are appropriate. A series elastic actuator (SEA) with an actuator and an elastic element was used to fabricate compact actuator modules. Actuator friction was eliminated using a friction compensation algorithm. A proportional differential (PD) controller, optimized by a linear quadratic (LQ) method featuring a disturbance observer (DOB), was used to ensure accurate force mode control even during motion. The force control performance of the actuator module was verified in force generation experiments including stationary and arbitrary end-effector motions. The forces applied to the fingertips, which are the principal parts of the hand that interact with objects, were kinematically analyzed via both simulations and experiments. To overcome the weak point of previous system, a wearable hand exoskeleton system featuring finger motion measurement and force feedback was developed and evaluated in terms of user experience (UX). The finger structures for the thumb, index, and middle fingers, which play important roles when grasping objects, satisfy full range of motion (ROM). The system estimates all joint angles of these three digits using a dedicated algorithmmeasurement accuracy was experimentally evaluated to verify system performance. The UX performance was evaluated by 15 undergraduate students who completed questionnaires assessing usability and utilitarian value following trials conducted in the laboratory. All subjects were highly satisfied with both usability and the utilitarian nature of the system, not only because control and feedback were intuitive but also because performance was accurate. For rehabilitation, a highly portable exoskeleton featuring flexion/extension finger exercises was developed. The exoskeleton features two four-bar linkages reflecting the natural metacarpophalangeal (MCP) and proximal phalangeal (PIP) joint angles. During optimization, the design parameters were adjusted to reflect normal finger trajectories, which vary by finger length and finger joint ROM. To allow for passive physical impedance, a spring was installed to generate the forces that guided the fingers. The moments transmitted to the MCP and PIP joints were estimated via finite element method (FEM) analysis and the cross-sectional areas of the links were manually designed by reference to the expected joint moments. Finger motion and force distribution experiments verified that the system guided the fingers effectively, allowed for the desired finger motions, and distributed the required moments to the joints (as revealed by FEM analysis).This thesis reports the development of hand exoskeleton systems, for use in virtual reality (VR) environments and for hand rehabilitationclos

Innovative robot hand designs of reduced complexity for dexterous manipulation

This thesis investigates the mechanical design of robot hands to sensibly reduce the system complexity in terms of the number of actuators and sensors, and control needs for performing grasping and in-hand manipulations of unknown objects. Human hands are known to be the most complex, versatile, dexterous manipulators in nature, from being able to operate sophisticated surgery to carry out a wide variety of daily activity tasks (e.g. preparing food, changing cloths, playing instruments, to name some). However, the understanding of why human hands can perform such fascinating tasks still eludes complete comprehension. Since at least the end of the sixteenth century, scientists and engineers have tried to match the sensory and motor functions of the human hand. As a result, many contemporary humanoid and anthropomorphic robot hands have been developed to closely replicate the appearance and dexterity of human hands, in many cases using sophisticated designs that integrate multiple sensors and actuators---which make them prone to error and difficult to operate and control, particularly under uncertainty. In recent years, several simplification approaches and solutions have been proposed to develop more effective and reliable dexterous robot hands. These techniques, which have been based on using underactuated mechanical designs, kinematic synergies, or compliant materials, to name some, have opened up new ways to integrate hardware enhancements to facilitate grasping and dexterous manipulation control and improve reliability and robustness. Following this line of thought, this thesis studies four robot hand hardware aspects for enhancing grasping and manipulation, with a particular focus on dexterous in-hand manipulation. Namely: i) the use of passive soft fingertips; ii) the use of rigid and soft active surfaces in robot fingers; iii) the use of robot hand topologies to create particular in-hand manipulation trajectories; and iv) the decoupling of grasping and in-hand manipulation by introducing a reconfigurable palm. In summary, the findings from this thesis provide important notions for understanding the significance of mechanical and hardware elements in the performance and control of human manipulation. These findings show great potential in developing robust, easily programmable, and economically viable robot hands capable of performing dexterous manipulations under uncertainty, while exhibiting a valuable subset of functions of the human hand.Open Acces

ARMin: a robot for patient-cooperative arm therapy

Task-oriented, repetitive and intensive arm training can enhance arm rehabilitation in patients with paralyzed upper extremities due to lesions of the central nervous system. There is evidence that the training duration is a key factor for the therapy progress. Robot-supported therapy can improve the rehabilitation allowing more intensive training. This paper presents the kinematics, the control and the therapy modes of the arm therapy robot ARMin. It is a haptic display with semi-exoskeleton kinematics with four active and two passive degrees of freedom. Equipped with position, force and torque sensors the device can deliver patient-cooperative arm therapy taking into account the activity of the patient and supporting him/her only as much as needed. The haptic display is combined with an audiovisual display that is used to present the movement and the movement task to the patient. It is assumed that the patient-cooperative therapy approach combined with a multimodal display can increase the patient's motivation and activity and, therefore, the therapeutic progres