Improving Fine Control of Grasping Force during Hand–Object Interactions for a Soft Synergy-Inspired Myoelectric Prosthetic Hand

abstract: The concept of postural synergies of the human hand has been shown to potentially reduce complexity in the neuromuscular control of grasping. By merging this concept with soft robotics approaches, a multi degrees of freedom soft-synergy prosthetic hand [SoftHand-Pro (SHP)] was created. The mechanical innovation of the SHP enables adaptive and robust functional grasps with simple and intuitive myoelectric control from only two surface electromyogram (sEMG) channels. However, the current myoelectric controller has very limited capability for fine control of grasp forces. We addressed this challenge by designing a hybrid-gain myoelectric controller that switches control gains based on the sensorimotor state of the SHP. This controller was tested against a conventional single-gain (SG) controller, as well as against native hand in able-bodied subjects. We used the following tasks to evaluate the performance of grasp force control: (1) pick and place objects with different size, weight, and fragility levels using power or precision grasp and (2) squeezing objects with different stiffness. Sensory feedback of the grasp forces was provided to the user through a non-invasive, mechanotactile haptic feedback device mounted on the upper arm. We demonstrated that the novel hybrid controller enabled superior task completion speed and fine force control over SG controller in object pick-and-place tasks. We also found that the performance of the hybrid controller qualitatively agrees with the performance of native human hands.View the article as published at https://www.frontiersin.org/articles/10.3389/fnbot.2017.00071/ful

Reflex: A Closed-Loop Tactile Feedback System for Use in Upper Limb Prosthesis Grip Control

Tactile sensing provides valuable insight to the environment in which we interact with. Upper limb amputees lack the sensations that generates the necessary information to stably grasp the wide variety of objects we interact with on a daily basis. Utilizing tactile sensing to provide feedback to a prosthetic hand provides a mechanism for replacing the grip control functionality of the mechanoreceptors found in human skin. Novel customizable, low cost tactile sensors for monitoring the dynamics of an object grasped by a prosthetic hand are developed and presented as part of this thesis. The response of sensors placed on a prosthetic hand provides information regarding the state of a grasped object, particularly contact and slip. The sensors are made up of various textile materials, including stretchable interfacing layers and conductive traces. Essentially a force sensitive resistor, each sensor is shaped into stretchable cu ff that can be placed around the finger of a prosthetic hand. An outer rubber layer on the sensor provides compliance, which is found to enhance grasping performance with a prosthesis. Two control algorithms were developed as part of the closed-loop tactile feedback system, called Reflex, to enhance grasping functionality with a prosthesis. A Contact Detection strategy uses force information to effectively reduce the user's electromyography (EMG) signals, which are used to control the prosthesis. Essentially, the goal of this strategy is to help a user grab fragile objects without breaking them. A second strategy, Slip Prevention, uses the derivative of a force signal to detect slip of a grasped object. Instances of slip trigger electrical pulses sent from the prosthesis control unit to close the hand in an effort to prevent additional slip. The Reflex system, comprised of two control strategies along with flexible textile based force sensors on the fingers of a prosthesis, was shown to improve the grasping functionality of a prosthesis under normal use conditions. Able body participants were used to test the system. Results show the sensors' ability to greatly enhance grasping fragile objects while also helping prevent object slip. The compliant nature of the sensors enables users to more confidently pick up and move small,fragile objects, such as foam peanuts and crackers. Without sensors and tactile feedback, users had a higher likelihood of breaking objects while grabbing them. The addition of sensors reduced this failure rate, and the failure rate was reduced even further with the implementation of control algorithms running in real-time. The slip prevention strategy was also shown to help reduce the amount of object movement after a grasp is initiated, although the most benefit comes from the compliant nature of the sensors. Reflex is the first closed-loop tactile feedback system with multiple control strategies that can be used on a prosthetic hand to enhance grasping functionality. The system allows one to switch between Contact Detection or Slip Prevention control strategies, giving the user the ability to use each control as needed. Feedback from the textile sensors directly to the prosthesis control unit provides valuable information regarding grasping forces. This research aims to help improve prosthetic technology so that one day amputees will feel as if their device is a natural extension of their body

Methods and Sensors for Slip Detection in Robotics: A Survey

The perception of slip is one of the distinctive abilities of human tactile sensing. The sense of touch allows recognizing a wide set of properties of a grasped object, such as shape, weight and dimension. Based on such properties, the applied force can be accordingly regulated avoiding slip of the grasped object. Despite the great importance of tactile sensing for humans, mechatronic hands (robotic manipulators, prosthetic hands etc.) are rarely endowed with tactile feedback. The necessity to grasp objects relying on robust slip prevention algorithms is not yet corresponded in existing artificial manipulators, which are relegated to structured environments then. Numerous approaches regarding the problem of slip detection and correction have been developed especially in the last decade, resorting to a number of sensor typologies. However, no impact on the industrial market has been achieved. This paper reviews the sensors and methods so far proposed for slip prevention in artificial tactile perception, starting from more classical techniques until the latest solutions tested on robotic systems. The strengths and weaknesses of each described technique are discussed, also in relation to the sensing technologies employed. The result is a summary exploring the whole state of art and providing a perspective towards the future research directions in the sector

A Methodology Towards Comprehensive Evaluation of Shape Memory Alloy Actuators for Prosthetic Finger Design

Presently, DC motors are the actuator of choice within intelligent upper limb prostheses. However, the weight and dimensions associated with suitable DC motors are not always compatible with the geometric restrictions of a prosthetic hand; reducing available degrees of freedom and ultimately rendering the prosthesis uncomfortable for the end-user. As a result, the search is on-going to find a more appropriate actuation solution that is lightweight, noiseless, strong and cheap. Shape memory alloy (SMA) actuators offer the potential to meet these requirements. To date, no viable upper limb prosthesis using SMA actuators has been developed. The primary reasons lie in low force generation as a result of unsuitable actuator designs, and significant difficulties in control owing to the highly nonlinear response of SMAs when subjected to joule heating. This work presents a novel and comprehensive methodology to facilitate evaluation of SMA bundle actuators for prosthetic finger design. SMA bundle actuators feature multiple SMA wires in parallel. This allows for increased force generation without compromising on dynamic performance. The SMA bundle actuator is tasked with reproducing the typical forces and contractions associated with the human finger in a prosthetic finger design, whilst maintaining a high degree of energy efficiency. A novel approach to SMA control is employed, whereby an adaptive controller is developed and tuned using the underlying thermo-mechanical principles of operation of SMA wires. A mathematical simulation of the kinematics and dynamics of motion provides a platform for designing, optimizing and evaluating suitable SMA bundle actuators offline. This significantly reduces the time and cost involved in implementing an appropriate actuation solution. Experimental results show iii that the performance of SMA bundle actuators is favourable for prosthesis applications. Phalangeal tip forces are shown to improve significantly through bundling of SMA wire actuators, while dynamic performance is maintained owing to the design and implementation of the selected control strategy. The work is intended to serve as a roadmap for fellow researchers seeking to design, implement and control SMA bundle actuators in a prosthesis design. Furthermore, the methodology can also be adopted to serve as a guide in the evaluation of other non-conventional actuation technologies in alternative applications

Adaptive robust interaction control for low-cost robotic grasping

Robotic grasping is a challenging area in the field of robotics. When a gripper starts interacting with an object to perform a grasp, the mechanical properties of the object (stiffness and damping) will play an important role. A gripper which is stable in isolated conditions, can become unstable when coupled to an object. This can lead to the extreme condition where the gripper becomes unstable and generates excessive or insufficient grip force resulting in the grasped object either being crushed, or falling and breaking. In addition to the stability issue, grasp maintenance is one of the most important requirements of any grasp where it guarantees a secure grasp in the presence of any unknown disturbance. The term grasp maintenance refers to the reaction of the controller in the presence of external disturbances, trying to prevent any undesired slippage. To do so, the controller continuously adjusts the grip force. This is a challenging task as it requires an accurate model of the friction and object’s weight to estimate a sufficient grip force to stop the object from slipping while incurring minimum deformation. Unfortunately, in reality, there is no solution which is able to obtain the mechanical properties, frictional coefficient and weight of an object before establishing a mechanical interaction with it. External disturbance forces are also stochastic meaning they are impossible to predict. This thesis addresses both of the problems mentioned above by:Creating a novel variable stiffness gripper, capable of grasping unknown objects, mainly those found in agricultural or food manufacturing companies. In addition to the stabilisation effect of the introduced variable stiffness mechanism, a novel force control algorithm has been designed that passively controls the grip force in variable stiffness grippers. Due to the passive nature of the suggested controller, it completely eliminates the necessity for any force sensor. The combination of both the proposed variable stiffness gripper and the passivity based control provides a unique solution for the stable grasp and force control problem in tendon driven, angular grippers.Introducing a novel active multi input-multi output slip prevention algorithm. The algorithm developed provides a robust control solution to endow direct drive parallel jaw grippers with the capability to stop held objects from slipping while incurring minimum deformation; this can be done without any prior knowledge of the object’s friction and weight. The large number of experiments provided in this thesis demonstrate the robustness of the proposed controller when controlling parallel jaw grippers in order to quickly grip, lift and place a broad range of objects firmly without dropping or crushing them. This is particularly useful for teleoperation and nuclear decommissioning tasks where there is often no accurate information available about the objects to be handled. This can mean that pre-programming of the gripper is required for each different object and for high numbers of objects this is impractical and overly time-consuming. A robust controller, which is able to compensate for any uncertainties regarding the object model and any unknown external disturbances during grasping, is implemented. This work has advanced the state of the art in the following two main areas: Direct impedance modulation for stable grasping in tendon driven, angular grippers. Active MIMO slip prevention grasp control for direct drive parallel jaw grippers

Performance of modified jatropha oil in combination with hexagonal boron nitride particles as a bio-based lubricant for green machining

This study evaluates the machining performance of newly developed modified jatropha oils (MJO1, MJO3 and MJO5), both with and without hexagonal boron nitride (hBN) particles (ranging between 0.05 and 0.5 wt%) during turning of AISI 1045 using minimum quantity lubrication (MQL). The experimental results indicated that, viscosity improved with the increase in MJOs molar ratio and hBN concentration. Excellent tribological behaviours is found to correlated with a better machining performance were achieved by MJO5a with 0.05 wt%. The MJO5a sample showed the lowest values of cutting force, cutting temperature and surface roughness, with a prolonged tool life and less tool wear, qualifying itself to be a potential alternative to the synthetic ester, with regard to the environmental concern

Nonlinear Control Synthesis for Facilitation of Human-Robot Interaction

Human-robot interaction is an area of interest that is becoming increasingly important in robotics research. Nonlinear control design techniques allow researchers to guarantee stability, performance, as well as safety, especially in cases involving physical human-robot interaction (PHRI). In this dissertation, we will propose two different nonlinear controllers and detail the design of an assistive robotic system to facilitate human-robot interaction. In Chapter 2, to facilitate physical human-robot interaction, the problem of making a safe compliant contact between a human and an assistive robot is considered. Users with disabilities have a need to utilize their assistive robots for physical interaction during activities such as hair-grooming, scratching, face-sponging, etc. Specifically, we propose a hybrid force/velocity/attitude control for our physical human-robot interaction system which is based on measurements from a force/torque sensor mounted on the robot wrist. While automatically aligning the end-effector surface with the unknown environmental (human) surface, a desired commanded force is applied in the normal direction while following desired velocity commands in the tangential directions. A Lyapunov based stability analysis is provided to prove both convergence as well as passivity of the interaction to ensure both performance and safety. Simulation as well as experimental results verify the performance and robustness of the proposed hybrid force/velocity/attitude controller in the presence of dynamic uncertainties as well as safety compliance of human-robot interactions for a redundant robot manipulator. Chapter 3 presents the design, analysis, and experimental implementation of an adaptive control enabled intelligent algorithm to facilitate 1-click grasping of novel objects by a robotic gripper since one of the most common types of tasks for an assistive robot is pick and place/object retrieval tasks. But there are a variety of objects in our daily life all of which need different optimal force to grasp them. This algorithm facilitates automated grasping force adjustment. The use of object-geometry free modeling coupled with utilization of interaction force and slip velocity measurements allows for the design of an adaptive backstepping controller that is shown to be asymptotically stable via a Lyapunov-based analysis. Experiments with multiple objects using a prototype gripper with embedded sensing show that the proposed scheme is able to effectively immobilize novel objects within the gripper fingers. Furthermore, it is seen that the adaptation allows for close estimation of the minimum grasp force required for safe grasping which results in minimal deformation of the grasped object. In Chapter 4, we present the design and implementation of the motion controller and adaptive interface for the second generation of the UCF-MANUS intelligent assistive robotic manipulator system. Based on usability testing for the system, several features were implemented in the interface that could reduce the complexity of the human-robot interaction while also compensating for the deficits in different human factors, such as Working Memory, Response Inhibition, Processing Speed; , Depth Perception, Spatial Ability, Contrast Sensitivity. For the controller part, we designed several new features to provide the user has a less complex and safer interaction with the robot, such as \u27One-click mode\u27, \u27Move suggestion mode\u27 and \u27Gripper Control Assistant\u27. As for the adaptive interface design, we designed and implemented compensators such as \u27Contrast Enhancement\u27, \u27Object Proximity Velocity Reduction\u27 and \u27Orientation Indicator\u27

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

A Biomimetic Approach to Controlling Restorative Robotics

Movement is the only way a person can interact with the world around them. When trauma to the neuromuscular systems disrupts the control of movement, quality of life suffers. To restore limb functionality, active robotic interventions and/or rehabilitation are required. Unfortunately, the primary obstacle in a person’s recovery is the limited robustness of the human-machine interfaces. Current systems rely on control approaches that rely on the person to learn how the system works instead of the system being more intuitive and working with the person naturally. My research goal is to design intuitive control mechanisms based on biological processes termed the biomimetic approach. I have applied this control scheme to problems with restorative robotics focused on the upper and lower limb control. Operating an advanced active prosthetic hand is a two-pronged problem of actuating a high-dimensional mechanism and controlling it with an intuitive interface. Our approach attempts to solve these problems by going from muscle activity, electromyography (EMG), to limb kinematics calculated through dynamic simulation of a musculoskeletal model. This control is more intuitive to the user because they attempt to move their hand naturally, and the prosthetic hand performs that movement. The key to this approach was validating simulated muscle paths using both experimental measurements and anatomical constraints where data is missing. After the validation, simulated muscle paths and forces are used to decipher the intended movement. After we have calculated the intended movement, we can move a prosthetic hand to match. This approach required minimal training to give an amputee the ability to control prosthetic hand movements, such as grasping. A more intuitive controller has the potential to improve how people interact and use their prosthetic hands. Similarly, the rehabilitation of the locomotor system in people with damaged motor pathways or missing limbs require appropriate interventions. The problem of decoding human motor intent in a treadmill walking task can be solved with a biomimetic approach. Estimated limb speed is essential for this approach according to the theoretical input-output computation performed by spinal central pattern generators (CPGs), which represents neural circuitry responsible for autonomous control of stepping. The system used the locomotor phases, swing and stance, to estimate leg speeds and enable self-paced walking as well as steering in virtual reality with congruent visual flow. The unique advantage of this system over the previous state-of-art is the independent leg speed control, which is required for multidirectional movement in VR. This system has the potential to contribute to VR gait rehab techniques. Creating biologically-inspired controllers has the potential to improve restorative robotics and allow people a better opportunity to recover lost functionality post-injury. Movement is the only way a person can interact with the world around them. When trauma to the neuromuscular systems disrupts the control of movement, quality of life suffers. To restore limb functionality, active robotic interventions and/or rehabilitation are required. Unfortunately, the primary obstacle in a person’s recovery is the limited robustness of the human-machine interfaces. Current systems rely on control approaches that rely on the person to learn how the system works instead of the system being more intuitive and working with the person naturally. My research goal is to design intuitive control mechanisms based on biological processes termed the biomimetic approach. I have applied this control scheme to problems with restorative robotics focused on the upper and lower limb control.Operating an advanced active prosthetic hand is a two-pronged problem of actuating a high-dimensional mechanism and controlling it with an intuitive interface. Our approach attempts to solve these problems by going from muscle activity, electromyography (EMG), to limb kinematics calculated through dynamic simulation of a musculoskeletal model. This control is more intuitive to the user because they attempt to move their hand naturally, and the prosthetic hand performs that movement. The key to this approach was validating simulated muscle paths using both experimental measurements and anatomical constraints where data is missing. After the validation, simulated muscle paths and forces are used to decipher the intended movement. After we have calculated the intended movement, we can move a prosthetic hand to match. This approach required minimal training to give an amputee the ability to control prosthetic hand movements, such as grasping. A more intuitive controller has the potential to improve how people interact and use their prosthetic hands.Similarly, the rehabilitation of the locomotor system in people with damaged motor pathways or missing limbs require appropriate interventions. The problem of decoding human motor intent in a treadmill walking task can be solved with a biomimetic approach. Estimated limb speed is essential for this approach according to the theoretical input-output computation performed by spinal central pattern generators (CPGs), which represents neural circuitry responsible for autonomous control of stepping. The system used the locomotor phases, swing and stance, to estimate leg speeds and enable self-paced walking as well as steering in virtual reality with congruent visual flow. The unique advantage of this system over the previous state-of-art is the independent leg speed control, which is required for multidirectional movement in VR. This system has the potential to contribute to VR gait rehab techniques.Creating biologically-inspired controllers has the potential to improve restorative robotics and allow people a better opportunity to recover lost functionality post-injury