Designing Prosthetic Hands With Embodied Intelligence: The KIT Prosthetic Hands

Hand prostheses should provide functional replacements of lost hands. Yet current prosthetic hands often are not intuitive to control and easy to use by amputees. Commercially available prostheses are usually controlled based on EMG signals triggered by the user to perform grasping tasks. Such EMG-based control requires long training and depends heavily on the robustness of the EMG signals. Our goal is to develop prosthetic hands with semi-autonomous grasping abilities that lead to more intuitive control by the user. In this paper, we present the development of prosthetic hands that enable such abilities as first results toward this goal. The developed prostheses provide intelligent mechatronics including adaptive actuation, multi-modal sensing and on-board computing resources to enable autonomous and intuitive control. The hands are scalable in size and based on an underactuated mechanism which allows the adaptation of grasps to the shape of arbitrary objects. They integrate a multi-modal sensor system including a camera and in the newest version a distance sensor and IMU. A resource-aware embedded system for in-hand processing of sensory data and control is included in the palm of each hand. We describe the design of the new version of the hands, the female hand prosthesis with a weight of 377 g, a grasping force of 40.5 N and closing time of 0.73 s. We evaluate the mechatronics of the hand, its grasping abilities based on the YCB Gripper Assessment Protocol as well as a task-oriented protocol for assessing the hand performance in activities of daily living. Further, we exemplarily show the suitability of the multi-modal sensor system for sensory-based, semi-autonomous grasping in daily life activities. The evaluation demonstrates the merit of the hand concept, its sensor and in-hand computing systems

Robotic Manipulation of Environmentally Constrained Objects Using Underactuated Hands

Robotics for agriculture represents the ultimate application of one of our society\u27s latest and most advanced innovations to its most ancient and vital industry. Over the course of history, mechanization and automation have increased crop output several orders of magnitude, enabling a geometric growth in population and an increase in quality of life across the globe. As a challenging step, manipulating objects in harvesting automation is still under investigation in literature. Harvesting or the process of gathering ripe crops can be described as breaking environmentally constrained objects into two or more pieces at the desired locations. In this thesis, the problem of purposefully failing (breaking) or yielding objects by a robotic gripper is investigated. A failure task is first formulated using mechanical failure theories. Next, a grasp quality measure is presented to characterize a suitable grasp configuration and systematically control the failure behavior of the object. This approach combines the failure task and the capability of the gripper for wrench insertion. The friction between the object and the gripper is used to formulate the capability of the gripper for wrench insertion. A new method inspired by the human pre-manipulation process is introduced to utilize the gripper itself as the measurement tool and obtain a friction model. The developed friction model is capable of capturing the anisotropic behavior of materials which is the case for most fruits and vegetables.The limited operating space for harvesting process, the vulnerability of agricultural products and clusters of crops demand strict conditions for the manipulation process. This thesis presents a new sensorized underactuated self-adaptive finger to address the stringent conditions in the agricultural environment. This design incorporates link-driven underactuated mechanism with an embedded load cell for contact force measurement and a trimmer potentiometer for acquiring joint variables. The integration of these sensors results in tactile-like sensations in the finger without compromising the size and complexity of the proposed design. To obtain an optimum finger design, the placement of the load cell is analyzed using Finite Element Method (FEM). The design of the finger features a particular round shape of the distal phalanx and specific size ratio between the phalanxes to enable both precision and power grasps. A quantitative evaluation of the grasp efficiency by constructing a grasp wrench space is also provided. The effectiveness of the proposed designs and theories are verified through real-time experiments. For conducting the experiments in real-time, a software/hardware platform capable of dataset management is crucial. In this thesis, a new comprehensive software interface for integration of industrial robots with peripheral tools and sensors is designed and developed. This software provides a real-time low-level access to the manipulator controller. Furthermore, Data Acquisition boards are integrated into the software which enables Rapid Prototyping methods. Additionally, Hardware-in-the-loop techniques can be implemented by adding the complexity of the plant under control to the test platform. The software is a collection of features developed and distributed under GPL V3.0

Signal and Information Processing Methods for Embedded Robotic Tactile Sensing Systems

The human skin has several sensors with different properties and responses that are able to detect stimuli resulting from mechanical stimulations. Pressure sensors are the most important type of receptors for the exploration and manipulation of objects. In the last decades, smart tactile sensing based on different sensing techniques have been developed as their application in robotics and prosthetics is considered of huge interest, mainly driven by the prospect of autonomous and intelligent robots that can interact with the environment. However, regarding object properties estimation on robots, hardness detection is still a major limitation due to the lack of techniques to estimate it. Furthermore, finding processing methods that can interpret the measured information from multiple sensors and extract relevant information is a Challenging task. Moreover, embedding processing methods and machine learning algorithms in robotic applications to extract meaningful information such as object properties from tactile data is an ongoing challenge, which is controlled by the device constraints (power constraint, memory constraints, etc.), the computational complexity of the processing and machine learning algorithms, the application requirements (real-time operations, high prediction performance). In this dissertation, we focus on the design and implementation of pre-processing methods and machine learning algorithms to handle the aforementioned challenges for a tactile sensing system in robotic application. First, we propose a tactile sensing system for robotic application. Then we present efficient preprocessing and feature extraction methods for our tactile sensors. Then we propose a learning strategy to reduce the computational cost of our processing unit in object classification using sensorized Baxter robot. Finally, we present a real-time robotic tactile sensing system for hardness classification on a resource-constrained devices. The first study represents a further assessment of the sensing system that is based on the PVDF sensors and the interface electronics developed in our lab. In particular, first, it presents the development of a skin patch (multilayer structure) that allows us to use the sensors in several applications such as robotic hand/grippers. Second, it shows the characterization of the developed skin patch. Third, it validates the sensing system. Moreover, we designed a filter to remove noise and detect touch. The experimental assessment demonstrated that the developed skin patch and the interface electronics indeed can detect different touch patterns and stimulus waveforms. Moreover, the results of the experiments defined the frequency range of interest and the response of the system to realistic interactions with the sensing system to grasp and release events. In the next study, we presented an easy integration of our tactile sensing system into Baxter gripper. Computationally efficient pre-processing techniques were designed to filter the signal and extract relevant information from multiple sensor signals, in addition to feature extraction methods. These processing methods aim in turn to reduce also the computational complexity of machine learning algorithms utilized for object classification. The proposed system and processing strategy were evaluated on object classification application by integrating our system into the gripper and we collected data by grasping multiple objects. We further proposed a learning strategy to accomplish a trade-off between the generalization accuracy and the computational cost of the whole processing unit. The proposed pre-processing and feature extraction techniques together with the learning strategy have led to models with extremely low complexity and very high generalization accuracy. Moreover, the support vector machine achieved the best trade-off between accuracy and computational cost on tactile data from our sensors. Finally, we presented the development and implementation on the edge of a real–time tactile sensing system for hardness classification on Baxter robot based on machine and deep learning algorithms. We developed and implemented in plain C a set of functions that provide the fundamental layer functionalities of the Machine learning and Deep Learning models (ML and DL), along with the pre–processing methods to extract the features and normalize the data. The models can be deployed to any device that supports C code since it does not rely on any of the existing libraries. Shallow ML/DL algorithms for the deployment on resource–constrained devices are designed. To evaluate our work, we collected data by grasping objects of different hardness and shape. Two classification problems were addressed: 5 levels of hardness classified on the same objects’ shape, and 5 levels of hardness classified on two different objects’ shape. Furthermore, optimization techniques were employed. The models and pre–processing were implemented on a resource constrained device, where we assessed the performance of the system in terms of accuracy, memory footprint, time latency, and energy consumption. We achieved for both classification problems a real-time inference (< 0.08 ms), low power consumption (i.e., 3.35 μJ), extremely small models (i.e., 1576 Byte), and high accuracy (above 98%)

On the development of a cybernetic prosthetic hand

The human hand is the end organ of the upper limb, which in humans serves the important function of prehension, as well as being an important organ for sensation and communication. It is a marvellous example of how a complex mechanism can be implemented, capable of realizing very complex and useful tasks using a very effective combination of mechanisms, sensing, actuation and control functions. In this thesis, the road towards the realization of a cybernetic hand has been presented. After a detailed analysis of the model, the human hand, a deep review of the state of the art of artificial hands has been carried out. In particular, the performance of prosthetic hands used in clinical practice has been compared with the research prototypes, both for prosthetic and for robotic applications. By following a biomechatronic approach, i.e. by comparing the characteristics of these hands with the natural model, the human hand, the limitations of current artificial devices will be put in evidence, thus outlining the design goals for a new cybernetic device. Three hand prototypes with a high number of degrees of freedom have been realized and tested: the first one uses microactuators embedded inside the structure of the fingers, and the second and third prototypes exploit the concept of microactuation in order to increase the dexterity of the hand while maintaining the simplicity for the control. In particular, a framework for the definition and realization of the closed-loop electromyographic control of these devices has been presented and implemented. The results were quite promising, putting in evidence that, in the future, there could be two different approaches for the realization of artificial devices. On one side there could be the EMG-controlled hands, with compliant fingers but only one active degree of freedom. On the other side, more performing artificial hands could be directly interfaced with the peripheral nervous system, thus establishing a bi-directional communication with the human brain

Force/position-based modular system for minimally invasive surgery,”

Abstract-The limitations of minimally invasive surgery include the inability to sense forces exerted by the instruments on tissue and the limited visual cues available through the endoscope. A modular laparoscopic instrument capable of measuring force and position has been designed to address these limitations. Novel image-based position tracking software has been developed and integrated within a graphical user interface. This modular system is low cost, versatile, and could be used for training, localization of critical features or for guidance during surgical procedures

Validity and usability of a smart ball–driven serious game to monitor grip strength in independent elderlies

Telemonitoring is one of the most expedient answers to the strong need for preventive care imposed by the rapidly aging society. We propose an innovative solution to the detection of early signs of frailty by presenting a serious game controlled by a smart sensorized soft plastic ball, designed to achieve continuous home-based monitoring of muscle weakness in older adults. Design, development, and testing of the smart ball and of the game interface devised to guide the monitoring procedure are presented. Reliability and concurrent validity of the system in measuring maximal grip strength against the clinical standard Jamar\uae were evaluated. Serious game usability and acceptance were investigated on 26 elderlies. Smart ball and Jamar measurements were well correlated (0.76 and 0.80 for dominant and non-dominant hands) and test\u2013retest reliability of pressure measurements was excellent (intraclass correlation coefficient >0.94). The serious game was well accepted by the 96.1 percent of participants, who provided a strongly positive usability score (87.7/100). The smart ball\u2013driven serious game demonstrated excellent reliability and good validity in measuring grip strength. The proposed smart ball\u2013driven serious game can be used for home self-monitoring of grip strength in elderlies

Robotic Grippers for Large and Soft Object Manipulation

Grasping has always been considered a key domain of cyber-physical systems, through which action physical interaction can be achieved. This paper presents a systematic review of the state-of-the-art robotic soft object gripping solutions aimed for the food-industry, focusing on red meat handling. A categorized analysis about the currently used grippers is provided, that could be used or adapted to robotic meat-processing. The paper enlists various solutions and gripping principles for low-payload applications too, although the emphasis is on the classic shapelocking and force-locking grippers that are potentially capable of grasping and manipulating heavier specimens. The purpose of the scientific literature survey is mainly to identify exceptional and/or remarkable gripper-designs, or completely new gripping concepts, while the patent research presents complete, commercially available solutions

ReHand - a portable assistive rehabilitation hand exoskeleton

This dissertation presents a synthesis of a novel underactuated exoskeleton (namely ReHand2) thought and designed for a task-oriented rehabilitation and/or for empower the human hand. The first part of this dissertation shows the current context about the robotic rehabilitation with a focus on hand pathologies, which influence the hand capability. The chapter is concluded with the presentation of ReHand2. The second chapter describes the human hand biomechanics. Starting from the definition of human hand anatomy, passing through anthropometric data, to taxonomy on hand grasps and finger constraints, both from static and dynamic point of view. In addition, some information about the hand capability are given. The third chapter analyze the current state of the art in hand exoskeleton for rehabilitation and empower tasks. In particular, the chapter presents exoskeleton technologies, from mechanisms to sensors, passing though transmission and actuators. Finally, the current state of the art in terms of prototype and commercial products is presented. The fourth chapter introduces the concepts of underactuation with the basic explanation and the classical notation used typically in the prosthetic field. In addition, the chapter describe also the most used differential elements in the prosthetic, follow by a statical analysis. Moreover typical transmission tree at inter-finger level as well as the intra- finger underactuation are explained . The fifth chapter presents the prototype called ReHand summarizing the device description and explanation of the working principle. It describes also the kinetostatic analysis for both, inter- and the intra-finger modules. in the last section preliminary results obtained with the exoskeleton are shown and discussed, attention is pointed out on prototype’s problems that have carry out at the second version of the device. The sixth chapter describes the evolution of ReHand, describing the kinematics and dynamics behaviors. In particular, for the mathematical description is introduced the notation used in order to analyze and optimize the geometry of the entire device. The introduced model is also implemented in Matlab Simulink environment. Finally, the chapter presents the new features. The seventh chapter describes the test bench and the methodologies used to evaluate the device statical, and dynamical performances. The chapter presents and discuss the experimental results and compare them with simulated one. Finally in the last chapter the conclusion about the ReHand project are proposed as well as the future development. In particular, the idea to test de device in relevant environments. In addition some preliminary considerations about the thumb and the wrist are introduced, exploiting the possibility to modify the entire layout of the device, for instance changing the actuator location