Hierarchical tactile sensation integration from prosthetic fingertips enables multi-texture surface recognition\u3csup\u3e†\u3c/sup\u3e

Multifunctional flexible tactile sensors could be useful to improve the control of prosthetic hands. To that end, highly stretchable liquid metal tactile sensors (LMS) were designed, manufactured via photolithography, and incorporated into the fingertips of a prosthetic hand. Three novel contributions were made with the LMS. First, individual fingertips were used to distinguish between different speeds of sliding contact with different surfaces. Second, differences in surface textures were reliably detected during sliding contact. Third, the capacity for hierarchical tactile sensor integration was demonstrated by using four LMS signals simultaneously to distinguish between ten complex multi-textured surfaces. Four different machine learning algorithms were compared for their successful classification capabilities: K-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and neural network (NN). The time-frequency features of the LMSs were extracted to train and test the machine learning algorithms. The NN generally performed the best at the speed and texture detection with a single finger and had a 99.2 ± 0.8% accuracy to distinguish between ten different multi-textured surfaces using four LMSs from four fingers simultaneously. The capability for hierarchical multi-finger tactile sensation integration could be useful to provide a higher level of intelligence for artificial hands

Increasing Transparency and Presence of Teleoperation Systems Through Human-Centered Design

Teleoperation allows a human to control a robot to perform dexterous tasks in remote, dangerous, or unreachable environments. A perfect teleoperation system would enable the operator to complete such tasks at least as easily as if he or she was to complete them by hand. This ideal teleoperator must be perceptually transparent, meaning that the interface appears to be nearly nonexistent to the operator, allowing him or her to focus solely on the task environment, rather than on the teleoperation system itself. Furthermore, the ideal teleoperation system must give the operator a high sense of presence, meaning that the operator feels as though he or she is physically immersed in the remote task environment. This dissertation seeks to improve the transparency and presence of robot-arm-based teleoperation systems through a human-centered design approach, specifically by leveraging scientific knowledge about the human motor and sensory systems. First, this dissertation aims to improve the forward (efferent) teleoperation control channel, which carries information from the human operator to the robot. The traditional method of calculating the desired position of the robot\u27s hand simply scales the measured position of the human\u27s hand. This commonly used motion mapping erroneously assumes that the human\u27s produced motion identically matches his or her intended movement. Given that humans make systematic directional errors when moving the hand under conditions similar to those imposed by teleoperation, I propose a new paradigm of data-driven human-robot motion mappings for teleoperation. The mappings are determined by having the human operator mimic the target robot as it autonomously moves its arm through a variety of trajectories in the horizontal plane. Three data-driven motion mapping models are described and evaluated for their ability to correct for the systematic motion errors made in the mimicking task. Individually-fit and population-fit versions of the most promising motion mapping model are then tested in a teleoperation system that allows the operator to control a virtual robot. Results of a user study involving nine subjects indicate that the newly developed motion mapping model significantly increases the transparency of the teleoperation system. Second, this dissertation seeks to improve the feedback (afferent) teleoperation control channel, which carries information from the robot to the human operator. We aim to improve a teleoperation system a teleoperation system by providing the operator with multiple novel modalities of haptic (touch-based) feedback. We describe the design and control of a wearable haptic device that provides kinesthetic grip-force feedback through a geared DC motor and tactile fingertip-contact-and-pressure and high-frequency acceleration feedback through a pair of voice-coil actuators mounted at the tips of the thumb and index finger. Each included haptic feedback modality is known to be fundamental to direct task completion and can be implemented without great cost or complexity. A user study involving thirty subjects investigated how these three modalities of haptic feedback affect an operator\u27s ability to control a real remote robot in a teleoperated pick-and-place task. This study\u27s results strongly support the utility of grip-force and high-frequency acceleration feedback in teleoperation systems and show more mixed effects of fingertip-contact-and-pressure feedback

Ground Robotic Hand Applications for the Space Program study (GRASP)

This document reports on a NASA-STDP effort to address research interests of the NASA Kennedy Space Center (KSC) through a study entitled, Ground Robotic-Hand Applications for the Space Program (GRASP). The primary objective of the GRASP study was to identify beneficial applications of specialized end-effectors and robotic hand devices for automating any ground operations which are performed at the Kennedy Space Center. Thus, operations for expendable vehicles, the Space Shuttle and its components, and all payloads were included in the study. Typical benefits of automating operations, or augmenting human operators performing physical tasks, include: reduced costs; enhanced safety and reliability; and reduced processing turnaround time

Método para evaluación de una estrategia de control realimentado en la funcionalidad de agarre de poder con una prótesis de mano robótica

Objective: In the design of a robotic hand prosthesis, a variety of problems arise for which there are multiple solutions proposed in the literature. One of them is the selection of a strategy for the automatic control of the movement of the hand. Within the functionalities of a prosthesis, the power grip functionality is one of the most important. Given the wide variety of proposals in terms of control techniques for power grip, the designer is faced with the dilemma of which of them is the most suitable for his particular design. Although there are metrics to quantify the quality of the grip, those that have been proposed address various aspects independently, which does not allow making a decision about the best option for the particular case. In this work, a method is proposed to calculate a composite indicator that allows the evaluation of the performance of control techniques in the power grip based on the following individual metrics that evaluate different aspects in the execution of the power grip: finger strength, grip strength, grip efficiency, grip cycle time, slip resistance, following error, and transient response overshoot. Thanks to the method proposed in this work, the question is answered: of several control options to be tested, which one offers better functional performance? Methodology: A set of metrics are adopted with which to obtain quantitative data related to the quality of the power grip and to the performance of the control target to govern the prosthesis; subsequently, selected individual metrics are calculated for each of the grip control techniques to be evaluated on a virtual environment, consisting of a robotic hand and an object to be grasped; then two composite indicators are constructed to obtain a quantitative assessment of the quality of the grip based on a statistical analysis, and the results are contrasted against the individual metrics used. Results: A method is proposed for the construction of a composite indicator, which allows the evaluation of the performance of control techniques, in power grips in robotic hands. By implementing this method, the best performance values were found in hybrid controllers. Conclusions: In this work, a method has been proposed to facilitate the decision-making of the designer regarding the most adequate control technique, among several available, to achieve the power grip with a specific prosthesis. The method seeks to build a composite indicator that groups together various metrics to evaluate particular grip functionality, and also to quantify the achievement of following instructions, facilitating decision-making about the incidence of the control technique in the achievement of the final objective. Financing: This work was supported by University of Cauca under the Master of Science in Automation program.Objetivo: En el diseño de una prótesis de mano robótica surgen variedad de problemáticas para las cuales existen múltiples soluciones propuestas en la literatura. Una de las problemáticas es la selección de una estrategia para el control automático del movimiento de la mano. Entre las funcionalidades de una prótesis, la de agarre de poder es una de las más importantes. Dada la gran cantidad de propuestas en cuanto técnicas de control para el agarre de poder, el diseñador se enfrenta al dilema de cuál de ellas es la más adecuada para su diseño particular. Aunque hay métricas para cuantificar la calidad del agarre, las que se han propuesto abordan diversos aspectos de manera independiente, lo cual no permite tomar una decisión acerca de la mejor opción para el caso particular. En este trabajo se propone un método para calcular un indicador compuesto que permite la evaluación del desempeño de técnicas de control en el agarre de poder, con base en las siguientes métricas individuales que evalúan diferentes aspectos en la ejecución del agarre de poder: la fuerza del dedo, la fuerza del agarre, la eficiencia del agarre, el tiempo de ciclo de agarre, la resistencia al deslizamiento, el error de seguimiento y el sobreimpulso de la respuesta transitoria. Gracias al método propuesto en este trabajo, se da respuesta a la pregunta: “De varias opciones de control a probar, ¿cuál ofrece un mejor desempeño funcional?”. Metodología: Se adoptó un conjunto de métricas con las cuales obtener datos cuantitativos relacionadas con la calidad del agarre de poder y con el desempeño del objetivo de control para gobernar la prótesis. Posteriormente, se calcularon métricas individuales seleccionadas para cada una de las técnicas de control de agarre a evaluar, sobre un entorno virtual, constituido por una mano robótica y un objeto a agarrar. Luego se construyeron dos indicadores compuestos para obtener una valoración cuantitativa de la calidad del agarre a partir de un análisis estadístico, y se contrastaron los resultados contra las métricas individuales utilizadas. Resultados: Se planteó un método para la construcción de un indicador compuesto que permitiera la evaluación del desempeño de técnicas de control, en agarres de poder en manos robóticas. Al implementar dicho método, se encontraron los mejores valores de desempeño en controladores híbridos. Conclusiones: En este trabajo se sugiere una alternativa tendiente a facilitar la toma de decisiones del diseñador en cuanto a la técnica de control más adecuada, entre varias disponibles, para el logro del agarre de poder con una prótesis específica. El método busca construir un indicador compuesto que agrupa variadas métricas para evaluar funcionalidad particular de agarre, y también para cuantificar el logro de seguimiento de consignas, facilitando la toma de decisiones acerca de la incidencia de la técnica de control en el logro del objetivo final

The design and control of an actively restrained passive mechatronic system for safety-critical applications

Development of manipulators that interact closely with humans has been a focus of research in fields such as robot-assisted surgery and haptic interfaces for many years. Recent introduction of powered surgical-assistant devices into the operating theatre has meant that robot manipulators have been required to interact with both patients and surgeons. Most of these manipulators are modified industrial robots. However, the use of high-powered mechanisms in the operating theatre could compromise safety of the patient, surgeon, and operating room staff. As a solution to the safety problem, the use of actively restrained passive arms has been proposed. Clutches or brakes at each joint are used to restrict the motion of the end-effector to restrain it to a pre-defined region or path. However, these devices have only had limited success in following pre-defined paths under human guidance. In this research, three major limitations of existing passive devices actively restrained are addressed. [Continues.

Robotics 2010

Without a doubt, robotics has made an incredible progress over the last decades. The vision of developing, designing and creating technical systems that help humans to achieve hard and complex tasks, has intelligently led to an incredible variety of solutions. There are barely technical fields that could exhibit more interdisciplinary interconnections like robotics. This fact is generated by highly complex challenges imposed by robotic systems, especially the requirement on intelligent and autonomous operation. This book tries to give an insight into the evolutionary process that takes place in robotics. It provides articles covering a wide range of this exciting area. The progress of technical challenges and concepts may illuminate the relationship between developments that seem to be completely different at first sight. The robotics remains an exciting scientific and engineering field. The community looks optimistically ahead and also looks forward for the future challenges and new development

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

Investigating the Effects of a Task-Specific Fatigue Protocol on Hand Tracking Performance Using a Wrist Robotic Device

The purpose of this work was to evaluate the effects of a dynamic submaximal fatigue protocol and forearm/hand anthropometrics on hand tracking performance. Participants traced a 2:3 Lissajous curve using a haptic wrist robotic device (WristBot). This same curve was traced before the fatigue (baseline), during the fatigue protocol, and after the fatigue protocol. Post fatigue trials were completed at 0, 1, 2, 4, 6, 8, and 10 minutes after the cessation of the fatigue protocol. Overall tracking performance and movement smoothness decreased immediately. Directional biases in the normal and longitudinal component of tracking error were present after the fatigue protocol. Proximal forearm circumference and forearm length had a negative correlation with movement smoothness. Hand tracking performance decreased due to the submaximal fatigue protocol. Those with a larger proximal forearm circumference and longer forearm length had better movement smoothness performance which can be applied to the workplace where hand and wrist are predominately used