Autoencoding a Soft Touch to Learn Grasping from On-land to Underwater

Robots play a critical role as the physical agent of human operators in exploring the ocean. However, it remains challenging to grasp objects reliably while fully submerging under a highly pressurized aquatic environment with little visible light, mainly due to the fluidic interference on the tactile mechanics between the finger and object surfaces. This study investigates the transferability of grasping knowledge from on-land to underwater via a vision-based soft robotic finger that learns 6D forces and torques (FT) using a Supervised Variational Autoencoder (SVAE). A high-framerate camera captures the whole-body deformations while a soft robotic finger interacts with physical objects on-land and underwater. Results show that the trained SVAE model learned a series of latent representations of the soft mechanics transferrable from land to water, presenting a superior adaptation to the changing environments against commercial FT sensors. Soft, delicate, and reactive grasping enabled by tactile intelligence enhances the gripper's underwater interaction with improved reliability and robustness at a much-reduced cost, paving the path for learning-based intelligent grasping to support fundamental scientific discoveries in environmental and ocean research.Comment: 17 pages, 5 figures, 1 table, submitted to Advanced Intelligent Systems for revie

Robust dexterous telemanipulation following object-orientation commands

This paper aims to present a procedure to change the orientation of a grasped object using dexterous manipulation. The manipulation is controlled by teleoperation in a very simple way, with the commands introduced by an operator using a keyboard. Design/methodology/approach - The paper shows a teleoperation scheme, hand kinematics and a manipulation strategy to manipulate different objects using the Schunk Dexterous Hand (SDH2). A state machine is used to model the teleoperation actions and the system states. A virtual link is used to include the contact point on the hand kinematics of the SDH2. Findings - Experiments were conducted to evaluate the proposed approach with different objects, varying the initial grasp configuration and the sequence of actions commanded by the operator. Originality/value - The proposed approach uses a shared telemanipulation schema to perform dexterous manipulation; in this schema, the operator sends high-level commands and a local system uses this information, jointly with tactile measurements and the current status of the system, to generate proper setpoints for the low-level control of the fingers, which may be a commercial close one. The main contribution of this work is the mentioned local system, simple enough for practical applications and robust enough to avoid object falls.Postprint (author's final draft

Robust dexterous telemanipulation following object-orientation commands

This paper aims to present a procedure to change the orientation of a grasped object using dexterous manipulation. The manipulation is controlled by teleoperation in a very simple way, with the commands introduced by an operator using a keyboard. Design/methodology/approach - The paper shows a teleoperation scheme, hand kinematics and a manipulation strategy to manipulate different objects using the Schunk Dexterous Hand (SDH2). A state machine is used to model the teleoperation actions and the system states. A virtual link is used to include the contact point on the hand kinematics of the SDH2. Findings - Experiments were conducted to evaluate the proposed approach with different objects, varying the initial grasp configuration and the sequence of actions commanded by the operator. Originality/value - The proposed approach uses a shared telemanipulation schema to perform dexterous manipulation; in this schema, the operator sends high-level commands and a local system uses this information, jointly with tactile measurements and the current status of the system, to generate proper setpoints for the low-level control of the fingers, which may be a commercial close one. The main contribution of this work is the mentioned local system, simple enough for practical applications and robust enough to avoid object falls.Postprint (author's final draft

TacFR-Gripper: A Reconfigurable Fin Ray-Based Compliant Robotic Gripper with Tactile Skin for In-Hand Manipulation

This paper introduces the TacFR-Gripper, a reconfigurable Fin Ray-based soft and compliant robotic gripper equipped with tactile skin, which can be used for dexterous in-hand manipulation tasks. This gripper can adaptively grasp objects of diverse shapes and stiffness levels. An array of Force Sensitive Resistor (FSR) sensors is embedded within the robotic finger to serve as the tactile skin, enabling the robot to perceive contact information during manipulation. We provide theoretical analysis for gripper design, including kinematic analysis, workspace analysis, and finite element analysis to identify the relationship between the gripper's load and its deformation. Moreover, we implemented a Graph Neural Network (GNN)-based tactile perception approach to enable reliable grasping without accidental slip or excessive force. Three physical experiments were conducted to quantify the performance of the TacFR-Gripper. These experiments aimed to i) assess the grasp success rate across various everyday objects through different configurations, ii) verify the effectiveness of tactile skin with the GNN algorithm in grasping, iii) evaluate the gripper's in-hand manipulation capabilities for object pose control. The experimental results indicate that the TacFR-Gripper can grasp a wide range of complex-shaped objects with a high success rate and deliver dexterous in-hand manipulation. Additionally, the integration of tactile skin with the GNN algorithm enhances grasp stability by incorporating tactile feedback during manipulations. For more details of this project, please view our website: https://sites.google.com/view/tacfr-gripper/homepage

Sensors for Robotic Hands: A Survey of State of the Art

Recent decades have seen significant progress in the field of artificial hands. Most of the surveys, which try to capture the latest developments in this field, focused on actuation and control systems of these devices. In this paper, our goal is to provide a comprehensive survey of the sensors for artificial hands. In order to present the evolution of the field, we cover five year periods starting at the turn of the millennium. At each period, we present the robot hands with a focus on their sensor systems dividing them into categories, such as prosthetics, research devices, and industrial end-effectors.We also cover the sensors developed for robot hand usage in each era. Finally, the period between 2010 and 2015 introduces the reader to the state of the art and also hints to the future directions in the sensor development for artificial hands

Manos Robóticas Antropomórficas: una revisión

This paper presents a review on main topic regarding to anthropomorphic robotic hands developed in the last years, taking into account the more important mechatronics designs submit on the literature, and making a comparison between them. The next chapters deepen on level of anthropomorphism and dexterity in advanced actuated hands and upper limbs prostheses, as well as a brief overview on issues such as grasping, transmission mechanisms, sensory and actuator system, and also a short introduction on under-actuated robotic hands is reported.Este artículo presenta una revisión de los principales desarrollos que se han hecho en los últimos años en manos robóticas antropomórficas. Las primeras secciones tratan temas como el grado de antropomorfismo y de destreza en las manos robóticas más avanzadas, incluyendo una comparación entre ellas. También se abordan temas como la capacidad de agarre de los efectores finales, los mecanismos de trasmisión, el sistema actuador y sensórico, así como una breve introducción al tema de manos robóticas sub-actuadas. Dirección de correspondencia: Carrera 11 # 101-80, Bogotá (Colombia)

Ground Robotic Hand Applications for the Space Program study (GRASP)

This document reports on a NASA-STDP effort to address research interests of the NASA Kennedy Space Center (KSC) through a study entitled, Ground Robotic-Hand Applications for the Space Program (GRASP). The primary objective of the GRASP study was to identify beneficial applications of specialized end-effectors and robotic hand devices for automating any ground operations which are performed at the Kennedy Space Center. Thus, operations for expendable vehicles, the Space Shuttle and its components, and all payloads were included in the study. Typical benefits of automating operations, or augmenting human operators performing physical tasks, include: reduced costs; enhanced safety and reliability; and reduced processing turnaround time

Development of a multi-modal tactile force sensing system for deep-sea applications

With the increasing demand for autonomy in robotic systems, there is a rising need for sensory data sensed via different modalities. In this way system states and the aspects of unstructured environments can be assessed in the most detailed fashion possible, thus providing a basis for making decisions regarding the robotâ s task. Com- pared to other sensing modalities, the sense of touch is underrepresented in todayâ s robots. That is where this thesis comes in. A tactile sensing system is developed that combines several modalities of contact sensing. The use of the tactile sense in robotic grippers is of great relevance especially for robotic systems in the deep sea. Up to now manipulation systems in master-slave control mode have been used in this area of application. An operator performing the manipulation task has to rely on visual feedback coming from cameras. Working on the oceanâ s seafloor means having to cope with conditions of limited visibility caused by swirled-up sediment

Progettazione e Controllo di Mani Robotiche

The application of dexterous robotic hands out of research laboratories has been limited by the intrinsic complexity that these devices present. This is directly reflected as an economically unreasonable cost and a low overall reliability. Within the research reported in this thesis it is shown how the problem of complexity in the design of robotic hands can be tackled, taking advantage of modern technologies (i.e. rapid prototyping), leading to innovative concepts for the design of the mechanical structure, the actuation and sensory systems. The solutions adopted drastically reduce the prototyping and production costs and increase the reliability, reducing the number of parts required and averaging their single reliability factors. In order to get guidelines for the design process, the problem of robotic grasp and manipulation by a dual arm/hand system has been reviewed. In this way, the requirements that should be fulfilled at hardware level to guarantee successful execution of the task has been highlighted. The contribution of this research from the manipulation planning side focuses on the redundancy resolution that arise in the execution of the task in a dexterous arm/hand system. In literature the problem of coordination of arm and hand during manipulation of an object has been widely analyzed in theory but often experimentally demonstrated in simplified robotic setup. Our aim is to cover the lack in the study of this topic and experimentally evaluate it in a complex system as a anthropomorphic arm hand system

Innovative robot hand designs of reduced complexity for dexterous manipulation

This thesis investigates the mechanical design of robot hands to sensibly reduce the system complexity in terms of the number of actuators and sensors, and control needs for performing grasping and in-hand manipulations of unknown objects. Human hands are known to be the most complex, versatile, dexterous manipulators in nature, from being able to operate sophisticated surgery to carry out a wide variety of daily activity tasks (e.g. preparing food, changing cloths, playing instruments, to name some). However, the understanding of why human hands can perform such fascinating tasks still eludes complete comprehension. Since at least the end of the sixteenth century, scientists and engineers have tried to match the sensory and motor functions of the human hand. As a result, many contemporary humanoid and anthropomorphic robot hands have been developed to closely replicate the appearance and dexterity of human hands, in many cases using sophisticated designs that integrate multiple sensors and actuators---which make them prone to error and difficult to operate and control, particularly under uncertainty. In recent years, several simplification approaches and solutions have been proposed to develop more effective and reliable dexterous robot hands. These techniques, which have been based on using underactuated mechanical designs, kinematic synergies, or compliant materials, to name some, have opened up new ways to integrate hardware enhancements to facilitate grasping and dexterous manipulation control and improve reliability and robustness. Following this line of thought, this thesis studies four robot hand hardware aspects for enhancing grasping and manipulation, with a particular focus on dexterous in-hand manipulation. Namely: i) the use of passive soft fingertips; ii) the use of rigid and soft active surfaces in robot fingers; iii) the use of robot hand topologies to create particular in-hand manipulation trajectories; and iv) the decoupling of grasping and in-hand manipulation by introducing a reconfigurable palm. In summary, the findings from this thesis provide important notions for understanding the significance of mechanical and hardware elements in the performance and control of human manipulation. These findings show great potential in developing robust, easily programmable, and economically viable robot hands capable of performing dexterous manipulations under uncertainty, while exhibiting a valuable subset of functions of the human hand.Open Acces