Proprioceptive Learning with Soft Polyhedral Networks

Proprioception is the "sixth sense" that detects limb postures with motor neurons. It requires a natural integration between the musculoskeletal systems and sensory receptors, which is challenging among modern robots that aim for lightweight, adaptive, and sensitive designs at a low cost. Here, we present the Soft Polyhedral Network with an embedded vision for physical interactions, capable of adaptive kinesthesia and viscoelastic proprioception by learning kinetic features. This design enables passive adaptations to omni-directional interactions, visually captured by a miniature high-speed motion tracking system embedded inside for proprioceptive learning. The results show that the soft network can infer real-time 6D forces and torques with accuracies of 0.25/0.24/0.35 N and 0.025/0.034/0.006 Nm in dynamic interactions. We also incorporate viscoelasticity in proprioception during static adaptation by adding a creep and relaxation modifier to refine the predicted results. The proposed soft network combines simplicity in design, omni-adaptation, and proprioceptive sensing with high accuracy, making it a versatile solution for robotics at a low cost with more than 1 million use cycles for tasks such as sensitive and competitive grasping, and touch-based geometry reconstruction. This study offers new insights into vision-based proprioception for soft robots in adaptive grasping, soft manipulation, and human-robot interaction.Comment: 20 pages, 10 figures, 2 tables, submitted to the International Journal of Robotics Research for revie

A survey of dextrous manipulation

technical reportThe development of mechanical end effectors capable of dextrous manipulation is a rapidly growing and quite successful field of research. It has in some sense put the focus on control issues, in particular, how to control these remarkably humanlike manipulators to perform the deft movement that we take for granted in the human hand. The kinematic and control issues surrounding manipulation research are clouded by more basic concerns such as: what is the goal of a manipulation system, is the anthropomorphic or functional design methodology appropriate, and to what degree does the control of the manipulator depend on other sensory systems. This paper examines the potential of creating a general purpose, anthropomorphically motivated, dextrous manipulation system. The discussion will focus on features of the human hand that permit its general usefulness as a manipulator. A survey of machinery designed to emulate these capabilities is presented. Finally, the tasks of grasping and manipulation are examined from the control standpoint to suggest a control paradigm which is descriptive, yet flexible and computationally efficient1

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Incipient slip detection and grasping automation for robotic surgery

Robotic minimally invasive surgery provides multiple improvements over traditional laparoscopic procedures, but one significant issue still encountered is their limited force control during the grasping and retraction of tissue, as the surgeon is separated from the instrument, and therefore denuded of their sense of touch and the applied forces. Prior solutions have largely looked towards haptic feedback to resolve this issue, but an alternative approach is to detect and monitor the occurrence of tissue slip events. This would allow the force to be automatically adjusted to prevent slip, minimising the clamp force used to maintain control, thus reducing the probability of tissue trauma. The aim of this work is to develop a method for the early detection and mitigation of tissue slip during robotic surgical manipulation tasks, helping to reduce tissue trauma and minimise tissue slip events. Initial investigations into literature, and evaluation of the slip mechanics when grasping soft, lubricated, deformable materials, indicated that small localised slips occur before the onset of macro slip. Two phenomena were identified in the slip mechanics investigation that could be employed to induce these slip in a measurable and repeatable manner. Firstly through using the tissue's deformable properties to create slip differentials between the front and rear of the grasper face, and secondly through using a curved surface to create a variation in the normal force, and thus frictional force, across the surface. Two instrumented grasper faces were developed, based on each of these phenomena, that were capable of monitoring the occurrence of localised tissue slip through monitoring the displacement of a series of independent movable islands that made up the grasper face. These were then demonstrated to be capable of automatically detecting slip events for a range of test conditions with tissue simulants, before being utilised to automatically control the grasping forces during a tissue retraction task. Both sensor systems provided similar levels of tissue control to one which utilised the maximum clamp force throughout the task, whilst applying lower forces during the early stages of retraction, reducing the probability of tissue damage. In addition the normal force based method, with the curved grasper face, was demonstrated to be effective for the early detection of slip when grasping porcine liver tissue, successfully detecting incipient slip in 77% of cases. This work provides a strong basis for further development of incipient slip sensing for surgical applications. It provides novel contributions in the understanding of slip mechanics of soft tissues, as well as presenting two separate novel sensing approaches for the automatic detection and mitigation of slip events, offering an opportunity for reducing the occurrence of tissue slip events whilst minimising tissue trauma, as well as surgeon fatigue

Adaptive robust interaction control for low-cost robotic grasping

Robotic grasping is a challenging area in the field of robotics. When a gripper starts interacting with an object to perform a grasp, the mechanical properties of the object (stiffness and damping) will play an important role. A gripper which is stable in isolated conditions, can become unstable when coupled to an object. This can lead to the extreme condition where the gripper becomes unstable and generates excessive or insufficient grip force resulting in the grasped object either being crushed, or falling and breaking. In addition to the stability issue, grasp maintenance is one of the most important requirements of any grasp where it guarantees a secure grasp in the presence of any unknown disturbance. The term grasp maintenance refers to the reaction of the controller in the presence of external disturbances, trying to prevent any undesired slippage. To do so, the controller continuously adjusts the grip force. This is a challenging task as it requires an accurate model of the friction and object’s weight to estimate a sufficient grip force to stop the object from slipping while incurring minimum deformation. Unfortunately, in reality, there is no solution which is able to obtain the mechanical properties, frictional coefficient and weight of an object before establishing a mechanical interaction with it. External disturbance forces are also stochastic meaning they are impossible to predict. This thesis addresses both of the problems mentioned above by:Creating a novel variable stiffness gripper, capable of grasping unknown objects, mainly those found in agricultural or food manufacturing companies. In addition to the stabilisation effect of the introduced variable stiffness mechanism, a novel force control algorithm has been designed that passively controls the grip force in variable stiffness grippers. Due to the passive nature of the suggested controller, it completely eliminates the necessity for any force sensor. The combination of both the proposed variable stiffness gripper and the passivity based control provides a unique solution for the stable grasp and force control problem in tendon driven, angular grippers.Introducing a novel active multi input-multi output slip prevention algorithm. The algorithm developed provides a robust control solution to endow direct drive parallel jaw grippers with the capability to stop held objects from slipping while incurring minimum deformation; this can be done without any prior knowledge of the object’s friction and weight. The large number of experiments provided in this thesis demonstrate the robustness of the proposed controller when controlling parallel jaw grippers in order to quickly grip, lift and place a broad range of objects firmly without dropping or crushing them. This is particularly useful for teleoperation and nuclear decommissioning tasks where there is often no accurate information available about the objects to be handled. This can mean that pre-programming of the gripper is required for each different object and for high numbers of objects this is impractical and overly time-consuming. A robust controller, which is able to compensate for any uncertainties regarding the object model and any unknown external disturbances during grasping, is implemented. This work has advanced the state of the art in the following two main areas: Direct impedance modulation for stable grasping in tendon driven, angular grippers. Active MIMO slip prevention grasp control for direct drive parallel jaw grippers

Material perception and action : The role of material properties in object handling

This dissertation is about visual perception of material properties and their role in preparation for object handling. Usually before an object is touched or picked-up we estimate its size and shape based on visual features to plan the grip size of our hand. After we have touched the object, the grip size is adjusted according to the provided haptic feedback and the object is handled safely. Similarly, we anticipate the required grip force to handle the object without slippage, based on its visual features and prior experience with similar objects. Previous studies on object handling have mostly examined object characteristics that are typical for object recognition, e.g., size, shape, weight, but in the recent years there has been a growing interest in object characteristics that are more typical to the type of material the object is made from. That said, in a series of studies we investigated the role of perceived material properties in decision-making and object handling, in which both digitally rendered materials and real objects made of different types of materials were presented to human subjects and a humanoid robot. Paper I is a reach-to-grasp study where human subjects were examined using motion capture technology. In this study, participants grasped and lifted paper cups that varied in appearance (i.e., matte vs. glossy) and weight. Here we were interested in both the temporal and spatial components of prehension to examine the role of material properties in grip preparation, and how visual features contribute to inferred hardness before haptic feedback has become available. We found the temporal and spatial components were not exclusively governed by the expected weight of the paper cups, instead glossiness and expected hardness has a significant role as well. In paper II, which is a follow-up on Paper I, we investigated the grip force component of prehension using the same experimental stimuli as used in paper I. In a similar experimental set up, using force sensors we examined the early grip force magnitudes applied by human subjects when grasping and lifting the same paper cups as used in Paper I. Here we found that early grip force scaling was not only guided by the object weight, but the visual characteristics of the material (i.e., matte vs. glossy) had a role as well. Moreover, the results suggest that grip force scaling during the initial object lifts is guided by expected hardness that is to some extend based on visual material properties. Paper III is a visual judgment task where psychophysical measurements were used to examine how the material properties, roughness and glossiness, influence perceived bounce height and consequently perceived hardness. In a paired-comparison task, human subjects observed a bouncing ball bounce on various surface planes and judged their bounce height. Here we investigated, what combination of surface properties, i.e., roughness or glossiness, makes a surface plane to be perceived bounceable. The results demonstrate that surface planes with rough properties are believed to afford higher bounce heights for the bouncing ball, compared to surface planes with smooth properties. Interestingly, adding shiny properties to the rough and smooth surface planes, reduced the judged difference, as if surface planes with gloss are believed to afford higher bounce heights irrespective of how smooth or rough the surface plane is beneath. This suggests that perceived bounce height involves not only the physical elements of the bounce height, but also the visual characteristics of the material properties of the surface planes the ball bounces on. In paper IV we investigated the development of material knowledge using a robotic system. A humanoid robot explored real objects made of different types of materials, using both camera and haptic systems. The objects varied in visual appearances (e.g., texture, color, shape, size), weight, and hardness, and in two experiments, the robot picked up and placed the experimental objects several times using its arm. Here we used the haptic signals from the servos controlling the arm and the shoulder of the robot, to obtain measurements of the weight and hardness of the objects, and the camera system to collect data on the visual features of the objects. After the robot had repeatedly explored the objects, an associative learning model was created based on the training data to demonstrate how the robotic system could produce multi-modal mapping between the visual and haptic features of the objects. In sum, in this thesis we show that visual material properties and prior knowledge of how materials look like and behave like has a significant role in action planning