72 research outputs found
Shape-independent hardness estimation using deep learning and a GelSight tactile sensor
Hardness is among the most important attributes of an object that humans learn about through touch. However, approaches for robots to estimate hardness are limited, due to the lack of information provided by current tactile sensors. In this work, we address these limitations by introducing a novel method for hardness estimation, based on the GelSight tactile sensor, and the method does not require accurate control of contact conditions or the shape of objects. A GelSight has a soft contact interface, and provides high resolution tactile images of contact geometry, as well as contact force and slip conditions. In this paper, we try to use the sensor to measure hardness of objects with multiple shapes, under a loosely controlled contact condition. The contact is made manually or by a robot hand, while the force and trajectory are unknown and uneven. We analyze the data using a deep constitutional (and recurrent) neural network. Experiments show that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale
Active Clothing Material Perception using Tactile Sensing and Deep Learning
Humans represent and discriminate the objects in the same category using
their properties, and an intelligent robot should be able to do the same. In
this paper, we build a robot system that can autonomously perceive the object
properties through touch. We work on the common object category of clothing.
The robot moves under the guidance of an external Kinect sensor, and squeezes
the clothes with a GelSight tactile sensor, then it recognizes the 11
properties of the clothing according to the tactile data. Those properties
include the physical properties, like thickness, fuzziness, softness and
durability, and semantic properties, like wearing season and preferred washing
methods. We collect a dataset of 153 varied pieces of clothes, and conduct 6616
robot exploring iterations on them. To extract the useful information from the
high-dimensional sensory output, we applied Convolutional Neural Networks (CNN)
on the tactile data for recognizing the clothing properties, and on the Kinect
depth images for selecting exploration locations. Experiments show that using
the trained neural networks, the robot can autonomously explore the unknown
clothes and learn their properties. This work proposes a new framework for
active tactile perception system with vision-touch system, and has potential to
enable robots to help humans with varied clothing related housework.Comment: ICRA 2018 accepte
Connecting Look and Feel: Associating the visual and tactile properties of physical materials
For machines to interact with the physical world, they must understand the
physical properties of objects and materials they encounter. We use fabrics as
an example of a deformable material with a rich set of mechanical properties. A
thin flexible fabric, when draped, tends to look different from a heavy stiff
fabric. It also feels different when touched. Using a collection of 118 fabric
sample, we captured color and depth images of draped fabrics along with tactile
data from a high resolution touch sensor. We then sought to associate the
information from vision and touch by jointly training CNNs across the three
modalities. Through the CNN, each input, regardless of the modality, generates
an embedding vector that records the fabric's physical property. By comparing
the embeddings, our system is able to look at a fabric image and predict how it
will feel, and vice versa. We also show that a system jointly trained on vision
and touch data can outperform a similar system trained only on visual data when
tested purely with visual inputs
Improved GelSight Tactile Sensor for Measuring Geometry and Slip
A GelSight sensor uses an elastomeric slab covered with a reflective membrane
to measure tactile signals. It measures the 3D geometry and contact force
information with high spacial resolution, and successfully helped many
challenging robot tasks. A previous sensor, based on a semi-specular membrane,
produces high resolution but with limited geometry accuracy. In this paper, we
describe a new design of GelSight for robot gripper, using a Lambertian
membrane and new illumination system, which gives greatly improved geometric
accuracy while retaining the compact size. We demonstrate its use in measuring
surface normals and reconstructing height maps using photometric stereo. We
also use it for the task of slip detection, using a combination of information
about relative motions on the membrane surface and the shear distortions. Using
a robotic arm and a set of 37 everyday objects with varied properties, we find
that the sensor can detect translational and rotational slip in general cases,
and can be used to improve the stability of the grasp.Comment: IEEE/RSJ International Conference on Intelligent Robots and System
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
For humans, the process of grasping an object relies heavily on rich tactile
feedback. Most recent robotic grasping work, however, has been based only on
visual input, and thus cannot easily benefit from feedback after initiating
contact. In this paper, we investigate how a robot can learn to use tactile
information to iteratively and efficiently adjust its grasp. To this end, we
propose an end-to-end action-conditional model that learns regrasping policies
from raw visuo-tactile data. This model -- a deep, multimodal convolutional
network -- predicts the outcome of a candidate grasp adjustment, and then
executes a grasp by iteratively selecting the most promising actions. Our
approach requires neither calibration of the tactile sensors, nor any
analytical modeling of contact forces, thus reducing the engineering effort
required to obtain efficient grasping policies. We train our model with data
from about 6,450 grasping trials on a two-finger gripper equipped with GelSight
high-resolution tactile sensors on each finger. Across extensive experiments,
our approach outperforms a variety of baselines at (i) estimating grasp
adjustment outcomes, (ii) selecting efficient grasp adjustments for quick
grasping, and (iii) reducing the amount of force applied at the fingers, while
maintaining competitive performance. Finally, we study the choices made by our
model and show that it has successfully acquired useful and interpretable
grasping behaviors.Comment: 8 pages. Published on IEEE Robotics and Automation Letters (RAL).
Website: https://sites.google.com/view/more-than-a-feelin
HySenSe: A Hyper-Sensitive and High-Fidelity Vision-Based Tactile Sensor
In this paper, to address the sensitivity and durability trade-off of
Vision-based Tactile Sensor (VTSs), we introduce a hyper-sensitive and
high-fidelity VTS called HySenSe. We demonstrate that by solely changing one
step during the fabrication of the gel layer of the GelSight sensor (as the
most well-known VTS), we can substantially improve its sensitivity and
durability. Our experimental results clearly demonstrate the outperformance of
the HySenSe compared with a similar GelSight sensor in detecting textural
details of various objects under identical experimental conditions and low
interaction forces (<= 1.5 N).Comment: Accepted to IEEE Sensors 2022 Conferenc
- …