1,422 research outputs found
Tactile Mapping and Localization from High-Resolution Tactile Imprints
This work studies the problem of shape reconstruction and object localization
using a vision-based tactile sensor, GelSlim. The main contributions are the
recovery of local shapes from contact, an approach to reconstruct the tactile
shape of objects from tactile imprints, and an accurate method for object
localization of previously reconstructed objects. The algorithms can be applied
to a large variety of 3D objects and provide accurate tactile feedback for
in-hand manipulation. Results show that by exploiting the dense tactile
information we can reconstruct the shape of objects with high accuracy and do
on-line object identification and localization, opening the door to reactive
manipulation guided by tactile sensing. We provide videos and supplemental
information in the project's website
http://web.mit.edu/mcube/research/tactile_localization.html.Comment: ICRA 2019, 7 pages, 7 figures. Website:
http://web.mit.edu/mcube/research/tactile_localization.html Video:
https://youtu.be/uMkspjmDbq
FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedback
In this paper, we address the problem of using visuo-tactile feedback for
6-DoF localization and 3D reconstruction of unknown in-hand objects. We propose
FingerSLAM, a closed-loop factor graph-based pose estimator that combines local
tactile sensing at finger-tip and global vision sensing from a wrist-mount
camera. FingerSLAM is constructed with two constituent pose estimators: a
multi-pass refined tactile-based pose estimator that captures movements from
detailed local textures, and a single-pass vision-based pose estimator that
predicts from a global view of the object. We also design a loop closure
mechanism that actively matches current vision and tactile images to previously
stored key-frames to reduce accumulated error. FingerSLAM incorporates the two
sensing modalities of tactile and vision, as well as the loop closure mechanism
with a factor graph-based optimization framework. Such a framework produces an
optimized pose estimation solution that is more accurate than the standalone
estimators. The estimated poses are then used to reconstruct the shape of the
unknown object incrementally by stitching the local point clouds recovered from
tactile images. We train our system on real-world data collected with 20
objects. We demonstrate reliable visuo-tactile pose estimation and shape
reconstruction through quantitative and qualitative real-world evaluations on 6
objects that are unseen during training.Comment: Submitted and accepted to 2023 IEEE International Conference on
Robotics and Automation (ICRA 2023
In-Hand Manipulation of Unknown Objects with Tactile Sensing for Insertion
In this paper, we present a method to manipulate unknown objects in-hand
using tactile sensing without relying on a known object model. In many cases,
vision-only approaches may not be feasible; for example, due to occlusion in
cluttered spaces. We address this limitation by introducing a method to
reorient unknown objects using tactile sensing. It incrementally builds a
probabilistic estimate of the object shape and pose during task-driven
manipulation. Our approach uses Bayesian optimization to balance exploration of
the global object shape with efficient task completion. To demonstrate the
effectiveness of our method, we apply it to a simulated Tactile-Enabled Roller
Grasper, a gripper that rolls objects in hand while collecting tactile data. We
evaluate our method on an insertion task with randomly generated objects and
find that it reliably reorients objects while significantly reducing the
exploration time
Shapes reconstruction from robot tactile sensing
Shapes reconstruction bridges real objects and their computer models. Most of the shape reconstruction techniques were derived for computer vision applications. A very important sense of human, tactile sensing can be applied to acquire shape information about 2D and 3D objects. Nevertheless, tactile data usually has a lot of noise. In this thesis, I present an applicable scheme that acquires shape data using a simple joystick sensor and then reconstructs 2D shapes and 3D patches. The 2D shapes are tracked by an Adept Cobra robot and represented as polynomial functions determined by the 3L fitting algorithm. The 3D shapes are composed of multiple patches, each of which is described by a polynomial function generated by least-square fitting. Experiments have been carried out with the robot. A display environment for 3D objects has also been developed
Shape-independent hardness estimation using deep learning and a GelSight tactile sensor
Hardness is among the most important attributes of an object that humans learn about through touch. However, approaches for robots to estimate hardness are limited, due to the lack of information provided by current tactile sensors. In this work, we address these limitations by introducing a novel method for hardness estimation, based on the GelSight tactile sensor, and the method does not require accurate control of contact conditions or the shape of objects. A GelSight has a soft contact interface, and provides high resolution tactile images of contact geometry, as well as contact force and slip conditions. In this paper, we try to use the sensor to measure hardness of objects with multiple shapes, under a loosely controlled contact condition. The contact is made manually or by a robot hand, while the force and trajectory are unknown and uneven. We analyze the data using a deep constitutional (and recurrent) neural network. Experiments show that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale
Soft, Round, High Resolution Tactile Fingertip Sensors for Dexterous Robotic Manipulation
High resolution tactile sensors are often bulky and have shape profiles that
make them awkward for use in manipulation. This becomes important when using
such sensors as fingertips for dexterous multi-fingered hands, where boxy or
planar fingertips limit the available set of smooth manipulation strategies.
High resolution optical based sensors such as GelSight have until now been
constrained to relatively flat geometries due to constraints on illumination
geometry.Here, we show how to construct a rounded fingertip that utilizes a
form of light piping for directional illumination. Our sensors can replace the
standard rounded fingertips of the Allegro hand.They can capture high
resolution maps of the contact surfaces,and can be used to support various
dexterous manipulation tasks
- …