374 research outputs found
Cable Manipulation with a Tactile-Reactive Gripper
Cables are complex, high dimensional, and dynamic objects. Standard
approaches to manipulate them often rely on conservative strategies that
involve long series of very slow and incremental deformations, or various
mechanical fixtures such as clamps, pins or rings. We are interested in
manipulating freely moving cables, in real time, with a pair of robotic
grippers, and with no added mechanical constraints. The main contribution of
this paper is a perception and control framework that moves in that direction,
and uses real-time tactile feedback to accomplish the task of following a
dangling cable. The approach relies on a vision-based tactile sensor, GelSight,
that estimates the pose of the cable in the grip, and the friction forces
during cable sliding. We achieve the behavior by combining two tactile-based
controllers: 1) Cable grip controller, where a PD controller combined with a
leaky integrator regulates the gripping force to maintain the frictional
sliding forces close to a suitable value; and 2) Cable pose controller, where
an LQR controller based on a learned linear model of the cable sliding dynamics
keeps the cable centered and aligned on the fingertips to prevent the cable
from falling from the grip. This behavior is possible by a reactive gripper
fitted with GelSight-based high-resolution tactile sensors. The robot can
follow one meter of cable in random configurations within 2-3 hand regrasps,
adapting to cables of different materials and thicknesses. We demonstrate a
robot grasping a headphone cable, sliding the fingers to the jack connector,
and inserting it. To the best of our knowledge, this is the first
implementation of real-time cable following without the aid of mechanical
fixtures.Comment: Accepted to RSS 202
Visual-Tactile Multimodality for Following Deformable Linear Objects Using Reinforcement Learning
Manipulation of deformable objects is a challenging task for a robot. It will
be problematic to use a single sensory input to track the behaviour of such
objects: vision can be subjected to occlusions, whereas tactile inputs cannot
capture the global information that is useful for the task. In this paper, we
study the problem of using vision and tactile inputs together to complete the
task of following deformable linear objects, for the first time. We create a
Reinforcement Learning agent using different sensing modalities and investigate
how its behaviour can be boosted using visual-tactile fusion, compared to using
a single sensing modality. To this end, we developed a benchmark in simulation
for manipulating the deformable linear objects using multimodal sensing inputs.
The policy of the agent uses distilled information, e.g., the pose of the
object in both visual and tactile perspectives, instead of the raw sensing
signals, so that it can be directly transferred to real environments. In this
way, we disentangle the perception system and the learned control policy. Our
extensive experiments show that the use of both vision and tactile inputs,
together with proprioception, allows the agent to complete the task in up to
92% of cases, compared to 77% when only one of the signals is given. Our
results can provide valuable insights for the future design of tactile sensors
and for deformable objects manipulation.Comment: 8 pages, 11 figure
GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger
This work describes the development of a high-resolution tactile-sensing
finger for robot grasping. This finger, inspired by previous GelSight sensing
techniques, features an integration that is slimmer, more robust, and with more
homogeneous output than previous vision-based tactile sensors. To achieve a
compact integration, we redesign the optical path from illumination source to
camera by combining light guides and an arrangement of mirror reflections. We
parameterize the optical path with geometric design variables and describe the
tradeoffs between the finger thickness, the depth of field of the camera, and
the size of the tactile sensing area. The sensor sustains the wear from
continuous use -- and abuse -- in grasping tasks by combining tougher materials
for the compliant soft gel, a textured fabric skin, a structurally rigid body,
and a calibration process that maintains homogeneous illumination and contrast
of the tactile images during use. Finally, we evaluate the sensor's durability
along four metrics that track the signal quality during more than 3000 grasping
experiments.Comment: RA-L Pre-print. 8 page
Towards Robust Autonomous Grasping with Reflexes Using High-Bandwidth Sensing and Actuation
Modern robotic manipulation systems fall short of human manipulation skills
partly because they rely on closing feedback loops exclusively around vision
data, which reduces system bandwidth and speed. By developing autonomous
grasping reflexes that rely on high-bandwidth force, contact, and proximity
data, the overall system speed and robustness can be increased while reducing
reliance on vision data. We are developing a new system built around a
low-inertia, high-speed arm with nimble fingers that combines a high-level
trajectory planner operating at less than 1 Hz with low-level autonomous reflex
controllers running upwards of 300 Hz. We characterize the reflex system by
comparing the volume of the set of successful grasps for a naive baseline
controller and variations of our reflexive grasping controller, finding that
our controller expands the set of successful grasps by 55% relative to the
baseline. We also deploy our reflexive grasping controller with a simple
vision-based planner in an autonomous clutter clearing task, achieving a grasp
success rate above 90% while clearing over 100 items.Comment: 6 pages, 1 page of references, supplementary video at
https://youtu.be/f8Coo02Jvdg. Submitted to ICRA 202
Tactile based robotic skills for cable routing operations
This paper proposes a set of tactile based skills to perform robotic cable routing operations for deformable linear objects (DLOs) characterized by considerable stiffness and constrained at both ends. In particular, tactile data are exploited to reconstruct the shape of the grasped portion of the DLO and to estimate the future local one. This information is exploited to obtain a grasping configuration aligned to the local shape of the DLO, starting from a rough initial grasping pose, and to follow the DLO's contour in the three-dimensional space. Taking into account the distance travelled along the arc length of the DLO, the robot can detect the cable segments that must be firmly grasped and inserted in intermediate clips, continuing then to slide along the contour until the next DLO's portion, that has to be clipped, is reached. The proposed skills are experimentally validated with an industrial robot on different DLOs in several configurations and on a cable routing use case
Augmenting Off-the-Shelf Grippers with Tactile Sensing
The development of tactile sensing and its fusion with computer vision is
expected to enhance robotic systems in handling complex tasks like deformable
object manipulation. However, readily available industrial grippers typically
lack tactile feedback, which has led researchers to develop and integrate their
own tactile sensors. This has resulted in a wide range of sensor hardware,
making it difficult to compare performance between different systems. We
highlight the value of accessible open-source sensors and present a set of
fingertips specifically designed for fine object manipulation, with readily
interpretable data outputs. The fingertips are validated through two difficult
tasks: cloth edge tracing and cable tracing. Videos of these demonstrations, as
well as design files and readout code can be found at
https://github.com/RemkoPr/icra-2023-workshop-tactile-fingertips.Comment: Project repo:
https://github.com/RemkoPr/icra-2023-workshop-tactile-fingertip
Visuotactile Affordances for Cloth Manipulation with Local Control
Cloth in the real world is often crumpled, self-occluded, or folded in on
itself such that key regions, such as corners, are not directly graspable,
making manipulation difficult. We propose a system that leverages visual and
tactile perception to unfold the cloth via grasping and sliding on edges. By
doing so, the robot is able to grasp two adjacent corners, enabling subsequent
manipulation tasks like folding or hanging. As components of this system, we
develop tactile perception networks that classify whether an edge is grasped
and estimate the pose of the edge. We use the edge classification network to
supervise a visuotactile edge grasp affordance network that can grasp edges
with a 90% success rate. Once an edge is grasped, we demonstrate that the robot
can slide along the cloth to the adjacent corner using tactile pose
estimation/control in real time. See
http://nehasunil.com/visuotactile/visuotactile.html for videos.Comment: Accepted at CoRL 2022. Project website:
http://nehasunil.com/visuotactile/visuotactile.htm
Visual-tactile learning of garment unfolding for robot-assisted dressing
Assistive robots have the potential to support disabled and elderly people in daily dressing activities. An intermediate stage of dressing is to manipulate the garment from a crumpled initial state to an unfolded configuration that facilitates robust dressing. Applying quasi-static grasping actions with vision feedback on garment unfolding usually suffers from occluded grasping points. In this work, we propose a dynamic manipulation strategy: tracing the garment edge until the hidden corner is revealed. We introduce a model-based approach, where a deep visual-tactile predictive model iteratively learns to perform servoing from raw sensor data. The predictive model is formalized as Conditional Variational Autoencoder with contrastive optimization, which jointly learns underlying visual-tactile latent representations, a latent garment dynamics model, and future predictions of garment states. Two cost functions are explored: the visual cost, defined by garment corner positions, guarantees the gripper to move towards the corner, while the tactile cost, defined by garment edge poses, prevents the garment from falling from the gripper. The experimental results demonstrate the improvement of our contrastive visual-tactile model predictive control over single sensing modality and baseline model learning techniques. The proposed method enables a robot to unfold back-opening hospital gowns and perform upper-body dressing
Enabling Robot Manipulation of Soft and Rigid Objects with Vision-based Tactile Sensors
Endowing robots with tactile capabilities opens up new possibilities for
their interaction with the environment, including the ability to handle fragile
and/or soft objects. In this work, we equip the robot gripper with low-cost
vision-based tactile sensors and propose a manipulation algorithm that adapts
to both rigid and soft objects without requiring any knowledge of their
properties. The algorithm relies on a touch and slip detection method, which
considers the variation in the tactile images with respect to reference ones.
We validate the approach on seven different objects, with different properties
in terms of rigidity and fragility, to perform unplugging and lifting tasks.
Furthermore, to enhance applicability, we combine the manipulation algorithm
with a grasp sampler for the task of finding and picking a grape from a bunch
without damaging~it.Comment: Published in IEEE International Conference on Automation Science and
Engineering (CASE2023
- …