283 research outputs found
Realtime State Estimation with Tactile and Visual sensing. Application to Planar Manipulation
Accurate and robust object state estimation enables successful object
manipulation. Visual sensing is widely used to estimate object poses. However,
in a cluttered scene or in a tight workspace, the robot's end-effector often
occludes the object from the visual sensor. The robot then loses visual
feedback and must fall back on open-loop execution.
In this paper, we integrate both tactile and visual input using a framework
for solving the SLAM problem, incremental smoothing and mapping (iSAM), to
provide a fast and flexible solution. Visual sensing provides global pose
information but is noisy in general, whereas contact sensing is local, but its
measurements are more accurate relative to the end-effector. By combining them,
we aim to exploit their advantages and overcome their limitations. We explore
the technique in the context of a pusher-slider system. We adapt iSAM's
measurement cost and motion cost to the pushing scenario, and use an
instrumented setup to evaluate the estimation quality with different object
shapes, on different surface materials, and under different contact modes
Optical Proximity Sensing for Pose Estimation During In-Hand Manipulation
During in-hand manipulation, robots must be able to continuously estimate the
pose of the object in order to generate appropriate control actions. The
performance of algorithms for pose estimation hinges on the robot's sensors
being able to detect discriminative geometric object features, but previous
sensing modalities are unable to make such measurements robustly. The robot's
fingers can occlude the view of environment- or robot-mounted image sensors,
and tactile sensors can only measure at the local areas of contact. Motivated
by fingertip-embedded proximity sensors' robustness to occlusion and ability to
measure beyond the local areas of contact, we present the first evaluation of
proximity sensor based pose estimation for in-hand manipulation. We develop a
novel two-fingered hand with fingertip-embedded optical time-of-flight
proximity sensors as a testbed for pose estimation during planar in-hand
manipulation. Here, the in-hand manipulation task consists of the robot moving
a cylindrical object from one end of its workspace to the other. We
demonstrate, with statistical significance, that proximity-sensor based pose
estimation via particle filtering during in-hand manipulation: a) exhibits 50%
lower average pose error than a tactile-sensor based baseline; b) empowers a
model predictive controller to achieve 30% lower final positioning error
compared to when using tactile-sensor based pose estimates.Comment: 8 pages, 6 figure
Sense, Think, Grasp: A study on visual and tactile information processing for autonomous manipulation
Interacting with the environment using hands is one of the distinctive
abilities of humans with respect to other species. This aptitude reflects on
the crucial role played by objects\u2019 manipulation in the world that we have
shaped for us. With a view of bringing robots outside industries for supporting
people during everyday life, the ability of manipulating objects
autonomously and in unstructured environments is therefore one of the basic
skills they need. Autonomous manipulation is characterized by great
complexity especially regarding the processing of sensors information to
perceive the surrounding environment. Humans rely on vision for wideranging
tridimensional information, prioprioception for the awareness of
the relative position of their own body in the space and the sense of touch
for local information when physical interaction with objects happens. The
study of autonomous manipulation in robotics aims at transferring similar
perceptive skills to robots so that, combined with state of the art control
techniques, they could be able to achieve similar performance in manipulating
objects. The great complexity of this task makes autonomous
manipulation one of the open problems in robotics that has been drawing
increasingly the research attention in the latest years.
In this work of Thesis, we propose possible solutions to some key components
of autonomous manipulation, focusing in particular on the perception
problem and testing the developed approaches on the humanoid robotic platform iCub. When available, vision is the first source of information
to be processed for inferring how to interact with objects. The object
modeling and grasping pipeline based on superquadric functions we designed
meets this need, since it reconstructs the object 3D model from partial
point cloud and computes a suitable hand pose for grasping the object.
Retrieving objects information with touch sensors only is a relevant skill
that becomes crucial when vision is occluded, as happens for instance during
physical interaction with the object. We addressed this problem with
the design of a novel tactile localization algorithm, named Memory Unscented
Particle Filter, capable of localizing and recognizing objects relying solely
on 3D contact points collected on the object surface. Another key point of
autonomous manipulation we report on in this Thesis work is bi-manual
coordination. The execution of more advanced manipulation tasks in fact
might require the use and coordination of two arms. Tool usage for instance
often requires a proper in-hand object pose that can be obtained via
dual-arm re-grasping. In pick-and-place tasks sometimes the initial and
target position of the object do not belong to the same arm workspace, then
requiring to use one hand for lifting the object and the other for locating it
in the new position. At this regard, we implemented a pipeline for executing
the handover task, i.e. the sequences of actions for autonomously passing an
object from one robot hand on to the other.
The contributions described thus far address specific subproblems of
the more complex task of autonomous manipulation. This actually differs
from what humans do, in that humans develop their manipulation
skills by learning through experience and trial-and-error strategy. Aproper
mathematical formulation for encoding this learning approach is given by
Deep Reinforcement Learning, that has recently proved to be successful in
many robotics applications. For this reason, in this Thesis we report also
on the six month experience carried out at Berkeley Artificial Intelligence
Research laboratory with the goal of studying Deep Reinforcement Learning
and its application to autonomous manipulation
Active Exploration Using Gaussian Random Fields and Gaussian Process Implicit Surfaces
In this work we study the problem of exploring surfaces and building compact
3D representations of the environment surrounding a robot through active
perception. We propose an online probabilistic framework that merges visual and
tactile measurements using Gaussian Random Field and Gaussian Process Implicit
Surfaces. The system investigates incomplete point clouds in order to find a
small set of regions of interest which are then physically explored with a
robotic arm equipped with tactile sensors. We show experimental results
obtained using a PrimeSense camera, a Kinova Jaco2 robotic arm and Optoforce
sensors on different scenarios. We then demonstrate how to use the online
framework for object detection and terrain classification.Comment: 8 pages, 6 figures, external contents (https://youtu.be/0-UlFRQT0JI
Data-Driven Grasp Synthesis - A Survey
We review the work on data-driven grasp synthesis and the methodologies for
sampling and ranking candidate grasps. We divide the approaches into three
groups based on whether they synthesize grasps for known, familiar or unknown
objects. This structure allows us to identify common object representations and
perceptual processes that facilitate the employed data-driven grasp synthesis
technique. In the case of known objects, we concentrate on the approaches that
are based on object recognition and pose estimation. In the case of familiar
objects, the techniques use some form of a similarity matching to a set of
previously encountered objects. Finally for the approaches dealing with unknown
objects, the core part is the extraction of specific features that are
indicative of good grasps. Our survey provides an overview of the different
methodologies and discusses open problems in the area of robot grasping. We
also draw a parallel to the classical approaches that rely on analytic
formulations.Comment: 20 pages, 30 Figures, submitted to IEEE Transactions on Robotic
- …