4 research outputs found
Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks
Material recognition enables robots to incorporate knowledge of material
properties into their interactions with everyday objects. For example, material
recognition opens up opportunities for clearer communication with a robot, such
as "bring me the metal coffee mug", and recognizing plastic versus metal is
crucial when using a microwave or oven. However, collecting labeled training
data with a robot is often more difficult than unlabeled data. We present a
semi-supervised learning approach for material recognition that uses generative
adversarial networks (GANs) with haptic features such as force, temperature,
and vibration. Our approach achieves state-of-the-art results and enables a
robot to estimate the material class of household objects with ~90% accuracy
when 92% of the training data are unlabeled. We explore how well this approach
can recognize the material of new objects and we discuss challenges facing
generalization. To motivate learning from unlabeled training data, we also
compare results against several common supervised learning classifiers. In
addition, we have released the dataset used for this work which consists of
time-series haptic measurements from a robot that conducted thousands of
interactions with 72 household objects.Comment: 11 pages, 6 figures, 6 tables, 1st Conference on Robot Learning (CoRL
2017
Inferring the Material Properties of Granular Media for Robotic Tasks
Granular media (e.g., cereal grains, plastic resin pellets, and pills) are
ubiquitous in robotics-integrated industries, such as agriculture,
manufacturing, and pharmaceutical development. This prevalence mandates the
accurate and efficient simulation of these materials. This work presents a
software and hardware framework that automatically calibrates a fast physics
simulator to accurately simulate granular materials by inferring material
properties from real-world depth images of granular formations (i.e., piles and
rings). Specifically, coefficients of sliding friction, rolling friction, and
restitution of grains are estimated from summary statistics of grain formations
using likelihood-free Bayesian inference. The calibrated simulator accurately
predicts unseen granular formations in both simulation and experiment;
furthermore, simulator predictions are shown to generalize to more complex
tasks, including using a robot to pour grains into a bowl, as well as to create
a desired pattern of piles and rings. Visualizations of the framework and
experiments can be viewed at https://youtu.be/OBvV5h2NMKAComment: 8 pages, 6 figures, appeared in ICRA 2020; fixed misplaced image in
figure 4; updated video link; fixed resolution for figure
Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects
Humans display the remarkable ability to sense the world through tools and
other held objects. For example, we are able to pinpoint impact locations on a
held rod and tell apart different textures using a rigid probe. In this work,
we consider how we can enable robots to have a similar capacity, i.e., to
embody tools and extend perception using standard grasped objects. We propose
that vibro-tactile sensing using dynamic tactile sensors on the robot fingers,
along with machine learning models, enables robots to decipher contact
information that is transmitted as vibrations along rigid objects. This paper
reports on extensive experiments using the BioTac micro-vibration sensor and a
new event dynamic sensor, the NUSkin, capable of multi-taxel sensing at 4~kHz.
We demonstrate that fine localization on a held rod is possible using our
approach (with errors less than 1 cm on a 20 cm rod). Next, we show that
vibro-tactile perception can lead to reasonable grasp stability prediction
during object handover, and accurate food identification using a standard fork.
We find that multi-taxel vibro-tactile sensing at sufficiently high sampling
rate led to the best performance across the various tasks and objects. Taken
together, our results provides both evidence and guidelines for using
vibro-tactile perception to extend tactile perception, which we believe will
lead to enhanced competency with tools and better physical
human-robot-interaction.Comment: 9 pages, 7 figures. This version adds additional related work and
updated result
STReSSD: Sim-To-Real from Sound for Stochastic Dynamics
Sound is an information-rich medium that captures dynamic physical events.
This work presents STReSSD, a framework that uses sound to bridge the
simulation-to-reality gap for stochastic dynamics, demonstrated for the
canonical case of a bouncing ball. A physically-motivated noise model is
presented to capture stochastic behavior of the balls upon collision with the
environment. A likelihood-free Bayesian inference framework is used to infer
the parameters of the noise model, as well as a material property called the
coefficient of restitution, from audio observations. The same inference
framework and the calibrated stochastic simulator are then used to learn a
probabilistic model of ball dynamics. The predictive capabilities of the
dynamics model are tested in two robotic experiments. First, open-loop
predictions anticipate probabilistic success of bouncing a ball into a cup. The
second experiment integrates audio perception with a robotic arm to track and
deflect a bouncing ball in real-time. We envision that this work is a step
towards integrating audio-based inference for dynamic robotic tasks.
Experimental results can be viewed at https://youtu.be/b7pOrgZrArk.Comment: 25 pages, 35 figures, The Conference on Robot Learning (CoRL) 202