9 research outputs found
Active Clothing Material Perception using Tactile Sensing and Deep Learning
Humans represent and discriminate the objects in the same category using
their properties, and an intelligent robot should be able to do the same. In
this paper, we build a robot system that can autonomously perceive the object
properties through touch. We work on the common object category of clothing.
The robot moves under the guidance of an external Kinect sensor, and squeezes
the clothes with a GelSight tactile sensor, then it recognizes the 11
properties of the clothing according to the tactile data. Those properties
include the physical properties, like thickness, fuzziness, softness and
durability, and semantic properties, like wearing season and preferred washing
methods. We collect a dataset of 153 varied pieces of clothes, and conduct 6616
robot exploring iterations on them. To extract the useful information from the
high-dimensional sensory output, we applied Convolutional Neural Networks (CNN)
on the tactile data for recognizing the clothing properties, and on the Kinect
depth images for selecting exploration locations. Experiments show that using
the trained neural networks, the robot can autonomously explore the unknown
clothes and learn their properties. This work proposes a new framework for
active tactile perception system with vision-touch system, and has potential to
enable robots to help humans with varied clothing related housework.Comment: ICRA 2018 accepte
Soft, Round, High Resolution Tactile Fingertip Sensors for Dexterous Robotic Manipulation
High resolution tactile sensors are often bulky and have shape profiles that
make them awkward for use in manipulation. This becomes important when using
such sensors as fingertips for dexterous multi-fingered hands, where boxy or
planar fingertips limit the available set of smooth manipulation strategies.
High resolution optical based sensors such as GelSight have until now been
constrained to relatively flat geometries due to constraints on illumination
geometry.Here, we show how to construct a rounded fingertip that utilizes a
form of light piping for directional illumination. Our sensors can replace the
standard rounded fingertips of the Allegro hand.They can capture high
resolution maps of the contact surfaces,and can be used to support various
dexterous manipulation tasks
Textile Taxonomy and Classification Using Pulling and Twisting
Identification of textile properties is an important milestone toward
advanced robotic manipulation tasks that consider interaction with clothing
items such as assisted dressing, laundry folding, automated sewing, textile
recycling and reusing. Despite the abundance of work considering this class of
deformable objects, many open problems remain. These relate to the choice and
modelling of the sensory feedback as well as the control and planning of the
interaction and manipulation strategies. Most importantly, there is no
structured approach for studying and assessing different approaches that may
bridge the gap between the robotics community and textile production industry.
To this end, we outline a textile taxonomy considering fiber types and
production methods, commonly used in textile industry. We devise datasets
according to the taxonomy, and study how robotic actions, such as pulling and
twisting of the textile samples, can be used for the classification. We also
provide important insights from the perspective of visualization and
interpretability of the gathered data
Visual Tactile Fusion Object Clustering
Object clustering, aiming at grouping similar objects into one cluster with
an unsupervised strategy, has been extensivelystudied among various data-driven
applications. However, most existing state-of-the-art object clustering methods
(e.g., single-view or multi-view clustering methods) only explore visual
information, while ignoring one of most important sensing modalities, i.e.,
tactile information which can help capture different object properties and
further boost the performance of object clustering task. To effectively benefit
both visual and tactile modalities for object clustering, in this paper, we
propose a deep Auto-Encoder-like Non-negative Matrix Factorization framework
for visual-tactile fusion clustering. Specifically, deep matrix factorization
constrained by an under-complete Auto-Encoder-like architecture is employed to
jointly learn hierarchical expression of visual-tactile fusion data, and
preserve the local structure of data generating distribution of visual and
tactile modalities. Meanwhile, a graph regularizer is introduced to capture the
intrinsic relations of data samples within each modality. Furthermore, we
propose a modality-level consensus regularizer to effectively align thevisual
and tactile data in a common subspace in which the gap between visual and
tactile data is mitigated. For the model optimization, we present an efficient
alternating minimization strategy to solve our proposed model. Finally, we
conduct extensive experiments on public datasets to verify the effectiveness of
our framework.Comment: 8 pages, 5 figure
Core dimensions of human material perception
Visually categorizing and comparing materials is crucial for our everyday behaviour. Given the dramatic variability in their visual appearance and functional significance, what organizational principles underly the internal representation of materials? To address this question, here we use a large-scale data-driven approach to uncover the core latent dimensions in our mental representation of materials. In a first step, we assembled a new image dataset (STUFF dataset) consisting of 600 photographs of 200 systematically sampled material classes. Next, we used these images to crowdsource 1.87 million triplet similarity judgments. Based on the responses, we then modelled the assumed cognitive process underlying these choices by quantifying each image as a sparse, non-negative vector in a multidimensional embedding space. The resulting embedding predicted material similarity judgments in an independent test set close to the human noise ceiling and accurately reconstructed the similarity matrix of all 600 images in the STUFF dataset. We found that representations of individual material images were captured by a combination of 36 material dimensions that were highly reproducible and interpretable, comprising perceptual (e.g., “grainy”, “blue”) as well as conceptual (e.g., “mineral”, “viscous”) dimensions. These results have broad implications for understanding material perception, its natural dimensions, and our ability to organize materials into classes