Active Clothing Material Perception using Tactile Sensing and Deep Learning

Humans represent and discriminate the objects in the same category using their properties, and an intelligent robot should be able to do the same. In this paper, we build a robot system that can autonomously perceive the object properties through touch. We work on the common object category of clothing. The robot moves under the guidance of an external Kinect sensor, and squeezes the clothes with a GelSight tactile sensor, then it recognizes the 11 properties of the clothing according to the tactile data. Those properties include the physical properties, like thickness, fuzziness, softness and durability, and semantic properties, like wearing season and preferred washing methods. We collect a dataset of 153 varied pieces of clothes, and conduct 6616 robot exploring iterations on them. To extract the useful information from the high-dimensional sensory output, we applied Convolutional Neural Networks (CNN) on the tactile data for recognizing the clothing properties, and on the Kinect depth images for selecting exploration locations. Experiments show that using the trained neural networks, the robot can autonomously explore the unknown clothes and learn their properties. This work proposes a new framework for active tactile perception system with vision-touch system, and has potential to enable robots to help humans with varied clothing related housework.Comment: ICRA 2018 accepte

Expanding Haptic Workspace for Coupled-Object Manipulation

Haptic force-feedback offers a valuable cue in exploration and manipulation of virtual environments. However, grounding of many commercial kinesthetic haptic devices limits the workspace accessible using a purely position-control scheme. The bubble technique has been recently presented as a method for expanding the user’s haptic workspace. The bubble technique is a hybrid position-rate control system in which a volume, or “bubble,” is defined entirely within the physical workspace of the haptic device. When the device’s end effector is within this bubble, interaction is through position control. When exiting this volume, an elastic restoring force is rendered, and a rate is applied that moves the virtual accessible workspace. Existing work on the bubble technique focuses on point-based touching tasks. When the bubble technique is applied to simulations where the user is grasping virtual objects with part-part collision detection, unforeseen interaction problems surface. This paper discusses three details of the user experience of coupled-object manipulation with the bubble technique. A few preliminary methods of addressing these interaction challenges are introduced

Enabling audio-haptics

This thesis deals with possible solutions to facilitate orientation, navigation and overview of non-visual interfaces and virtual environments with the help of sound in combination with force-feedback haptics. Applications with haptic force-feedback, s

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Tactile perception of randomly rough surfaces

Most everyday surfaces are randomly rough and self-similar on sufficiently small scales. We investigated the tactile perception of randomly rough surfaces using 3D-printed samples, where the topographic structure and the statistical properties of scale-dependent roughness were varied independently. We found that the tactile perception of similarity between surfaces was dominated by the statistical micro-scale roughness rather than by their topographic resemblance. Participants were able to notice differences in the Hurst roughness exponent of 0.2, or a difference in surface curvature of 0.8 mm−1 for surfaces with curvatures between 1 and 3 mm−1. In contrast, visual perception of similarity between color-coded images of the surface height was dominated by their topographic resemblance. We conclude that vibration cues from roughness at the length scale of the finger ridge distance distract the participants from including the topography into the judgement of similarity. The interaction between surface asperities and fingertip skin led to higher friction for higher micro-scale roughness. Individual friction data allowed us to construct a psychometric curve which relates similarity decisions to differences in friction. Participants noticed differences in the friction coefficient as small as 0.035 for samples with friction coefficients between 0.34 and 0.45

Tactile perception of randomly rough surfaces

Most everyday surfaces are randomly rough and self-similar on sufficiently small scales. We investigated the tactile perception of randomly rough surfaces using 3D-printed samples, where the topographic structure and the statistical properties of scale-dependent roughness were varied independently. We found that the tactile perception of similarity between surfaces was dominated by the statistical micro-scale roughness rather than by their topographic resemblance. Participants were able to notice differences in the Hurst roughness exponent of 0.2, or a difference in surface curvature of 0.8 mm−1 for surfaces with curvatures between 1 and 3 mm−1. In contrast, visual perception of similarity between color-coded images of the surface height was dominated by their topographic resemblance. We conclude that vibration cues from roughness at the length scale of the finger ridge distance distract the participants from including the topography into the judgement of similarity. The interaction between surface asperities and fingertip skin led to higher friction for higher micro-scale roughness. Individual friction data allowed us to construct a psychometric curve which relates similarity decisions to differences in friction. Participants noticed differences in the friction coefficient as small as 0.035 for samples with friction coefficients between 0.34 and 0.45