Active Haptic Exploration of Softness: Indentation Force Is Systematically Related to Prediction, Sensation and Motivation

Active finger movements play a crucial role in natural haptic perception. For the perception of different haptic properties people use different well-chosen movement schemes (Lederman and Klatzky, 1987). The haptic property of softness is stereotypically judged by repeatedly pressing one’s finger against an objects’ surface, actively indenting the object. It has been shown that people adjust the peak indentation forces of their pressing movements to the expected stimulus’ softness in order to improve perception (Kaim and Drewing, 2011). Here, we aim to clarify the mechanisms underlying such adjustments. We disentangle how people modulate executed peak indentation forces depending on predictive vs. sensory signals to softness, and investigate the influence of the participants’ motivational state on movement adjustments. In Experiment 1, participants performed a two alternative forced-choice (2AFC) softness discrimination task for stimulus pairs from one of four softness categories. We manipulated the predictability of the softness category. Either all stimuli of the same category were presented in a blocked fashion, which allowed predicting the softness category of the upcoming pair (predictive signals high), or stimuli from different categories were randomly intermixed, which made prediction impossible (predictive signals low). Sensory signals to softness category of the two stimuli in a pair are gathered during exploration. We contrasted the first indentation (sensory signals low) and last indentation (sensory signals high) in order to examine the effect of sensory signals. The results demonstrate that participants systematically apply lower forces when softer objects (as compared to harder objects) are indicated by predictive signals. Notably, sensory signals seemed to be not as relevant as predictive signals. However, in Experiment 2, we manipulated participant motivation by introducing rewards for good performance, and showed that the use of sensory information for movement adjustments can be fostered by high motivation. Overall, the present study demonstrates that exploratory movements are adjusted to the actual perceptual situation and that in the process of fine-tuning, closed- and open-loop mechanisms interact, with varying contributions depending on the observer’s motivation

Integration of serial sensory information in haptic perception of softness.

Redundant estimates of an environmental property derived simultaneously from different senses or cues are typically integrated according to the maximum likelihood estimation model (MLE): Sensory estimates are weighted according to their reliabilities, maximizing the percept’s reliability. Mechanisms underlying the integration of sequentially derived estimates from one sense are less clear. Here we investigate the integration of serially sampled redundant information in softness perception. We developed a method to manipulate haptically perceived softness of silicone rubber stimuli during bare-finger exploration. We then manipulated softness estimates derived from single movement segments (indentations) in a multisegmented exploration to assess their contributions to the overall percept. Participants explored two stimuli in sequence, using 2–5 indentations, and reported which stimulus felt softer. Estimates of the first stimulus’s softness contributed to the judgments similarly, whereas for the second stimulus estimates from later compared to earlier indentations contributed less. In line with unequal weighting, the percept’s reliability increased with increasing exploration length less than was predicted by the MLE model. This pattern of results is well explained by assuming that the representation of the first stimulus fades when the second stimulus is explored, which fits with a neurophysiological model of perceptual decisions (Deco, Rolls, & Romo, 2010)

Temporal modulations of contact force during haptic surface exploration

Individuals constantly modulate their exploratory movements and adapt their internal hypotheses to incoming sensory information to achieve a thorough and realistic percept. Perception depends on the exploratory movements as well as influencing them. While this seems to be common sense, scientifically we know very little about the temporal dynamics during haptic exploration. To address this, we investigated the exploratory force modulations of two groups of healthy young adults during the exploration of grated surfaces with differing detection difficulty during successive (n = 20) and random stimulus presentation (n = 20). Results showed that exploratory force depended on stimulus properties and increased with increasing detection difficulty. Both experiments yielded the same direction of results with slightly smaller effects in the random stimulus presentation group. Across exploration time average fingertip force also increased. The biggest increase occurred systematically at the beginning (within the first 40 percent) of exploration time per stimulus indicating that most critical information is received during the initial contact phase and is directly transformed into the exploration procedure and force application. Furthermore, video-analyses and comparisons to our high temporal resolution data revealed strong dynamic changes in pressure application during test stimulus exploration with differences in the force dynamics and exploration strategies of simple and difficult stimuli

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Modeling of frictional forces during bare-finger interactions with solid surfaces

Touching an object with our fingers yields frictional forces that allow us to perceive and explore its texture, shape, and other features, facilitating grasping and manipulation. While the relevance of dynamic frictional forces to sensory and motor function in the hand is well established, the way that they reflect the shape, features, and composition of touched objects is poorly understood. Haptic displays -electronic interfaces for stimulating the sense of touch- often aim to elicit the perceptual experience of touching real surfaces by delivering forces to the fingers that mimic those felt when touching real surfaces. However, the design and applications of such displays have been limited by the lack of knowledge about what forces are felt during real touch interactions. This represents a major gap in current knowledge about tactile function and haptic engineering. This dissertation addresses some aspects that would assist in their understanding. The goal of this research was to measure, characterize, and model frictional forces produced by a bare finger sliding over surfaces of multiple shapes. The major contributions of this work are (1) the design and development of a sensing system for capturing fingertip motion and forces during tactile exploration of real surfaces; (2) measurement and characterization of contact forces and the deformation of finger tissues during sliding over relief surfaces; (3) the development of a low order model of frictional force production based on surface specifications; (4) the analysis and modeling of contact geometry, interfacial mechanics, and their effects in frictional force production during tactile exploration of relief surfaces. This research aims to guide the design of algorithms for the haptic rendering of surface textures and shape. Such algorithms can be used to enhance human-machine interfaces, such as touch-screen displays, by (1) enabling users to feel surface characteristics also presented visually; (2) facilitating interaction with these devices; and (3) reducing the need for visual input to interact with them.Ph.D., Electrical Engineering -- Drexel University, 201

Material perception and action : The role of material properties in object handling

This dissertation is about visual perception of material properties and their role in preparation for object handling. Usually before an object is touched or picked-up we estimate its size and shape based on visual features to plan the grip size of our hand. After we have touched the object, the grip size is adjusted according to the provided haptic feedback and the object is handled safely. Similarly, we anticipate the required grip force to handle the object without slippage, based on its visual features and prior experience with similar objects. Previous studies on object handling have mostly examined object characteristics that are typical for object recognition, e.g., size, shape, weight, but in the recent years there has been a growing interest in object characteristics that are more typical to the type of material the object is made from. That said, in a series of studies we investigated the role of perceived material properties in decision-making and object handling, in which both digitally rendered materials and real objects made of different types of materials were presented to human subjects and a humanoid robot. Paper I is a reach-to-grasp study where human subjects were examined using motion capture technology. In this study, participants grasped and lifted paper cups that varied in appearance (i.e., matte vs. glossy) and weight. Here we were interested in both the temporal and spatial components of prehension to examine the role of material properties in grip preparation, and how visual features contribute to inferred hardness before haptic feedback has become available. We found the temporal and spatial components were not exclusively governed by the expected weight of the paper cups, instead glossiness and expected hardness has a significant role as well. In paper II, which is a follow-up on Paper I, we investigated the grip force component of prehension using the same experimental stimuli as used in paper I. In a similar experimental set up, using force sensors we examined the early grip force magnitudes applied by human subjects when grasping and lifting the same paper cups as used in Paper I. Here we found that early grip force scaling was not only guided by the object weight, but the visual characteristics of the material (i.e., matte vs. glossy) had a role as well. Moreover, the results suggest that grip force scaling during the initial object lifts is guided by expected hardness that is to some extend based on visual material properties. Paper III is a visual judgment task where psychophysical measurements were used to examine how the material properties, roughness and glossiness, influence perceived bounce height and consequently perceived hardness. In a paired-comparison task, human subjects observed a bouncing ball bounce on various surface planes and judged their bounce height. Here we investigated, what combination of surface properties, i.e., roughness or glossiness, makes a surface plane to be perceived bounceable. The results demonstrate that surface planes with rough properties are believed to afford higher bounce heights for the bouncing ball, compared to surface planes with smooth properties. Interestingly, adding shiny properties to the rough and smooth surface planes, reduced the judged difference, as if surface planes with gloss are believed to afford higher bounce heights irrespective of how smooth or rough the surface plane is beneath. This suggests that perceived bounce height involves not only the physical elements of the bounce height, but also the visual characteristics of the material properties of the surface planes the ball bounces on. In paper IV we investigated the development of material knowledge using a robotic system. A humanoid robot explored real objects made of different types of materials, using both camera and haptic systems. The objects varied in visual appearances (e.g., texture, color, shape, size), weight, and hardness, and in two experiments, the robot picked up and placed the experimental objects several times using its arm. Here we used the haptic signals from the servos controlling the arm and the shoulder of the robot, to obtain measurements of the weight and hardness of the objects, and the camera system to collect data on the visual features of the objects. After the robot had repeatedly explored the objects, an associative learning model was created based on the training data to demonstrate how the robotic system could produce multi-modal mapping between the visual and haptic features of the objects. In sum, in this thesis we show that visual material properties and prior knowledge of how materials look like and behave like has a significant role in action planning

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility