The effect of indentation force and displacement on visual perception of compliance

This paper investigates the effect of maximum indentation force and depth on people's ability to accurately discriminate compliance using indirect visual information only. Participants took part in two psychophysical experiments in which they were asked to choose the 'softest' sample out of a series of presented sample pairs. In the experiments, participants observed a computer-actuated tip indent the sample pairs to one of two conditions; maximum depth (10mm) or maximum force (4N). This indentation process simulates tool operated palpation in laparoscopic surgery. Results were used to plot psychometric functions as a measure of accuracy of compliance discriminability. A comparison indicated that participants performed best in the task where they judged samples being indented to a pre-set maximum force relying solely on visual cues, which demonstrates the effect of visual information on compliance discrimination. Results also show that indentation cues such as force and deformation depth have different effects on our ability to visually discriminate compliance. These findings will inform future work on designing a haptic feedback system capable of augmenting visual and haptic information independently for optimal compliance discrimination performance

ReSolve Research Engineering: Bespoke Technologies for Research and Innovation

Impactful empirical research is underpinned by high-quality data, and therefore, robust experimental equipment. However, unfortunately, researchers often lack the technical support they require to implement the specialist tools that their investigations demand. This can lead to severe time and cost inefficiencies and sub-optimal outcomes. These issues call for a more objective approach to implementing specialist experimental equipment. With a proven record of accomplishment working across various sectors in academic and industrial research, ReSolve Research Engineering help to transform the research process through the provision of bespoke technologies that integrate hardware, software and measurement systems

The impact of visual cues on haptic compliance discrimination using a pseudo-haptic robotic system

A psychophysical magnitude estimation experiment was set up to determine the extent of the contribution of visual feedback during haptic compliance discrimination. Subjects remotely palpated physical compliant samples using a novel pseudo-haptic feedback system which allowed for independent manipulation of visual and haptic feedback. Subjects were asked to rate the compliance of a test sample based on that of a reference sample. While visual feedback was modified by switching the physical test samples shown to participants during indentation, haptic compliance of the test samples was always identical to that of the reference sample. Any variations in haptic sensation was a result of pseudo-haptic illusions. Ratings were collated and fitted to Steven's power law as well as Weber's law. A 0.18 power exponent suggests that the system was successful in generating viscoelastic properties through variations in visual information only. A 19.6% visual change from the reference compliance was necessary in order to perceive a change in haptic compliance using the pseudo-haptic system. These findings could prove beneficial in research and educationalfacilities where advanced force feedback devices are limited or inaccessible, where the concept of pseudo-haptics could be used to simulate various mechanical properties of virtual tissue for training purposes without the needfor complicated or costly force feedback

Material perception and action : The role of material properties in object handling

This dissertation is about visual perception of material properties and their role in preparation for object handling. Usually before an object is touched or picked-up we estimate its size and shape based on visual features to plan the grip size of our hand. After we have touched the object, the grip size is adjusted according to the provided haptic feedback and the object is handled safely. Similarly, we anticipate the required grip force to handle the object without slippage, based on its visual features and prior experience with similar objects. Previous studies on object handling have mostly examined object characteristics that are typical for object recognition, e.g., size, shape, weight, but in the recent years there has been a growing interest in object characteristics that are more typical to the type of material the object is made from. That said, in a series of studies we investigated the role of perceived material properties in decision-making and object handling, in which both digitally rendered materials and real objects made of different types of materials were presented to human subjects and a humanoid robot. Paper I is a reach-to-grasp study where human subjects were examined using motion capture technology. In this study, participants grasped and lifted paper cups that varied in appearance (i.e., matte vs. glossy) and weight. Here we were interested in both the temporal and spatial components of prehension to examine the role of material properties in grip preparation, and how visual features contribute to inferred hardness before haptic feedback has become available. We found the temporal and spatial components were not exclusively governed by the expected weight of the paper cups, instead glossiness and expected hardness has a significant role as well. In paper II, which is a follow-up on Paper I, we investigated the grip force component of prehension using the same experimental stimuli as used in paper I. In a similar experimental set up, using force sensors we examined the early grip force magnitudes applied by human subjects when grasping and lifting the same paper cups as used in Paper I. Here we found that early grip force scaling was not only guided by the object weight, but the visual characteristics of the material (i.e., matte vs. glossy) had a role as well. Moreover, the results suggest that grip force scaling during the initial object lifts is guided by expected hardness that is to some extend based on visual material properties. Paper III is a visual judgment task where psychophysical measurements were used to examine how the material properties, roughness and glossiness, influence perceived bounce height and consequently perceived hardness. In a paired-comparison task, human subjects observed a bouncing ball bounce on various surface planes and judged their bounce height. Here we investigated, what combination of surface properties, i.e., roughness or glossiness, makes a surface plane to be perceived bounceable. The results demonstrate that surface planes with rough properties are believed to afford higher bounce heights for the bouncing ball, compared to surface planes with smooth properties. Interestingly, adding shiny properties to the rough and smooth surface planes, reduced the judged difference, as if surface planes with gloss are believed to afford higher bounce heights irrespective of how smooth or rough the surface plane is beneath. This suggests that perceived bounce height involves not only the physical elements of the bounce height, but also the visual characteristics of the material properties of the surface planes the ball bounces on. In paper IV we investigated the development of material knowledge using a robotic system. A humanoid robot explored real objects made of different types of materials, using both camera and haptic systems. The objects varied in visual appearances (e.g., texture, color, shape, size), weight, and hardness, and in two experiments, the robot picked up and placed the experimental objects several times using its arm. Here we used the haptic signals from the servos controlling the arm and the shoulder of the robot, to obtain measurements of the weight and hardness of the objects, and the camera system to collect data on the visual features of the objects. After the robot had repeatedly explored the objects, an associative learning model was created based on the training data to demonstrate how the robotic system could produce multi-modal mapping between the visual and haptic features of the objects. In sum, in this thesis we show that visual material properties and prior knowledge of how materials look like and behave like has a significant role in action planning

Determining the Contribution of Visual and Haptic Cues during Compliance Discrimination in the Context of Minimally Invasive Surgery

While minimally invasive surgery is replacing open surgery in an increasing number of surgical procedures, it still poses risks such as unintended tissue damage due to reduced visual and haptic feedback. Surgeons assess tissue health by analysing mechanical properties such as compliance. The literature shows that while both types of feedback contribute to the final percept, visual information is dominant during compliance discrimination tasks. The magnitude of that contribution, however, was never quantitatively determined. To determine the effect of the type of visual feedback on compliance discrimination, a psychophysical experiment was set up using different combinations of direct and indirect visual and haptic cues. Results reiterated the significance of visual information and suggested a visio-haptic cross-modal integration. Consequently, to determine which cues contributed most to visual feedback, the impact of force and position on the ability to discriminate compliance using visual information only was assessed. Results showed that isolating force and position cues during indentation enhanced performance. Furthermore, under force and position constraints, visual information was shown to be sufficient to determine the compliance of deformable objects. A pseudo-haptic feedback system was developed to quantitatively determine the contribution of visual feedback during compliance discrimination. A psychophysical experiment showed that the system realistically simulated viscoelastic behaviour of compliant objects. Through a magnitude estimation experiment, the pseudo-haptic system was shown to be successful at generating haptic sensations of compliance during stimuli indentation only by modifying the visual feedback presented to participants. This can be implemented in research and educational facilities where advanced force feedback devices are inaccessible. Moreover, it can be incorporated into virtual reality simulators to enhance force ranges. Future work will assess the value of visual cue augmentation in more complicated surgical tasks