Design and characterization of a fabric-based softness display

To enable a realistic tactile interaction with remote or virtual objects, softness information represents a fundamental property to be rendered via haptic devices. What is challenging is to reduce the complexity of such an information as it arises from contact mechanics and to find suitable simplifications that can lead an effective development of softness displays. A possible approach is to surrogate detailed tactile cues with information on the rate of spread of the contact area between the object and the finger as the contact force increases, i.e. force/area relation. This paradigm is called contact area spread rate. In this paper we discuss how such a paradigm has inspired the design of a tactile device (hereinafter referred to as Fabric Yielding Display, FYD-2), which exploits the elasticity of a fabric to mimic different levels of stiffness, while the contact area on the finger indenting the fabric is measured. In this manner, the FYD-2 can be controlled to reproduce force-area characteristics. In this work, we describe the FYD-2 architecture and report a psychophysical characterization. FYD-2 is shown to be able to accurately reproduce force-area curves of typical objects and to enable a reliable softness discrimination in human users

Spatial-Temporal Characteristics of Multisensory Integration

abstract: We experience spatial separation and temporal asynchrony between visual and haptic information in many virtual-reality, augmented-reality, or teleoperation systems. Three studies were conducted to examine the spatial and temporal characteristic of multisensory integration. Participants interacted with virtual springs using both visual and haptic senses, and their perception of stiffness and ability to differentiate stiffness were measured. The results revealed that a constant visual delay increased the perceived stiffness, while a variable visual delay made participants depend more on the haptic sensations in stiffness perception. We also found that participants judged stiffness stiffer when they interact with virtual springs at faster speeds, and interaction speed was positively correlated with stiffness overestimation. In addition, it has been found that participants could learn an association between visual and haptic inputs despite the fact that they were spatially separated, resulting in the improvement of typing performance. These results show the limitations of Maximum-Likelihood Estimation model, suggesting that a Bayesian inference model should be used.Dissertation/ThesisDoctoral Dissertation Human Systems Engineering 201

Stiffness Discrimination with Visual and Proprioceptive Cues

This study compares the Weber fraction for human perception of stiffness among three conditions: vision, proprioceptive motion feedback, and their combination. To make comparisons between these feedback conditions, a novel haptic device was designed that senses the spring behavior through encoder and force measurements, and implements a controller to render linear virtual springs so that the stimuli displayed haptically could be compared with their visual counterparts. The custom-designed, torque-controlled haptic interface non-invasively controls the availability of proprioceptive motion feedback in unimpaired individuals using a virtual environment. When proprioception is available, the user feels an MCP joint rotation that is proportional to his or her finger force. When proprioception is not available, the actual finger is not allowed to move, but a virtual finger displayed graphically moves in proportion to the user\u27s applied force. Visual feedback is provided and removed by turning on and off this graphical display. Weber fractions were generated from an experiment in which users examined pairs of springs and attempted to identify the spring with higher stiffness. To account for slight trial-to-trial variations in the relationship between force and position in the proprioceptive feedback conditions, our analysis uses measurements of the actual rendered stiffness, rather than the commanded stiffness. Results for 10 users give average Weber fractions of 0.056 for vision, 0.036 for proprioception, and 0.039 for their combination, indicating that proprioception is important for stiffness perception for this experimental setup. The long-term goal of this research is to motivate and develop methods for proprioception feedback to wearers of dexterous upper-limb prostheses

Doctor of Philosophy

dissertationVirtual reality is becoming a common technology with applications in fields such as medical training, product development, and entertainment. Providing haptic (sense of touch) information along with visual and audio information can create an immersive vi

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Tactile displays, design and evaluation

Fritschi M. Tactile displays, design and evaluation. Bielefeld: Universität Bielefeld; 2016.This thesis presents the design and development of several tactile displays, as well as their eventual integration into a framework of tactile and kinesthetic stimulation. As a basis for the design of novel devices, an extensive survey of existing actuator principles and existing realizations of tactile displays is complemented by neurobiological and psychophysical findings. The work is structured along three main goals: First, novel actuator concepts are explored whose performance can match the challenging capabilities of human tactile perception. Second, novel kinematic concepts for experimental platforms are investigated that target an almost unknown sub-modality of tactile perception: The perception of shear force. Third, a setup for integrated tactile-kinesthetic displays is realized, and a first study on the psychophysical correlation between the tactile and the kinesthetic portion of haptic information is conducted. The developed devices proved to exceed human tactile capabilities and have already been used to learn more about the human tactile sense

Design of a stylus with variable tip compliance

Humans are known to modulate the impedance properties of their fingers in order to physically interact with the environment. For instance, painting or palpating fragile objects require high compliance of the fingers, while writing and measuring entails high precision position control, for which the stiffness of the fingers is increased considerably. In this thesis, we present the design, modeling, implementation, characterization and user verification of a stylus with variable tip compliance. In particular, we propose a variable stiffness mechanism as a compliant stylus that features an adjustable tip stiffness such that users can modulate compliance as needed to match the requirements of the task they perform. The variable stiffness of the stylus tip is achieved through transverse stiffness variations of axially loaded beams around their critical buckling load. Integrating an axially loaded beam with a compliant transmission mechanism, the stylus tip stiffness can be modulated over a large range. In particular, very low stiffness levels can be rendered with high fidelity, without sacrificing the mechanical integrity and load bearing capacity of the stylus. Compliant transmission mechanism of the stylus is analyzed through pseudo rigid body modeling which is a convenient and efficient way of modeling flexible elements exhibiting non-linear characteristics under large deflections. Furthermore, a novel pseudo rigid body model for a fixed-guided buckling beam that captures the iii transverse stiffness variations around the first critical buckling load is proposed and verified. These models are integrated to derive a lumped parameter model of the compliant stylus with adjustable tip stiffness. The lumped parameter model due to pseudo rigid body modeling promotes ease of analysis for design, by hiding the underlying modeling complexities of continuum mechanics from the designer. We provide experimental characterization results detailing the range of stiffness modulation achieved with several prototypes and verifying the accuracy of the equivalent pseudo rigid body model. We also present a set of human subject experiments that provide evidence in establishing the efficacy of the modulated stylus stiffness on the human performance

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web

Designing a New Tactile Display Technology and its Disability Interactions

People with visual impairments have a strong desire for a refreshable tactile interface that can provide immediate access to full page of Braille and tactile graphics. Regrettably, existing devices come at a considerable expense and remain out of reach for many. The exorbitant costs associated with current tactile displays stem from their intricate design and the multitude of components needed for their construction. This underscores the pressing need for technological innovation that can enhance tactile displays, making them more accessible and available to individuals with visual impairments. This research thesis delves into the development of a novel tactile display technology known as Tacilia. This technology's necessity and prerequisites are informed by in-depth qualitative engagements with students who have visual impairments, alongside a systematic analysis of the prevailing architectures underpinning existing tactile display technologies. The evolution of Tacilia unfolds through iterative processes encompassing conceptualisation, prototyping, and evaluation. With Tacilia, three distinct products and interactive experiences are explored, empowering individuals to manually draw tactile graphics, generate digitally designed media through printing, and display these creations on a dynamic pin array display. This innovation underscores Tacilia's capability to streamline the creation of refreshable tactile displays, rendering them more fitting, usable, and economically viable for people with visual impairments