Six Degrees-of-Freedom Haptic Interaction with Fluids

International audienceIn this work, we propose a novel approach that allows real-time six Degrees of Freedom (DoF) haptic interaction with fluids of variable viscosity. Our haptic rendering technique, based on a Smoothed-Particle Hydrodynamics physical model, provides a realistic haptic feedback through physically based forces. 6DoF haptic interaction with fluids is made possible thanks to a new coupling scheme and a unified particle model, allowing the use of arbitrary-shaped rigid bodies. Particularly, fluid containers can be created to hold fluid and hence transmit to the user force feedback coming from fluid stirring, pouring, shaking, and scooping, to name a few. In addition, we adapted an existing visual rendering algorithm to meet the frame rate requirements of the haptic algorithms. We evaluate and illustrate the main features of our approach through different scenarios, highlighting the 6DoF haptic feedback and the use of containers

An augmented haptic interface as applied to flow visualization

A novel 3D computer interface is proposed in which a physical handle containing force sensors and capable of simulating virtual touch through force feedback is coupled to a variety of virtual tools in a 3D virtual environment. The visual appearance of each tool reflects its capabilities. At one moment a user might feel they are holding a virtual grabber, activated by squeezing, and at another moment they are holding a virtual turntable activated by physical motion of a virtual wheel. In this way it is intended that form and function can be combined so that users rapidly learn the functional capabilities of the tools and retain this learning. It is also intended that the tools be easy to use because of intuitive mappings of forces to actions. A virtual environment is constructed to test this concept, and an evaluation of the interface conducted

Real-time hybrid cutting with dynamic fluid visualization for virtual surgery

It is widely accepted that a reform in medical teaching must be made to meet today's high volume training requirements. Virtual simulation offers a potential method of providing such trainings and some current medical training simulations integrate haptic and visual feedback to enhance procedure learning. The purpose of this project is to explore the capability of Virtual Reality (VR) technology to develop a training simulator for surgical cutting and bleeding in a general surgery

Factors Affecting Human Force Perception and Performance in Haptic-Enabled Virtual Environments

Haptic technology enables computer users to touch and/or manipulate virtual objects in virtual environments (VEs). Similar to other human-in-the-loop applications, haptic applications require interactions between humans and computers. Thus, human-factors studies are required to recognize the limitations and capabilities of the user. This thesis establishes human-factors criteria to improve various haptic applications such as perception-based haptic compression techniques and haptic-enabled computer-aided design (CAD). Today, data compression plays a significant role in the transmission of haptic information since the efficient use of the available bandwidth is a concern. Most lossy haptic compression techniques rely on the limitations of human force perception, and this is used in the design of perception-based haptic compression techniques. Researchers have studied force perception when a user is in static interaction with a stationary object. This thesis focuses on cases where the human user and the object are in relative motion. The limitations of force perception are quantified using psychophysical methods, and the effects of several factors, including user hand velocity and sensory adaptation, are investigated. The results indicate that fewer haptic details need to be calculated or transmitted when the user's hand is in motion. In traditional CAD systems, users usually design virtual prototypes using a mouse via their vision system only, and it is difficult to design curved surfaces due to the number, shape, and position of the curves. Adding haptics to CAD systems enables users to explore and manipulate virtual objects using the sense of touch. In addition, human performance is important in CAD environments. To maintain the accuracy, active haptic manipulation of the user response can be incorporated in CAD applications. This thesis investigates the effect of forces on the accuracy of movement in VEs. The results indicate that factors such as the base force intensity and force increment/decrement can be incorporated in the control of users' movements in VEs. In other words, we can pull/push the users' hands by increasing/decreasing the force without the users being aware of it

Factors Affecting Human Force Perception and Performance in Haptic-Enabled Virtual Environments

Haptic technology enables computer users to touch and/or manipulate virtual objects in virtual environments (VEs). Similar to other human-in-the-loop applications, haptic applications require interactions between humans and computers. Thus, human-factors studies are required to recognize the limitations and capabilities of the user. This thesis establishes human-factors criteria to improve various haptic applications such as perception-based haptic compression techniques and haptic-enabled computer-aided design (CAD). Today, data compression plays a significant role in the transmission of haptic information since the efficient use of the available bandwidth is a concern. Most lossy haptic compression techniques rely on the limitations of human force perception, and this is used in the design of perception-based haptic compression techniques. Researchers have studied force perception when a user is in static interaction with a stationary object. This thesis focuses on cases where the human user and the object are in relative motion. The limitations of force perception are quantified using psychophysical methods, and the effects of several factors, including user hand velocity and sensory adaptation, are investigated. The results indicate that fewer haptic details need to be calculated or transmitted when the user's hand is in motion. In traditional CAD systems, users usually design virtual prototypes using a mouse via their vision system only, and it is difficult to design curved surfaces due to the number, shape, and position of the curves. Adding haptics to CAD systems enables users to explore and manipulate virtual objects using the sense of touch. In addition, human performance is important in CAD environments. To maintain the accuracy, active haptic manipulation of the user response can be incorporated in CAD applications. This thesis investigates the effect of forces on the accuracy of movement in VEs. The results indicate that factors such as the base force intensity and force increment/decrement can be incorporated in the control of users' movements in VEs. In other words, we can pull/push the users' hands by increasing/decreasing the force without the users being aware of it

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web