Electrostatic Friction Displays to Enhance Touchscreen Experience

Touchscreens are versatile devices that can display visual content and receive touch input, but they lack the ability to provide programmable tactile feedback. This limitation has been addressed by a few approaches generally called surface haptics technology. This technology modulates the friction between a user’s fingertip and a touchscreen surface to create different tactile sensations when the finger explores the touchscreen. This functionality enables the user to see and feel digital content simultaneously, leading to improved usability and user experiences. One major approach in surface haptics relies on the electrostatic force induced between the finger and an insulating surface on the touchscreen by supplying high AC voltage. The use of AC also induces a vibrational sensation called electrovibration to the user. Electrostatic friction displays require only electrical components and provide uniform friction over the screen. This tactile feedback technology not only allows easy and lightweight integration into touchscreen devices but also provides dynamic, rich, and satisfactory user interfaces. In this chapter, we review the fundamental operation of the electrovibration technology as well as applications have been built upon

Realistic tool-tissue interaction models for surgical simulation and planning

Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in pre- and intra-operative surgical planning. Realistic modeling of medical interventions involving tool-tissue interactions has been considered to be a key requirement in the development of high-fidelity simulators and planners. The soft-tissue constitutive laws, organ geometry and boundary conditions imposed by the connective tissues surrounding the organ, and the shape of the surgical tool interacting with the organ are some of the factors that govern the accuracy of medical intervention planning.\ud \ud This thesis is divided into three parts. First, we compare the accuracy of linear and nonlinear constitutive laws for tissue. An important consequence of nonlinear models is the Poynting effect, in which shearing of tissue results in normal force; this effect is not seen in a linear elastic model. The magnitude of the normal force for myocardial tissue is shown to be larger than the human contact force discrimination threshold. Further, in order to investigate and quantify the role of the Poynting effect on material discrimination, we perform a multidimensional scaling study. Second, we consider the effects of organ geometry and boundary constraints in needle path planning. Using medical images and tissue mechanical properties, we develop a model of the prostate and surrounding organs. We show that, for needle procedures such as biopsy or brachytherapy, organ geometry and boundary constraints have more impact on target motion than tissue material parameters. Finally, we investigate the effects surgical tool shape on the accuracy of medical intervention planning. We consider the specific case of robotic needle steering, in which asymmetry of a bevel-tip needle results in the needle naturally bending when it is inserted into soft tissue. We present an analytical and finite element (FE) model for the loads developed at the bevel tip during needle-tissue interaction. The analytical model explains trends observed in the experiments. We incorporated physical parameters (rupture toughness and nonlinear material elasticity) into the FE model that included both contact and cohesive zone models to simulate tissue cleavage. The model shows that the tip forces are sensitive to the rupture toughness. In order to model the mechanics of deflection of the needle, we use an energy-based formulation that incorporates tissue-specific parameters such as rupture toughness, nonlinear material elasticity, and interaction stiffness, and needle geometric and material properties. Simulation results follow similar trends (deflection and radius of curvature) to those observed in macroscopic experimental studies of a robot-driven needle interacting with gels

Haptic Discrimination and Matching of Viscosity

In three experiments, viscosity perception of liquids using the sense of touch was studied. The first two were discrimination experiments in which Weber fractions were determined for a number of viscosities spanning the range of what is encountered in daily life, and for two ways of perceiving viscosity (stirring with a spatula or with the index finger). For high viscosities, Weber fractions were around 0.3, whereas they increased for lower viscosities. For low viscosities, discrimination performance was much worse with the finger than with the spatula. In the third experiment, subjects matched liquids perceived with these two methods, which resulted in biases of around 80 %. Control experiments and force measurements were performed to find an explanation for these results. It was concluded that the relationship between perceived and physical viscosity is steeper for stirring liquids with a spatula than stirring with the finger

Towards an efficient haptic rendering using data-driven modeling

This thesis focuses on the optimisation of haptic rendering of interactions with deformable models. The research demonstrated that data-driven techniques can produce a real-time, accurate and complex simulation experience. Applications include, but not limited to, virtual training, rapid prototyping, virtual presence, and entertainment

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web

Design and development of a VR system for exploration of medical data using haptic rendering and high quality visualization

[no abstract

Animation, Simulation, and Control of Soft Characters using Layered Representations and Simplified Physics-based Methods

Realistic behavior of computer generated characters is key to bringing virtual environments, computer games, and other interactive applications to life. The plausibility of a virtual scene is strongly influenced by the way objects move around and interact with each other. Traditionally, actions are limited to motion capture driven or pre-scripted motion of the characters. Physics enhance the sense of realism: physical simulation is required to make objects act as expected in real life. To make gaming and virtual environments truly immersive,it is crucial to simulate the response of characters to collisions and to produce secondary effects such as skin wrinkling and muscle bulging. Unfortunately, existing techniques cannot generally achieve these effects in real time, do not address the coupled response of a character's skeleton and skin to collisions nor do they support artistic control. In this dissertation, I present interactive algorithms that enable physical simulation of deformable characters with high surface detail and support for intuitive deformation control. I propose a novel unified framework for real-time modeling of soft objects with skeletal deformations and surface deformation due to contact, and their interplay for object surfaces with up to tens of thousands of degrees of freedom.I make use of layered models to reduce computational complexity. I introduce dynamic deformation textures, which map three dimensional deformations in the deformable skin layer to a two dimensional domain for extremely efficient parallel computation of the dynamic elasticity equations and optimized hierarchical collision detection. I also enhance layered models with responsive contact handling, to support the interplay between skeletal motion and surface contact and the resulting two-way coupling effects. Finally, I present dynamic morph targets, which enable intuitive control of dynamic skin deformations at run-time by simply sculpting pose-specific surface shapes. The resulting framework enables real-time and directable simulation of soft articulated characters with frictional contact response, capturing the interplay between skeletal dynamics and complex,non-linear skin deformations

Early characterisation of neurodegeneration with high-resolution magnetic resonance elastography

This thesis contributes to recent interest within medical imaging regarding the development and clinical application of magnetic resonance elastography (MRE) to the human brain. MRE is a non-invasive phase-contrast MRI technique for measurement of brain mechanical properties in vivo, shown to reflect the composition and organisation of the complex tissue microstructure. MRE is a promising imaging biomarker for the early characterisation of neurodegeneration due to its exquisite sensitivity to variation among healthy and pathological tissue. Neurodegenerative diseases are debilitating conditions of the human nervous system for which there is currently no cure. Novel biomarkers are required to improve early detection, differential diagnosis and monitoring of disease progression, and could also ultimately improve our understanding of the pathophysiological mechanisms underlying degenerative processes. This thesis begins with a theoretical background of brain MRE and a description of the experimental considerations. A systematic review of the literature is then performed to summarise brain MRE quantitative measurements in healthy participants and to determine the success of MRE to characterise neurological disorders. This review further identified the most promising acquisition and analysis methods within the field. As such, subsequent visits to three brain MRE research centres, within the USA and Germany, enabled the acquisition of exemplar phantom and brain data to assist in discussions to refine an experimental protocol for installation at the Edinburgh Imaging Facility, QMRI (EIF-QMRI). Through collaborations with world-leading brain MRE centres, two high-resolution - yet fundamentally different - MRE pipelines were installed at the EIF-QMRI. Several optimisations were implemented to improve MRE image quality, while the clinical utility of MRE was enhanced by the novel development of a Graphical User Interface (GUI) for the optimised and automatic MRE-toanatomical coregistration and generation of MRE derived output measures. The first experimental study was performed in 6 young and 6 older healthy adults to compare the results from the two MRE pipelines to investigate test-retest agreement of the whole brain and a brain structure of interest: the hippocampal formation. The MRE protocol shown to possess superior reproducibility was subsequently applied in a second experimental study of 12 young and 12 older cognitively healthy adults. Results include finding that the MRE imaging procedure is very well tolerated across the recruited population. Novel findings include significantly softer brains in older adults both across the global cerebrum and in the majority of subcortical grey matter structures including the pallidum, putamen, caudate, and thalamus. Changes in tissue stiffness likely reflect an alteration to the strength in the composition of the tissue network. All MRE effects persist after correcting for brain structure volume suggesting changes in volume alone were not reflective of the detected MRE age differences. Interestingly, no age-related differences to tissue stiffness were found for the amygdala or hippocampus. As for brain viscosity, no group differences were detected for either the brain globally or subcortical structures, suggesting a preservation of the organisation of the tissue network in older age. The third experiment performed in this thesis finds a direct structure-function relationship in older adults between hippocampal viscosity and episodic memory as measured with verbal-paired recall. The source of this association was located to the left hippocampus, thus complementing previous literature suggesting unilateral hippocampal specialisation. Additionally, a more significant relationship was found between left hippocampal viscosity and memory after a new procedure was developed to remove voxels containing cerebrospinal fluid from the MRE analysis. Collectively, these results support the transition of brain MRE into a clinically useful neuroimaging modality that could, in particular, be used in the early characterisation of memory specific disorders such as amnestic Mild Cognitive Impairment and Alzheimer’s disease