Advancing proxy-based haptic feedback in virtual reality

This thesis advances haptic feedback for Virtual Reality (VR). Our work is guided by Sutherland's 1965 vision of the ultimate display, which calls for VR systems to control the existence of matter. To push towards this vision, we build upon proxy-based haptic feedback, a technique characterized by the use of passive tangible props. The goal of this thesis is to tackle the central drawback of this approach, namely, its inflexibility, which yet hinders it to fulfill the vision of the ultimate display. Guided by four research questions, we first showcase the applicability of proxy-based VR haptics by employing the technique for data exploration. We then extend the VR system's control over users' haptic impressions in three steps. First, we contribute the class of Dynamic Passive Haptic Feedback (DPHF) alongside two novel concepts for conveying kinesthetic properties, like virtual weight and shape, through weight-shifting and drag-changing proxies. Conceptually orthogonal to this, we study how visual-haptic illusions can be leveraged to unnoticeably redirect the user's hand when reaching towards props. Here, we contribute a novel perception-inspired algorithm for Body Warping-based Hand Redirection (HR), an open-source framework for HR, and psychophysical insights. The thesis concludes by proving that the combination of DPHF and HR can outperform the individual techniques in terms of the achievable flexibility of the proxy-based haptic feedback.Diese Arbeit widmet sich haptischem Feedback für Virtual Reality (VR) und ist inspiriert von Sutherlands Vision des ultimativen Displays, welche VR-Systemen die Fähigkeit zuschreibt, Materie kontrollieren zu können. Um dieser Vision näher zu kommen, baut die Arbeit auf dem Konzept proxy-basierter Haptik auf, bei der haptische Eindrücke durch anfassbare Requisiten vermittelt werden. Ziel ist es, diesem Ansatz die für die Realisierung eines ultimativen Displays nötige Flexibilität zu verleihen. Dazu bearbeiten wir vier Forschungsfragen und zeigen zunächst die Anwendbarkeit proxy-basierter Haptik durch den Einsatz der Technik zur Datenexploration. Anschließend untersuchen wir in drei Schritten, wie VR-Systeme mehr Kontrolle über haptische Eindrücke von Nutzern erhalten können. Hierzu stellen wir Dynamic Passive Haptic Feedback (DPHF) vor, sowie zwei Verfahren, die kinästhetische Eindrücke wie virtuelles Gewicht und Form durch Gewichtsverlagerung und Veränderung des Luftwiderstandes von Requisiten vermitteln. Zusätzlich untersuchen wir, wie visuell-haptische Illusionen die Hand des Nutzers beim Greifen nach Requisiten unbemerkt umlenken können. Dabei stellen wir einen neuen Algorithmus zur Body Warping-based Hand Redirection (HR), ein Open-Source-Framework, sowie psychophysische Erkenntnisse vor. Abschließend zeigen wir, dass die Kombination von DPHF und HR proxy-basierte Haptik noch flexibler machen kann, als es die einzelnen Techniken alleine können

Understanding interaction mechanics in touchless target selection

Indiana University-Purdue University Indianapolis (IUPUI)We use gestures frequently in daily life—to interact with people, pets, or objects. But interacting with computers using mid-air gestures continues to challenge the design of touchless systems. Traditional approaches to touchless interaction focus on exploring gesture inputs and evaluating user interfaces. I shift the focus from gesture elicitation and interface evaluation to touchless interaction mechanics. I argue for a novel approach to generate design guidelines for touchless systems: to use fundamental interaction principles, instead of a reactive adaptation to the sensing technology. In five sets of experiments, I explore visual and pseudo-haptic feedback, motor intuitiveness, handedness, and perceptual Gestalt effects. Particularly, I study the interaction mechanics in touchless target selection. To that end, I introduce two novel interaction techniques: touchless circular menus that allow command selection using directional strokes and interface topographies that use pseudo-haptic feedback to guide steering–targeting tasks. Results illuminate different facets of touchless interaction mechanics. For example, motor-intuitive touchless interactions explain how our sensorimotor abilities inform touchless interface affordances: we often make a holistic oblique gesture instead of several orthogonal hand gestures while reaching toward a distant display. Following the Gestalt theory of visual perception, we found similarity between user interface (UI) components decreased user accuracy while good continuity made users faster. Other findings include hemispheric asymmetry affecting transfer of training between dominant and nondominant hands and pseudo-haptic feedback improving touchless accuracy. The results of this dissertation contribute design guidelines for future touchless systems. Practical applications of this work include the use of touchless interaction techniques in various domains, such as entertainment, consumer appliances, surgery, patient-centric health settings, smart cities, interactive visualization, and collaboration

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Aspheric intraocular lens in cataract surgery

This thesis, by prior publication, encompasses an overview and critical analysis of 6 publications on aspheric intraocular lenses (IOLs) carried out at St. Thomas’ Hospital, Guy’s & St. Thomas’ NHS Foundation Trust, London between 2006 2012. The focus is on two areas: the visual and optical performance of aspheric IOLs with different values of asphericity, and their effect on posterior capsule opacification (PCO). Chapter 1 describes the history of IOLs and the evolution of aspheric IOLs. It also presents relevant optical concepts. The three publications presented in Chapter 2 compare aspheric IOLs with spherical IOLs. The results come from two prospective, randomised, fellow-eye comparison studies: one comparing a spherical Alcon AcrySof SN60AT versus the aspheric Alcon AcrySof SN60WF (with negative asphericity), which partially corrects corneal spherical aberrations, and the other comparing the Zeiss AcriSmart 36A (with negative asphericity), which partially corrects corneal spherical aberrations, and the Bausch & Lomb Akreos MI60 (with neutral asphericity), which has zero spherical aberration. The third publication reports changes in vertical coma after implantation of all the above lenses and additionally includes data on two spherical IOLs from a previous study by our group (the three piece Alcon MA60AC IOL and the plate haptic HumanOptic MC611MI IOL). For this study, the data were divided according to their asphericity and not design. To avoid confounding from aberrometric differences due to astigmatism and surgical techniques, standard incision sizes of 2.75 mm and 2.4 mm were used in the two felloweye, randomised studies and a single surgeon performed all the surgeries using same incision size for each study. Results demonstrated that the aspheric IOLs significantly reduced spherical aberration, improved mesopic contrast sensitivity and reduced depth-of-focus. Asphericity differences up to 20 μm were not associated with depth-of-focus and the degree of asphericity was not associated with best-corrected distance visual acuity. The vertical coma varied within IOL groups of the same asphericity but there was no statistically significant difference between the two spherical IOLs (AcrySof SN60AT and AcrySof MA60AC). Further critical analysis of our data already published showed no statistically significant difference in mean vertical coma between the AcrySof MA60AC (-0.060 ± 0.211μm) with that of Akreos MI60 (- 0.042 ± 0.148μm) (p=0.70) and AcriSmart 36A (-0.034 ± 0.141μm) (p=0.58) even though the AcrySof MA60AC, Akreos MI60 and AcriSmart 36A IOLs had different asphericity. This may be due to the difference in the IOL designs, decentration and difference in the sample sizes between groups. Chapter 3 discusses the findings of three publications related to posterior capsule opacification (PCO). The first publication is an in vitro study assessing posterior optic square edges of various commercially available spherical and aspherical IOLs. Results demonstrated hydrophilic IOLs had less sharp square edges compared to hydrophobic IOLs. Although these in vitro results are potentially significant, there are currently no in vivo studies on the lenses used to compare our results with. The other two publications are based on the PCO outcomes from the prospective, randomised, fellow-eye studies described above. There was no difference in the PCO rates between hydrophobic spherical AcrySof SN60AT and aspheric AcrySof SN60WF IOLs of the same design at 2 years. In contrast, we found a significant increase in the PCO with both the hydrophilic acrylic AcriSmart 36A and Akreos MI60 IOLs (one with negative asphericity and one with neutral asphericity). Variation in asphericity did not appear to be an important factor contributing to the PCO but it was apparent that the edge design and material of the IOL were important with regards to the PCO formation. Finally, since these publications, some of manufacturers have changed their IOL models and today a majority of the IOLs have are aspheric and have a sharp edge profile. The overall, benefit of using aspheric IOLs seems to be limited as it is dependent on the natural pupil size

Redirected Touching

In immersive virtual environments, virtual objects cannot be touched. One solution is to use passive haptics - physical props to which virtual objects are registered. The result is compelling; when a user reaches out with a virtual hand to touch a virtual object, her real hand touches and feels a real object. However, for every virtual object to be touched, there must be an analogous physical prop. In the limit, an entire real-world infrastructure would need to be built and changed whenever a virtual scene is changed. Virtual objects and passive haptics have historically been mapped one-to-one. I demonstrate that the mapping need not be one-to-one. One can make a single passive real object provide useful haptic feedback for many virtual objects by exploiting human perception. I developed and investigated three categories of such techniques: 1. Move the virtual world to align different virtual objects in turn with the same real object 2. Move a virtual object into alignment with a real object 3. Map real hand motion to different virtual hand motion, e.g., when the real hand traces a real object, the virtual hand traces a differently shaped virtual object. The first two techniques were investigated for feasibility, and the third was explored more deeply. The first technique (Redirected Passive Haptics) enables users to touch multiple instances of a virtual object, with haptic feedback provided by a single real object. The second technique (The Haptic Hand) attaches a larger-than-hand virtual user interface to the non-dominant hand, mapping the currently relevant part of the interface onto the palm. The third technique (Redirected Touching) warps virtual space to map many differently shaped virtual objects onto a single real object, introducing a discrepancy between real and virtual hand motions. Two studies investigated the technique's effect on task performance and its potential for use in aircraft cockpit procedures training. Users adapt rather quickly to real-virtual discrepancy, and after adaptation, users perform no worse with discrepant virtual objects than with one-to-one virtual objects. Redirected Touching shows promise for training and entertainment applications.Doctor of Philosoph

Embodiment Sensitivity to Movement Distortion and Perspective Taking in Virtual Reality

Despite recent technological improvements of immersive technologies, Virtual Reality suffers from severe intrinsic limitations, in particular the immateriality of the visible 3D environment. Typically, any simulation and manipulation in a cluttered environment would ideally require providing feedback of collisions to every body parts (arms, legs, trunk, etc.) and not only to the hands as has been originally explored with haptic feedback. This thesis addresses these limitations by relying on a cross modal perception and cognitive approach instead of haptic or force feedback. We base our design on scientific knowledge of bodily self-consciousness and embodiment. It is known that the instantaneous experience of embodiment emerges from the coherent multisensory integration of bodily signals taking place in the brain, and that altering this mechanism can temporarily change how one perceives properties of their own body. This mechanism is at stake during a VR simulation, and this thesis explores the new venues of interaction design based on these fundamental scientific findings about the embodied self. In particular, we explore the use of third person perspective (3PP) instead of permanently offering the traditional first person perspective (1PP), and we manipulate the user-avatar motor mapping to achieve a broader range of interactions while maintaining embodiment. We are guided by two principles, to explore the extent to which we can enhance VR interaction through the manipulation of bodily aspects, and to identify the extent to which a given manipulation affects the embodiment of a virtual body. Our results provide new evidence supporting strong embodiment of a virtual body even when viewed from 3PP, and in particular that voluntarily alternating point of view between 1PP and 3PP is not detrimental to the experience of ownership over the virtual body. Moreover, detailed analysis of movement quality show highly similar reaching behavior in both perspective conditions, and only obvious advantages or disadvantages of each perspective depending on the situation (e.g. occlusion of target by the body in 3PP, limited field of view in 1PP). We also show that subjects are insensitive to visuo-proprioceptive movement distortions when the nature of the distortion was not made explicit, and that subjects are biased toward self-attributing distorted movements that make the task easier

The application of three-dimensional mass-spring structures in the real-time simulation of sheet materials for computer generated imagery

Despite the resources devoted to computer graphics technology over the last 40 years, there is still a need to increase the realism with which flexible materials are simulated. However, to date reported methods are restricted in their application by their use of two-dimensional structures and implicit integration methods that lend themselves to modelling cloth-like sheets but not stiffer, thicker materials in which bending moments play a significant role. This thesis presents a real-time, computationally efficient environment for simulations of sheet materials. The approach described differs from other techniques principally through its novel use of multilayer sheet structures. In addition to more accurately modelling bending moment effects, it also allows the effects of increased temperature within the environment to be simulated. Limitations of this approach include the increased difficulties of calibrating a realistic and stable simulation compared to implicit based methods. A series of experiments are conducted to establish the effectiveness of the technique, evaluating the suitability of different integration methods, sheet structures, and simulation parameters, before conducting a Human Computer Interaction (HCI) based evaluation to establish the effectiveness with which the technique can produce credible simulations. These results are also compared against a system that utilises an established method for sheet simulation and a hybrid solution that combines the use of 3D (i.e. multilayer) lattice structures with the recognised sheet simulation approach. The results suggest that the use of a three-dimensional structure does provide a level of enhanced realism when simulating stiff laminar materials although the best overall results were achieved through the use of the hybrid model