LoCoMoTe – a framework for classification of natural locomotion in VR by task, technique and modality

Virtual reality (VR) research has provided overviews of locomotion techniques, how they work, their strengths and overall user experience. Considerable research has investigated new methodologies, particularly machine learning to develop redirection algorithms. To best support the development of redirection algorithms through machine learning, we must understand how best to replicate human navigation and behaviour in VR, which can be supported by the accumulation of results produced through live-user experiments. However, it can be difficult to identify, select and compare relevant research without a pre-existing framework in an ever-growing research field. Therefore, this work aimed to facilitate the ongoing structuring and comparison of the VR-based natural walking literature by providing a standardised framework for researchers to utilise. We applied thematic analysis to study methodology descriptions from 140 VR-based papers that contained live-user experiments. From this analysis, we developed the LoCoMoTe framework with three themes: navigational decisions, technique implementation, and modalities. The LoCoMoTe framework provides a standardised approach to structuring and comparing experimental conditions. The framework should be continually updated to categorise and systematise knowledge and aid in identifying research gaps and discussions

Advancing proxy-based haptic feedback in virtual reality

This thesis advances haptic feedback for Virtual Reality (VR). Our work is guided by Sutherland's 1965 vision of the ultimate display, which calls for VR systems to control the existence of matter. To push towards this vision, we build upon proxy-based haptic feedback, a technique characterized by the use of passive tangible props. The goal of this thesis is to tackle the central drawback of this approach, namely, its inflexibility, which yet hinders it to fulfill the vision of the ultimate display. Guided by four research questions, we first showcase the applicability of proxy-based VR haptics by employing the technique for data exploration. We then extend the VR system's control over users' haptic impressions in three steps. First, we contribute the class of Dynamic Passive Haptic Feedback (DPHF) alongside two novel concepts for conveying kinesthetic properties, like virtual weight and shape, through weight-shifting and drag-changing proxies. Conceptually orthogonal to this, we study how visual-haptic illusions can be leveraged to unnoticeably redirect the user's hand when reaching towards props. Here, we contribute a novel perception-inspired algorithm for Body Warping-based Hand Redirection (HR), an open-source framework for HR, and psychophysical insights. The thesis concludes by proving that the combination of DPHF and HR can outperform the individual techniques in terms of the achievable flexibility of the proxy-based haptic feedback.Diese Arbeit widmet sich haptischem Feedback für Virtual Reality (VR) und ist inspiriert von Sutherlands Vision des ultimativen Displays, welche VR-Systemen die Fähigkeit zuschreibt, Materie kontrollieren zu können. Um dieser Vision näher zu kommen, baut die Arbeit auf dem Konzept proxy-basierter Haptik auf, bei der haptische Eindrücke durch anfassbare Requisiten vermittelt werden. Ziel ist es, diesem Ansatz die für die Realisierung eines ultimativen Displays nötige Flexibilität zu verleihen. Dazu bearbeiten wir vier Forschungsfragen und zeigen zunächst die Anwendbarkeit proxy-basierter Haptik durch den Einsatz der Technik zur Datenexploration. Anschließend untersuchen wir in drei Schritten, wie VR-Systeme mehr Kontrolle über haptische Eindrücke von Nutzern erhalten können. Hierzu stellen wir Dynamic Passive Haptic Feedback (DPHF) vor, sowie zwei Verfahren, die kinästhetische Eindrücke wie virtuelles Gewicht und Form durch Gewichtsverlagerung und Veränderung des Luftwiderstandes von Requisiten vermitteln. Zusätzlich untersuchen wir, wie visuell-haptische Illusionen die Hand des Nutzers beim Greifen nach Requisiten unbemerkt umlenken können. Dabei stellen wir einen neuen Algorithmus zur Body Warping-based Hand Redirection (HR), ein Open-Source-Framework, sowie psychophysische Erkenntnisse vor. Abschließend zeigen wir, dass die Kombination von DPHF und HR proxy-basierte Haptik noch flexibler machen kann, als es die einzelnen Techniken alleine können

Object Manipulation in Virtual Reality Under Increasing Levels of Translational Gain

Room-scale Virtual Reality (VR) has become an affordable consumer reality, with applications ranging from entertainment to productivity. However, the limited physical space available for room-scale VR in the typical home or office environment poses a significant problem. To solve this, physical spaces can be extended by amplifying the mapping of physical to virtual movement (translational gain). Although amplified movement has been used since the earliest days of VR, little is known about how it influences reach-based interactions with virtual objects, now a standard feature of consumer VR. Consequently, this paper explores the picking and placing of virtual objects in VR for the first time, with translational gains of between 1x (a one-to-one mapping of a 3.5m*3.5m virtual space to the same sized physical space) and 3x (10.5m*10.5m virtual mapped to 3.5m*3.5m physical). Results show that reaching accuracy is maintained for up to 2x gain, however going beyond this diminishes accuracy and increases simulator sickness and perceived workload. We suggest gain levels of 1.5x to 1.75x can be utilized without compromising the usability of a VR task, significantly expanding the bounds of interactive room-scale VR

Multimodality in VR: A survey

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer

VR Lab: User Interaction in Virtual Environments using Space and Time Morphing

Virtual Reality (VR) allows exploring changes in space and time that would otherwise be difficult to simulate in the real world. It becomes possible to transform the virtual world by increasing or diminishing distances or playing with time delays. Analysing the adaptability of users to different space-time conditions allows studying human perception and finding the right combination of interaction paradigms. Different methods have been proposed in the literature to offer users intuitive techniques for navigating wide virtual spaces, even if restricted to small physical play areas. Other studies investigate latency tolerance, suggesting humans’ inability to detect slight discrepancies between visual and proprioceptive sensory information. These studies contribute valuable insights for designing immersive virtual experiences and interaction techniques suitable for each task. This dissertation presents the design, implementation, and evaluation of a tangible VR Lab where spatiotemporal morphing scenarios can be studied. As a case study, we restricted the scope of the research to three spatial morphing scenarios and one temporal morphing scenario. The spatial morphing scenarios compared Euclidean and hyperbolic geometries, studied size discordance between physical and virtual objects, and the representation of hands in VR. The temporal morphing scenario investigated from what visual delay the task performance is affected. The users’ adaptability to the different spatiotemporal conditions was assessed based on task completion time, questionnaires, and observed behaviours. The results revealed significant differences between Euclidean and hyperbolic spaces. They also showed a preference for handling virtual and physical objects with concordant sizes, without any virtual representation of the hands. Although task performance was affected from 200 ms onwards, participants considered the ease of the task to be affected only from 500 ms visual delay onwards.A Realidade Virtual (RV) permite explorar mudanças no espaço e no tempo que de outra forma seriam difíceis de simular no mundo real. Torna-se possível transformar o mundo virtual aumentando ou diminuindo as distâncias ou manipulando os atrasos no tempo. A análise da adaptabilidade dos utilizadores a diferentes condições espaço-temporais permite estudar a perceção humana e encontrar a combinação certa de paradigmas de interação. Diferentes métodos têm sido propostos na literatura para oferecer aos utilizadores técnicas intuitivas de navegação em espaços virtuais amplos, mesmo que restritos a pequenas áreas físicas de jogo. Outros estudos investigam a tolerância à latência, sugerindo a incapacidade do ser humano de detetar ligeiras discrepâncias entre a informação sensorial visual e propriocetiva. Estes estudos contribuem com valiosas informações para conceber experiências virtuais imersivas e técnicas de interação adequadas a cada tarefa. Esta dissertação apresenta o desenho, implementação e avaliação de um Laboratório de RV tangível onde podem ser estudados cenários de distorção espaço-temporal. Como estudo de caso, restringimos o âmbito da investigação a três cenários de distorção espacial e um cenário de distorção temporal. Os cenários de distorção espacial compararam geometrias Euclidianas e hiperbólicas, estudaram a discordância de tamanho entre objetos físicos e virtuais, e a representação das mãos em RV. O cenário de distorção temporal investigou a partir de que atraso visual o desempenho da tarefa é afetado. A adaptabilidade dos utilizadores às diferentes condições espaço-temporais foi avaliada com base no tempo de conclusão da tarefa, questionários, e comportamentos observados. Os resultados revelaram diferenças significativas entre os espaços Euclidiano e hiperbólico. Também mostraram a preferência pelo manuseamento de objetos virtuais e físicos com tamanhos concordantes, sem qualquer representação virtual das mãos. Embora o desempenho da tarefa tenha sido afetado a partir dos 200 ms, os participantes consideraram que a facilidade da tarefa só foi afetada a partir dos 500 ms de atraso visual

Implicit active constraints for concentric tube robots based on analysis of the safe and dexterous workspace

The use of concentric tube robots has recognized advantages for accessing target lesions while conforming to certain anatomical constraints. However, their complex kinematics makes their safe telemanipulation in convoluted anatomy a challenging task. Collaborative control schemes, which guide the operator through haptic and visual feedback, can simplify this task and reduce the cognitive burden of the operator. Guaranteeing stable, collision-free robot configurations during manipulation, however, is computationally demanding and, until now, either required long periods of pre-computation time or distributed computing clusters. Furthermore, the operator is often presented with guidance paths which have to be followed approximately. This paper presents a heterogeneous (CPU/GPU) computing approach to enable rapid workspace analysis on a single computer. The method is used in a new navigation scheme that guides the robot operator towards locations of high dexterity or manipulability of the robot. Under this guidance scheme, the user can make informed decisions and maintain full control of the path planning and manipulation processes, with intuitive visual feedback on when the robot's limitations are being reached