A Quest for Co-Located Mixed Reality: Aligning and Assessing SLAM Tracking for Same-Space Multi-User Experiences

Current solutions for creating co-located Mixed Reality (MR) experiences typically rely on platform-specific synchronisation of spatial anchors or Simultaneous Localisation and Mapping (SLAM) data across clients, often coupled to cloud services. This introduces significant costs (in development and deployment), constraints (with interoperability across platforms often limited), and privacy concerns. For practitioners, support is needed for creating platform-agnostic co-located MR experiences. This paper explores the utility of aligned SLAM solutions by 1) surveying approaches toward aligning disparate device coordinate spaces, formalizing their theoretical accuracy and limitations; 2) providing skeleton implementations for audience-based, small-scale and large-scale co-location using said alignment approaches; and 3) detailing how we can assess the accuracy and safety of 6DoF/SLAM tracking solutions for any arbitrary device and dynamic environment without the need for an expensive ground truth optical tracking, by using trilateration and a $30 laser distance meter. Through this, we hope to further democratise the creation of cross-platform co-located MR experiences

Memory Manipulations in Extended Reality

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR

VRception: Rapid Prototyping of Cross-Reality Systems in Virtual Reality

Cross-reality systems empower users to transition along the reality-virtuality continuum or collaborate with others experiencing different manifestations of it. However, prototyping these systems is challenging, as it requires sophisticated technical skills, time, and often expensive hardware. We present VRception, a concept and toolkit for quick and easy prototyping of cross-reality systems. By simulating all levels of the reality-virtuality continuum entirely in Virtual Reality, our concept overcomes the asynchronicity of realities, eliminating technical obstacles. Our VRception Toolkit leverages this concept to allow rapid prototyping of cross-reality systems and easy remixing of elements from all continuum levels. We replicated six cross-reality papers using our toolkit and presented them to their authors. Interviews with them revealed that our toolkit sufficiently replicates their core functionalities and allows quick iterations. Additionally, remote participants used our toolkit in pairs to collaboratively implement prototypes in about eight minutes that they would have otherwise expected to take days

What makes XR dark? Examining emerging dark patterns in augmented and virtual reality through expert co-design

Dark Patterns are deceptive designs that influence a user's interactions with an interface to benefit someone other than the user. Prior work has identified dark patterns in WIMP interfaces and ubicomp environments, but how dark patterns can manifest in Augmented and Virtual Reality (collectively XR) requires more attention. We therefore conducted ten co-design workshops with 20 experts in XR and deceptive design. Our participants co-designed 42 scenarios containing dark patterns, based on application archetypes presented in recent HCI/XR literature. In the co-designed scenarios, we identified ten novel dark patterns in addition to 39 existing ones, as well as ten examples in which specific characteristics associated with XR potentially amplified the effect dark patterns could have on users. Based on our findings and prior work, we present a classification of XR-specific properties that facilitate dark patterns: perception, spatiality, physical/virtual barriers, and XR device sensing. We also present the experts’ assessments of the likelihood and severity of the co-designed scenarios and highlight key aspects they considered for this evaluation, for example, technological feasibility, ease of upscaling and distributing malicious implementations, and the application's context of use. Finally, we discuss means to mitigate XR dark patterns and support regulatory bodies to reduce potential harms

Knowledge-driven Biometric Authentication in Virtual Reality

With the increasing adoption of virtual reality (VR) in public spaces, protecting users from observation attacks is becoming essential to prevent attackers from accessing context-sensitive data or performing malicious payment transactions in VR. In this work, we propose RubikBiom, a knowledge-driven behavioural biometric authentication scheme for authentication in VR. We show that hand movement patterns performed during interactions with a knowledge-based authentication scheme (e.g., when entering a PIN) can be leveraged to establish an additional security layer. Based on a dataset gathered in a lab study with 23 participants, we show that knowledge-driven behavioural biometric authentication increases security in an unobtrusive way. We achieve an accuracy of up to 98.91% by applying a Fully Convolutional Network (FCN) on 32 authentications per subject. Our results pave the way for further investigations towards knowledge-driven behavioural biometric authentication in VR

Exploring Potentially Abusive Ethical, Social and Political Implications of Mixed Reality Research in HCI

In recent years, Mixed Reality (MR) headsets have increasingly made advances in terms of capability, affordability and end-user adoption, slowly becoming everyday technology. HCI research typically explores positive aspects of these technologies, focusing on interaction, presence and immersive experiences. However, such technological advances and paradigm shifts often fail to consider the ``dark patterns'', with potential abusive scenarios, made possible by new technologies (cf. smartphone addiction, social media anxiety disorder). While these topics are getting recent attention in related fields and with the general population, this workshop is aimed at starting an active exploration of abusive, ethical, social and political scenarios of MR research inside the HCI community. With an HCI lens, workshop participants will engage in critical reviews of emerging MR technologies and applications and develop a joint research agenda to address them

Nomadic virtual reality : overcoming challenges of mobile virtual reality head-mounted displays

Technological advancements in the fields of optics, display technology and miniaturization have enabled high-quality virtual reality (VR) head-mounted displays (HMDs) to be used beyond research labs and become available as consumer products. In turn, this development enabled mobile VR HMDs, which are untethered and self-contained headsets, allowing users to immerse themselves wherever and whenever they wish. This creates a novel interaction scenario in which a user is immersed in a virtual environment using a mobile VR HMD inside an unknown context (e.g., watching a 360-degree video while commuting by public transport). This thesis defines this novel interaction scenario as nomadic VR and systematically explores its upcoming challenges and opportunities. For this, the interaction scenario is embedded into a larger vision of ubiquitous mixed reality, using models and approaches from the field of context-aware computing which already explain a similar transformation and paradigm shift from stationary PCs to mobile computing (smartphones): The form factor changed dramatically, cursor-based input was replaced with multi-touch, sound and visual feedback was extended with vibration and the constant changing environment enabled a variety of location-based features and services. We argue that a similar transformation will happen from stationary VR HMDs to mobile VR HMDs: the input will be adapted, novel output modalities will be added and the context of use will be incorporated into the virtual environment. This dissertation consists of six case studies, each addressing one aspect of these challenges (input, output and context). To enable fast and precise input we present FaceTouch, a novel interaction concept leveraging the backside of the HMD as a touch-sensitive surface. FaceTouch allows the user to select virtual content inside the nomadic VR interaction scenario without the need for additional accessories or expansive gestures. To extend the output capabilities of mobile VR HMDs, we propose GyroVR, a set of HMD-attached flywheels, leveraging the gyroscopic effect of resistance when changing the spinning axis of rotation and generating the perception of inertia. GyroVR was designed as a mobile and ungrounded feedback device fitting into the nomadic VR interaction scenario. The context was divided into the physical environment and human factors. With CarVR, we explored how to enable the usage of VR HMDs inside of moving vehicles such as cars. The CarVR system senses and incorporates the additional motions arising inside of these dynamic physical environments, enabling an increment of enjoyment and reduction of simulator sickness compared to a stationary setup. The SwiVRChair system presents a motorized office chair, exploring how everyday objects inside a static physical environment can be incorporated into the nomadic VR interaction scenario to enhance the overall user experience. Since the nomadic VR interaction scenario often takes place inside of public environments, for the human factor context we focused on social scenarios in which people use VR HMDs when people without HMDs (non-HMD users) are in the vicinity. With the ShareVR system, we present a prototype which uses floor projection and mobile displays combined with positional tracking to visualize the virtual world to (non-HMD) users and enable an asymmetric interaction. In a followup case study, we adapted the ShareVR concept to fit into a mobile VR HMD. FaceDisplay is a modified VR HMD that consists of three touch-sensitive displays and a depth camera attached to the back of the HMD, allowing the non-HMD user to perceive and interact with the virtual world through touch or gestures. We conclude this dissertation with three overarching findings that resulted not out of the individual research questions but emerged throughout the whole process of this thesis: (1) We argue that current HMDs are mainly optimized for the wearer and ignore the whole social context; future HMDs have to be designed to be able to include non-HMD users. (2) We show that the physical environment should not only be seen as a challenge, but can be leveraged to reduce problems such as simulator sickness and increase immersion. (3) We propose that similar to the very first smartphone, current HMDs should be seen as an unfinished device type. We argue for an engineering research approach that extends the current form factor through novel sensors and actuators