The Hitchhiker’s Guide to the Metaverse

Non peer reviewe

Eyes-Off Physically Grounded Mobile Interaction

This thesis explores the possibilities, challenges and future scope for eyes-off, physically grounded mobile interaction. We argue that for interactions with digital content in physical spaces, our focus should not be constantly and solely on the device we are using, but fused with an experience of the places themselves, and the people who inhabit them. Through the design, development and evaluation of a series ofnovel prototypes we show the benefits of a more eyes-off mobile interaction style.Consequently, we are able to outline several important design recommendations for future devices in this area.The four key contributing chapters of this thesis each investigate separate elements within this design space. We begin by evaluating the need for screen-primary feedback during content discovery, showing how a more exploratory experience can be supported via a less-visual interaction style. We then demonstrate how tactilefeedback can improve the experience and the accuracy of the approach. In our novel tactile hierarchy design we add a further layer of haptic interaction, and show how people can be supported in finding and filtering content types, eyes-off. We then turn to explore interactions that shape the ways people interact with aphysical space. Our novel group and solo navigation prototypes use haptic feedbackfor a new approach to pedestrian navigation. We demonstrate how variations inthis feedback can support exploration, giving users autonomy in their navigationbehaviour, but with an underlying reassurance that they will reach the goal.Our final contributing chapter turns to consider how these advanced interactionsmight be provided for people who do not have the expensive mobile devices that areusually required. We extend an existing telephone-based information service to support remote back-of-device inputs on low-end mobiles. We conclude by establishingthe current boundaries of these techniques, and suggesting where their usage couldlead in the future

HandSight: A Touch-Based Wearable System to Increase Information Accessibility for People with Visual Impairments

Many activities of daily living such as getting dressed, preparing food, wayfinding, or shopping rely heavily on visual information, and the inability to access that information can negatively impact the quality of life for people with vision impairments. While numerous researchers have explored solutions for assisting with visual tasks that can be performed at a distance, such as identifying landmarks for navigation or recognizing people and objects, few have attempted to provide access to nearby visual information through touch. Touch is a highly attuned means of acquiring tactile and spatial information, especially for people with vision impairments. By supporting touch-based access to information, we may help users to better understand how a surface appears (e.g., document layout, clothing patterns), thereby improving the quality of life. To address this gap in research, this dissertation explores methods to augment a visually impaired user’s sense of touch with interactive, real-time computer vision to access information about the physical world. These explorations span three application areas: reading and exploring printed documents, controlling mobile devices, and identifying colors and visual textures. At the core of each application is a system called HandSight that uses wearable cameras and other sensors to detect touch events and identify surface content beneath the user’s finger. To create HandSight, we designed and implemented the physical hardware, developed signal processing and computer vision algorithms, and designed real-time feedback that enables users to interpret visual or digital content. We involve visually impaired users throughout the design and development process, conducting several user studies to assess usability and robustness and to improve our prototype designs. The contributions of this dissertation include: (i) developing and iteratively refining HandSight, a novel wearable system to assist visually impaired users in their daily lives; (ii) evaluating HandSight across a diverse set of tasks, and identifying tradeoffs of a finger-worn approach in terms of physical design, algorithmic complexity and robustness, and usability; and (iii) identifying broader design implications for future wearable systems and for the fields of accessibility, computer vision, augmented and virtual reality, and human-computer interaction

Designing User-Centric Private Conversation Methods in the Metaverse

The metaverse is an emerging medium for remote interactions, allowing users to engage in immersive experiences with others in virtual environments, such as attending concerts, business meetings, or social gatherings with friends. Private conversation is an important feature that improves the overall experience in the metaverse. This essential element of virtual interactions allows the exchange of sensitive information and promotes self-disclosure, a key factor in building interpersonal relationships. However, current methods for establishing private conversations have several limitations. In Private Talk, floating icons above the users' avatars do not feel natural and break the immersion. Meanwhile, creating private rooms and teleporting to them disrupts the flow of experience. The goal of this thesis is to design private conversations in the metaverse. First, we surveyed existing methods for establishing private conversations by assessing popular applications and online sources. Second, we developed our own application where we implemented two baseline methods for private conversations, Private Talk and private room. Next, we conducted a user study where we invited 12 participants to evaluate the baseline methods and propose their own methods. We employed questionnaires and conducted interviews to gather suggestions and valuable insights. A thematic analysis of the interview transcripts identified six themes; minimizing background noise, isolation for enhanced feeling of privacy, indicators and distinctions of privacy mode, easy and natural methods in virtual environments, and privacy concerns. From our results, we developed design implications for improving private conversation methods in the metaverse. Our findings aim to guide the design of the future metaverse

Integrating passive ubiquitous surfaces into human-computer interaction

Mobile technologies enable people to interact with computers ubiquitously. This dissertation investigates how ordinary, ubiquitous surfaces can be integrated into human-computer interaction to extend the interaction space beyond the edge of the display. It turns out that acoustic and tactile features generated during an interaction can be combined to identify input events, the user, and the surface. In addition, it is shown that a heterogeneous distribution of different surfaces is particularly suitable for realizing versatile interaction modalities. However, privacy concerns must be considered when selecting sensors, and context can be crucial in determining whether and what interaction to perform.Mobile Technologien ermöglichen den Menschen eine allgegenwärtige Interaktion mit Computern. Diese Dissertation untersucht, wie gewöhnliche, allgegenwärtige Oberflächen in die Mensch-Computer-Interaktion integriert werden können, um den Interaktionsraum über den Rand des Displays hinaus zu erweitern. Es stellt sich heraus, dass akustische und taktile Merkmale, die während einer Interaktion erzeugt werden, kombiniert werden können, um Eingabeereignisse, den Benutzer und die Oberfläche zu identifizieren. Darüber hinaus wird gezeigt, dass eine heterogene Verteilung verschiedener Oberflächen besonders geeignet ist, um vielfältige Interaktionsmodalitäten zu realisieren. Bei der Auswahl der Sensoren müssen jedoch Datenschutzaspekte berücksichtigt werden, und der Kontext kann entscheidend dafür sein, ob und welche Interaktion durchgeführt werden soll

A personalized support tool for the training of mindful walking: The mobile “MindfulWalk” application

Digital health prevention is a trend that becomes increasingly important in various domains. Health insurers crave for effective methods that can be offered to their customers. Moreover, smart mobile devices pose many advantages as they can be easily used in everyday life without being burdensome. Taking these advantages into account, completely new applications become possible. This thesis presents an application that is intended to support users to walk mindfully. It is a mobile personalized tool that senses the walking speed and provides haptic feedback thereof. The procedure of mindful walking, the technical prototype as well as preliminary study results are presented and discussed. The reported user experience and the study result indicate promising perspectives for a tool that supports a mindful walking behavior. Altogether, the use of modern smart mobile device sensors paves the way for useful mobile application in the context of health prevention in particular and health care in general