Performance Envelopes of In-Air Direct and Smartwatch Indirect Control for Head-Mounted Augmented Reality

The scarcity of established input methods for augmented reality (AR) head-mounted displays (HMD) motivates us to investigate the performance envelopes of two easily realisable solutions: indirect cursor control via a smartwatch and direct control by in-air touch. Indirect cursor control via a smartwatch has not been previously investigated for AR HMDs. We evaluate these two techniques in three fundamental user interface actions: target acquisition, goal crossing, and circular steering. We find that in-air is faster than smartwatch (p<0.001) for target acquisition and circular steering. We observe, however, that in-air selection can lead to discomfort after extended use and suggest that smartwatch control offers a complementary alternative.This work was supported by EPSRC (grant number EP/N010558/1) and the Trimble Fund. Part of this work was conducted within the Transregional Collaborative Research Centre SFB/TRR 62 Companion-Technology of Cognitive Technical Systems funded by the German Research Foundation (DFG)

Implementing a Wizard of Oz Tool for Augmented Reality

This thesis aims to explore Wizard of Oz testing in conjunction with Augmented Reality (AR) and focus has been put on testing AR with Head Mounted Displays. The recent increase of interest in HMDs with products such as MOD Live from Recon Instruments and Google's Project Glass puts new demands and possibilities on human-computer interaction. Since the commercial market for HMDs is still in its infancy the need to explore different design approaches is very much present. One way to conduct experiments on human-machine interaction is with the help of a Wizard of Oz tool. During the thesis we have developed such a tool to support designers in researching usability and interaction. The tool provides a user friendly framework to carry out user case studies focused on AR with HMDs. After input and feedback from stakeholders and experts we believe that, even though the tool is mainly meant to be used in conjunction with AR in HMDs, the tool can be applied to other areas as well

Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality

Virtual reality (VR) headsets are enabling a wide range of new opportunities for the user. For example, in the near future users may be able to visit virtual shopping malls and virtually join international conferences. These and many other scenarios pose new questions with regards to privacy and security, in particular authentication of users within the virtual environment. As a first step towards seamless VR authentication, this paper investigates the direct transfer of well-established concepts (PIN, Android unlock patterns) into VR. In a pilot study (N = 5) and a lab study (N = 25), we adapted existing mechanisms and evaluated their usability and security for VR. The results indicate that both PINs and patterns are well suited for authentication in VR. We found that the usability of both methods matched the performance known from the physical world. In addition, the private visual channel makes authentication harder to observe, indicating that authentication in VR using traditional concepts already achieves a good balance in the trade-off between usability and security. The paper contributes to a better understanding of authentication within VR environments, by providing the first investigation of established authentication methods within VR, and presents the base layer for the design of future authentication schemes, which are used in VR environments only

Novel Interaction Techniques for Mobile Augmented Reality applications. A Systematic Literature Review

This study reviews the research on interaction techniques and methods that could be applied in mobile augmented reality scenarios. The review is focused on themost recent advances and considers especially the use of head-mounted displays. Inthe review process, we have followed a systematic approach, which makes the reviewtransparent, repeatable, and less prone to human errors than if it was conducted in amore traditional manner. The main research subjects covered in the review are headorientation and gaze-tracking, gestures and body part-tracking, and multimodality– as far as the subjects are related to human-computer interaction. Besides these,also a number of other areas of interest will be discussed.Siirretty Doriast

Around-Body Interaction: Leveraging Limb Movements for Interacting in a Digitally Augmented Physical World

Recent technological advances have made head-mounted displays (HMDs) smaller and untethered, fostering the vision of ubiquitous interaction with information in a digitally augmented physical world. For interacting with such devices, three main types of input - besides not very intuitive finger gestures - have emerged so far: 1) Touch input on the frame of the devices or 2) on accessories (controller) as well as 3) voice input. While these techniques have both advantages and disadvantages depending on the current situation of the user, they largely ignore the skills and dexterity that we show when interacting with the real world: Throughout our lives, we have trained extensively to use our limbs to interact with and manipulate the physical world around us. This thesis explores how the skills and dexterity of our upper and lower limbs, acquired and trained in interacting with the real world, can be transferred to the interaction with HMDs. Thus, this thesis develops the vision of around-body interaction, in which we use the space around our body, defined by the reach of our limbs, for fast, accurate, and enjoyable interaction with such devices. This work contributes four interaction techniques, two for the upper limbs and two for the lower limbs: The first contribution shows how the proximity between our head and hand can be used to interact with HMDs. The second contribution extends the interaction with the upper limbs to multiple users and illustrates how the registration of augmented information in the real world can support cooperative use cases. The third contribution shifts the focus to the lower limbs and discusses how foot taps can be leveraged as an input modality for HMDs. The fourth contribution presents how lateral shifts of the walking path can be exploited for mobile and hands-free interaction with HMDs while walking.Comment: thesi

Usable privacy and security in smart homes

Ubiquitous computing devices increasingly dominate our everyday lives, including our most private places: our homes. Homes that are equipped with interconnected, context-aware computing devices, are considered “smart” homes. To provide their functionality and features, these devices are typically equipped with sensors and, thus, are capable of collecting, storing, and processing sensitive user data, such as presence in the home. At the same time, these devices are prone to novel threats, making our homes vulnerable by opening them for attackers from outside, but also from within the home. For instance, remote attackers who digitally gain access to presence data can plan for physical burglary. Attackers who are physically present with access to devices could access associated (sensitive) user data and exploit it for further cyberattacks. As such, users’ privacy and security are at risk in their homes. Even worse, many users are unaware of this and/or have limited means to take action. This raises the need to think about usable mechanisms that can support users in protecting their smart home setups. The design of such mechanisms, however, is challenging due to the variety and heterogeneity of devices available on the consumer market and the complex interplay of user roles within this context. This thesis contributes to usable privacy and security research in the context of smart homes by a) understanding users’ privacy perceptions and requirements for usable mechanisms and b) investigating concepts and prototypes for privacy and security mechanisms. Hereby, the focus is on two specific target groups, that are inhabitants and guests of smart homes. In particular, this thesis targets their awareness of potential privacy and security risks, enables them to take control over their personal privacy and security, and illustrates considerations for usable authentication mechanisms. This thesis provides valuable insights to help researchers and practitioners in designing and evaluating privacy and security mechanisms for future smart devices and homes, particularly targeting awareness, control, and authentication, as well as various roles.Computer und andere „intelligente“, vernetzte Geräte sind allgegenwärtig und machen auch vor unserem privatesten Zufluchtsort keinen Halt: unserem Zuhause. Ein „intelligentes Heim“ verspricht viele Vorteile und nützliche Funktionen. Um diese zu erfüllen, sind die Geräte mit diversen Sensoren ausgestattet – sie können also in unserem Zuhause sensitive Daten sammeln, speichern und verarbeiten (bspw. Anwesenheit). Gleichzeitig sind die Geräte anfällig für (neuartige) Cyberangriffe, gefährden somit unser Zuhause und öffnen es für potenzielle – interne sowie externe – Angreifer. Beispielsweise könnten Angreifer, die digital Zugriff auf sensitive Daten wie Präsenz erhalten, einen physischen Überfall in Abwesenheit der Hausbewohner planen. Angreifer, die physischen Zugriff auf ein Gerät erhalten, könnten auf assoziierte Daten und Accounts zugreifen und diese für weitere Cyberangriffe ausnutzen. Damit werden die Privatsphäre und Sicherheit der Nutzenden in deren eigenem Zuhause gefährdet. Erschwerend kommt hinzu, dass viele Nutzenden sich dessen nicht bewusst sind und/oder nur limitierte Möglichkeiten haben, effiziente Gegenmaßnahmen zu ergreifen. Dies macht es unabdingbar, über benutzbare Mechanismen nachzudenken, die Nutzende beim Schutz ihres intelligenten Zuhauses unterstützen. Die Umsetzung solcher Mechanismen ist allerdings eine große Herausforderung. Das liegt unter anderem an der großen Vielfalt erhältlicher Geräte von verschiedensten Herstellern, was das Finden einer einheitlichen Lösung erschwert. Darüber hinaus interagieren im Heimkontext meist mehrere Nutzende in verschieden Rollen (bspw. Bewohner und Gäste), was die Gestaltung von Mechanismen zusätzlich erschwert. Diese Doktorarbeit trägt dazu bei, benutzbare Privatsphäre- und Sicherheitsmechanismen im Kontext des „intelligenten Zuhauses“ zu entwickeln. Insbesondere werden a) die Wahrnehmung von Privatsphäre sowie Anforderungen an potenzielle Mechanismen untersucht, sowie b) Konzepte und Prototypen für Privatsphäre- und Sicherheitsmechanismen vorgestellt. Der Fokus liegt hierbei auf zwei Zielgruppen, den Bewohnern sowie den Gästen eines intelligenten Zuhauses. Insbesondere werden in dieser Arbeit deren Bewusstsein für potenzielle Privatsphäre- und Sicherheits-Risiken adressiert, ihnen Kontrolle über ihre persönliche Privatsphäre und Sicherheit ermöglicht, sowie Möglichkeiten für benutzbare Authentifizierungsmechanismen für beide Zielgruppen aufgezeigt. Die Ergebnisse dieser Doktorarbeit legen den Grundstein für zukünftige Entwicklung und Evaluierung von benutzbaren Privatsphäre und Sicherheitsmechanismen im intelligenten Zuhause

Novel Interaction Techniques for Mobile Augmented Reality Applications – A Systematic Literature Review

This study reviews the research on interaction techniques and methods that could be applied in mobile augmented reality scenarios. The review is focused on the most recent advances and considers especially the use of head-mounted displays. In the review process, we have followed a systematic approach, which makes the review transparent, repeatable, and less prone to human errors than if it was conducted in a more traditional manner. The main research subjects covered in the review are head orientation and gaze tracking, gestures and body part tracking, and multimodality&ndash; as far as the subjects are related to human-computer interaction. Besides these, also a number of other areas of interest will be discussed.</p

Vision based 3D Gesture Tracking using Augmented Reality and Virtual Reality for Improved Learning Applications

3D gesture recognition and tracking based augmented reality and virtual reality have become a big interest of research because of advanced technology in smartphones. By interacting with 3D objects in augmented reality and virtual reality, users get better understanding of the subject matter where there have been requirements of customized hardware support and overall experimental performance needs to be satisfactory. This research investigates currently various vision based 3D gestural architectures for augmented reality and virtual reality. The core goal of this research is to present analysis on methods, frameworks followed by experimental performance on recognition and tracking of hand gestures and interaction with virtual objects in smartphones. This research categorized experimental evaluation for existing methods in three categories, i.e. hardware requirement, documentation before actual experiment and datasets. These categories are expected to ensure robust validation for practical usage of 3D gesture tracking based on augmented reality and virtual reality. Hardware set up includes types of gloves, fingerprint and types of sensors. Documentation includes classroom setup manuals, questionaries, recordings for improvement and stress test application. Last part of experimental section includes usage of various datasets by existing research. The overall comprehensive illustration of various methods, frameworks and experimental aspects can significantly contribute to 3D gesture recognition and tracking based augmented reality and virtual reality.Peer reviewe

VStore: Feasibility and acceptability of a novel virtual reality functional cognition task

Virtual reality (VR) is becoming an increasingly popular tool in neuroscience and mental health research. In recent years, efforts have been made to virtualise neuropsychological testing with the intent to increase the ecological validity of cognitive assessments. However, there are some limitations in the current literature—feasibility and acceptability data are often not reported or available and sample sizes have generally been small. In this study, we describe the development and establish the feasibility and acceptability of use of a novel functional cognition VR shopping task, VStore, in three separate samples with data from a total of 210 participants. Two samples include healthy volunteers between the ages of 20 and 79 and there is one clinical cohort of patients with psychosis. Main VStore outcomes were: 1) verbal recall of 12 grocery items, 2) time to collect items, 3) time to select items on a self-checkout machine, 4) time to make the payment, 5) time to order hot drink, and 6) total time. Feasibility and acceptability were assessed by the completion rate across the three studies. VR induced adverse effects were assessed preand post-VStore administration to establish tolerability. Finally, as an exploratory objective, VStore’s ability to differentiate between younger and older age groups, and between patients and matched healthy controls was examined as preliminary indication of its potential utility. The overall completion rate across the studies was exceptionally high (99.95%), and VStore did not induce any adverse effects. Additionally, there was a clear difference in VStore performance metrics between both the patients and controls and between younger and older age groups, suggesting potential clinical utility of this VR assessment. These findings demonstrate that VStore is a promising neuropsychological tool that is well-tolerated and feasible to administer to both healthy and clinical populations. We discuss the implications for future research involving neuropsychological testing based on our experience and the contemporary literature