Human factors in instructional augmented reality for intravehicular spaceflight activities and How gravity influences the setup of interfaces operated by direct object selection

In human spaceflight, advanced user interfaces are becoming an interesting mean to facilitate human-machine interaction, enhancing and guaranteeing the sequences of intravehicular space operations. The efforts made to ease such operations have shown strong interests in novel human-computer interaction like Augmented Reality (AR). The work presented in this thesis is directed towards a user-driven design for AR-assisted space operations, iteratively solving issues arisen from the problem space, which also includes the consideration of the effect of altered gravity on handling such interfaces.Auch in der bemannten Raumfahrt steigt das Interesse an neuartigen Benutzerschnittstellen, um nicht nur die Mensch-Maschine-Interaktion effektiver zu gestalten, sondern auch um einen korrekten Arbeitsablauf sicherzustellen. In der Vergangenheit wurden wiederholt Anstrengungen unternommen, Innenbordarbeiten mit Hilfe von Augmented Reality (AR) zu erleichtern. Diese Arbeit konzentriert sich auf einen nutzerorientierten AR-Ansatz, welcher zum Ziel hat, die Probleme schrittweise in einem iterativen Designprozess zu lösen. Dies erfordert auch die Berücksichtigung veränderter Schwerkraftbedingungen

Viewport- and World-based Personal Device Point-Select Interactions in the Augmented Reality

Personal smart devices have demonstrated a variety of efficient techniques for pointing and selecting on physical displays. However, when migrating these input techniques to augmented reality, it is both unclear what the relative performance of different techniques will be given the immersive nature of the environment, and it is unclear how viewport-based versus world-based pointing methods will impact performance. To better understand the impact of device and viewing perspectives on pointing in augmented reality, in this thesis, we present the results of two controlled experiments comparing pointing conditions that leverage various smartphone- and smartwatch-based external display pointing techniques and examine viewport-based versus world-based target acquisition paradigms. Our results demonstrate that viewport-based techniques offer faster selection and that both smartwatch- and smartphone-based pointing techniques represent high-performance options for performing distant target acquisition tasks in augmented reality

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance

IATK:An immersive analytics toolkit

International audienceWe introduce IATK, the Immersive Analytics Toolkit, a software package for Unity that allows interactive authoring and exploration of data visualisation in immersive environments. The design of IATK was informed by interdisciplinary expert-collaborations as well as visual analytics applications and iterative refinement over several years. IATK allows for easy assembly of visualisations through a grammar of graphics that a user can configure in a GUI— in addition to a dedicated visualisation API that supports the creation of novel immersive visualisation designs and interactions. IATK is designed with scalability in mind, allowing visualisation and fluid responsive interactions in the order of several million points at a usable frame rate. This paper outlines our design requirements, IATK’s framework design and technical features, its user interface, as well as application examples

Spatial embodied augmented reality: design of AR for spatial productivity applications

Recently there has been a growth in commercial systems for Augmented Reality (AR), such as the Microsoft HoloLens and Magic Leap, which aim to subsume traditionally 2D productivity tasks performed by previous technologies through the use of a 3D User Interface (3DUI). However, there is currently a lack of research into how to design effective AR interfaces and no consensus on how to design general purpose 3DUIs. Augmented reality is an emergent technology with little prior design precedent. As AR becomes more widespread, it becomes apparent that new design paradigms are required to translate potential benefits of these new interfaces. This thesis aims to explore the design of AR productivity software that can exploit the spatial nature of augmented reality to complement the spatial nature of human interaction. To investigate this, three user studies were performed using an email client as a use case scenario. Each study focused on a different aspect in the design of an augmented reality system starting with user elicitation using informance design methods. This progressed to testing of alternative interfaces for three-dimensional document presentation using a formal lab experiment, finally ending with an observation of how users arrange documents in a simulated limited prototype system. The findings of this thesis include: • A new method of user elicitation for augmented reality interface design called “spatial informance design”. • Design recommendations of spatial interface augmentations for email. • Data to support that space can be used to triage email more effectively. • Different layouts of documents in space provide greater or lesser time, accuracy and memorability. • Presentation of document layouts with either an ego or exocentric view alters performance. • Users have a preference to use space over colour to group documents. • Users take advantage of space when a 3rd dimension is available. The conclusion of this thesis is that augmented reality has the potential to improve the user experience over the traditional two-dimensional GUI for knowledge work tasks

The Effects of Sharing Awareness Cues in Collaborative Mixed Reality

Augmented and Virtual Reality provide unique capabilities for Mixed Reality collaboration. This paper explores how different combinations of virtual awareness cues can provide users with valuable information about their collaborator's attention and actions. In a user study (n = 32, 16 pairs), we compared different combinations of three cues: Field-of-View (FoV) frustum, Eye-gaze ray, and Head-gaze ray against a baseline condition showing only virtual representations of each collaborator's head and hands. Through a collaborative object finding and placing task, the results showed that awareness cues significantly improved user performance, usability, and subjective preferences, with the combination of the FoV frustum and the Head-gaze ray being best. This work establishes the feasibility of room-scale MR collaboration and the utility of providing virtual awareness cues

The always best positioned paradigm for mobile indoor applications

In this dissertation, methods for personal positioning in outdoor and indoor environments are investigated. The Always Best Positioned paradigm, which has the goal of providing a preferably consistent self-positioning, will be defined. Furthermore, the localization toolkit LOCATO will be presented, which allows to easily realize positioning systems that follow the paradigm. New algorithms were developed, which particularly address the robustness of positioning systems with respect to the Always Best Positioned paradigm. With the help of this toolkit, three example positioning-systems were implemented, each designed for different applications and requirements: a low-cost system, which can be used in conjunction with user-adaptive public displays, a so-called opportunistic system, which enables positioning with room-level accuracy in any building that provides a WiFi infrastructure, and a high-accuracy system for instrumented environments, which works with active RFID tags and infrared beacons. Furthermore, a new and unique evaluation-method for positioning systems is presented, which uses step-accurate natural walking-traces as ground truth. Finally, six location based services will be presented, which were realized either with the tools provided by LOCATO or with one of the example positioning-systems.In dieser Doktorarbeit werden Methoden zur Personenpositionierung im Innen- und Außenbereich von Gebäuden untersucht. Es wird das ,,Always Best Positioned” Paradigma definiert, welches eine möglichst lückenlose Selbstpositionierung zum Ziel hat. Weiterhin wird die Lokalisierungsplattform LOCATO vorgestellt, welche eine einfache Umsetzung von Positionierungssystemen ermöglicht. Hierzu wurden neue Algorithmen entwickelt, welche gezielt die Robustheit von Positionierungssystemen unter Berücksichtigung des ,,Always Best Positioned” Paradigmas angehen. Mit Hilfe dieser Plattform wurden drei Beispiel Positionierungssysteme entwickelt, welche unterschiedliche Einsatzgebiete berücksichtigen: Ein kostengünstiges System, das im Zusammenhang mit benutzeradaptiven öffentlichen Bildschirmen benutzt werden kann; ein sogenanntes opportunistisches Positionierungssystem, welches eine raumgenaue Positionierung in allen Gebäuden mit WLAN-Infrastruktur ermöglicht, sowie ein metergenaues Positionierungssystem, welches mit Hilfe einer Instrumentierung aus aktiven RFID-Tags und Infrarot-Baken arbeitet. Weiterhin wird erstmalig eine Positionierungsevaluation vorgestellt, welche schrittgenaue, natürliche Bewegungspfade als Referenzsystem einsetzt. Im Abschluss werden 6 lokationsbasierte Dienste vorgestellt, welche entweder mit Hilfe von LOCATO oder mit Hilfe einer der drei Beispiel-Positionierungssysteme entwickelt wurden

Sensing and Visualizing Social Context from Spatial Proximity

The concept of pervasive computing, as introduced by Marc Weiser under the name ubiquitous computing in the early 90s, spurred research into various kinds of context-aware systems and applications. There is a wide range of contextual parameters, including location, time, temperature, devices and people in proximity, which have been part of the initial ideas about context-aware computing. While locational context is already a well understood concept, social context---based on the people around us---proves to be harder to grasp and to operationalize. This work continues the line of research into social context, which is based on the proximity and meeting patterns of people in the physical space. It takes this research out of the lab and out of well controlled situations into our urban environments, which are full of ambiguity and opportunities. The key to this research is the tool that caused dramatic change in individual and collective behavior during the last 20 years and which is a manifestation of many of the ideas of the pervasive computing paradigm: the mobile phone. In this work, the mobile is regarded as a proxy for people. Through it, the social environment becomes accessible to digital measurement and processing. To understand the large amount of data that now becomes available to automatic measurement, we will turn to the discipline of social network analysis. It provides powerful methods, that are able to condense data and extract relevant meaning. Visualization helps to understand and interpret the results. This thesis contains a number of experiments, that demonstrate how the automatic measurement of social proximity data through Bluetooth can be used to measure variables of personal behavior, group behavior and the behavior of groups in relation to places. The principal contributions are: * A methodology to visualize personal social context by using an ego proximity network. Specific episodes can be localized and compared. * method to compare different days in terms of social context, e.g. to support automatic diary applications. * A method to compose social geographic maps. Locations of similar social context are detected and combined. * Functions to measure short-term changes in social activity, based on the distinction between strange and familiar devices. * The characterization of Bluetooth inquiries for social proximity sensing. * A dataset of Bluetooth sightings from an ego perspective in seven different settings. Additionally, some settings feature multiple stationary scanners and Cell-ID measurements. * Soft- and hardware to capture, collect, store and analyze Bluetooth proximity data

User Expectations and Experiences of Mobile Augmented Reality Services

Mobile Augmented Reality (MAR) as an emerging field of technology has the potential to engender services that demonstrate novel aspects like enriching the reality with digital information, location-based interaction, and tangible user interfaces. The early visions of MAR anticipated it to revolutionize the way of accessing and interacting with information in mobile contexts. However, one hindrance in this path is the lack of research understanding of the subjective user experience (UX) resulting from, e.g., the novel interaction metaphors and the mixing of realities that MAR embodies. What is more, little is known about users’ expectations of the futuristic concept of MAR and the experiences it could evoke. Because of the increasing importance of UX as a quality attribute in products and services, there is a need to understand the characteristics and expectations of UX in specific emerging fields like MAR. The goal of this thesis research is twofold: (1) to understand potential users’ expectations with regard to UX of future MAR services and (2) to understand the actual UX of the recent first-generation MAR applications like Junaio and Layar. By understanding the scope of expectations and experience that can take place in the field of MAR, it is possible to help the design and engineering of AR-based services to consider also the experiential aspirations of potential end users. This compound thesis belongs to the research field of Human-Computer Interaction. It contains four studies, in which altogether 401 persons participated in either interviews or online surveys. The empirical findings on expected and actual experiences are reported in six publications. The theoretical contribution is mostly conceptual, culminating to a framework that describes the facets of UX and categories of meaningful experiences in MAR. Based on the empirical findings and the framework, the practical contribution is concretized in the form of (1) design implications and (2) subjective evaluation measures to help designing future MAR services with an experience-oriented approach. According to the results, potential users (early adopters) expected MAR services to create a great extent of pleasurable experiences, such as empowerment, surprise, awareness, liveliness, playfulness, tangibility, collectivity, inspiration and creativity. Furthermore, the expectations were attributed to a variety of service components, also relating to other ubiquitous computing aspects (e.g., the augmentation as an output, proactive functionalities, and embedding of digital content to the reality). This implies that emerging technological concepts like MAR are perceived in very diverse ways and that people’s expectations of them consist largely of general needs and desires. The existing first-generation MAR applications, however, seem generally not to fulfill the expectations, showing a much narrower extent of actualized experience characteristics. This experiential gap, as well as the narrower extent of functionalities in current applications, contains much potential with regard to pursuing a rich and pleasurable UX in future design of MAR services. The empirical results, conceptualizations and practical implications can be utilized and built on in academic research as well as in development of MAR. The novelty and complexity of both MAR and UX as concepts elicit an extensive breadth of aspects to be studied in detail in future research and development – regarding both MAR as a field of technology and UX as a field of theory