From corporeality to virtual reality: theorizing literacy, bodies, and technology in the emerging media of virtual, augmented, and mixed realities

This dissertation explores the relationships between literacy, technology, and bodies in the emerging media of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). In response to the recent, rapid emergence of new media forms, questions arise as to how and why we should prepare to compose in new digital media. To interrogate the newness accorded to new media composing, I historicize the literacy practices demanded by new media by examining digital texts, such as video games and software applications, alongside analogous “antiquated” media, such as dioramas and museum exhibits. Comparative textual analysis of analogous digital and non-digital VR, AR, and MR texts reveals new media and “antiquated” media utilize common characteristics of dimensionality, layering, and absence/presence, respectively. The establishment of shared traits demonstrates how media operate on a continuum of mutually held textual practices; despite their distinctive forms, new media texts do not represent either a hierarchical or linear progression of maturing development. Such an understanding aids composing in new VR, AR, and MR media by enabling composers to make fuller use of prior knowledge in a rapidly evolving new media environment, a finding significant both for educators and communicators. As these technologies mature, we will continue to compose both traditional and new forms of texts. As such, we need literacy theory that attends to both the traditional and the new and also is comprehensive enough to encompass future acts of composing in media yet to emerge

On Inter-referential Awareness in Collaborative Augmented Reality

For successful collaboration to occur, a workspace must support inter-referential awareness - or the ability for one participant to refer to a set of artifacts in the environment, and for that reference to be correctly interpreted by others. While referring to objects in our everyday environment is a straight-forward task, the non-tangible nature of digital artifacts presents us with new interaction challenges. Augmented reality (AR) is inextricably linked to the physical world, and it is natural to believe that the re-integration of physical artifacts into the workspace makes referencing tasks easier; however, we find that these environments combine the referencing challenges from several computing disciplines, which compound across scenarios. This dissertation presents our studies of this form of awareness in collaborative AR environments. It stems from our research in developing mixed reality environments for molecular modeling, where we explored spatial and multi-modal referencing techniques. To encapsulate the myriad of factors found in collaborative AR, we present a generic, theoretical framework and apply it to analyze this domain. Because referencing is a very human-centric activity, we present the results of an exploratory study which examines the behaviors of participants and how they generate references to physical and virtual content in co-located and remote scenarios; we found that participants refer to content using physical and virtual techniques, and that shared video is highly effective in disambiguating references in remote environments. By implementing user feedback from this study, a follow-up study explores how the environment can passively support referencing, where we discovered the role that virtual referencing plays during collaboration. A third study was conducted in order to better understand the effectiveness of giving and interpreting references using a virtual pointer; the results suggest the need for participants to be parallel with the arrow vector (strengthening the argument for shared viewpoints), as well as the importance of shadows in non-stereoscopic environments. Our contributions include a framework for analyzing the domain of inter-referential awareness, the development of novel referencing techniques, the presentation and analysis of our findings from multiple user studies, and a set of guidelines to help designers support this form of awareness

Augmented Reality Interfaces for Procedural Tasks

Procedural tasks involve people performing established sequences of activities while interacting with objects in the physical environment to accomplish particular goals. These tasks span almost all aspects of human life and vary greatly in their complexity. For some simple tasks, little cognitive assistance is required beyond an initial learning session in which a person follows one-time compact directions, or even intuition, to master a sequence of activities. In the case of complex tasks, procedural assistance may be continually required, even for the most experienced users. Approaches for rendering this assistance employ a wide range of written, audible, and computer-based technologies. This dissertation explores an approach in which procedural task assistance is rendered using augmented reality. Augmented reality integrates virtual content with a user's natural view of the environment, combining real and virtual objects interactively, and aligning them with each other. Our thesis is that an augmented reality interface can allow individuals to perform procedural tasks more quickly while exerting less effort and making fewer errors than other forms of assistance. This thesis is supported by several significant contributions yielded during the exploration of the following research themes: What aspects of AR are applicable and beneficial to the procedural task problem? In answering this question, we developed two prototype AR interfaces that improve procedural task accomplishment. The first prototype was designed to assist mechanics carrying out maintenance procedures under field conditions. An evaluation involving professional mechanics showed our prototype reduced the time required to locate procedural tasks and resulted in fewer head movements while transitioning between tasks. Following up on this work, we constructed another prototype that focuses on providing assistance in the underexplored psychomotor phases of procedural tasks. This prototype presents dynamic and prescriptive forms of instruction and was evaluated using a demanding and realistic alignment task. This evaluation revealed that the AR prototype allowed participants to complete the alignment more quickly and accurately than when using an enhanced version of currently employed documentation systems. How does the user interact with an AR application assisting with procedural tasks? The application of AR to the procedural task problem poses unique user interaction challenges. To meet these challenges, we present and evaluate a novel class of user interfaces that leverage naturally occurring and otherwise unused affordances in the native environment to provide a tangible user interface for augmented reality applications. This class of techniques, which we call Opportunistic Controls, combines hand gestures, overlaid virtual widgets, and passive haptics to form an interface that was proven effective and intuitive during quantitative evaluation. Our evaluation of these techniques includes a qualitative exploration of various preferences and heuristics for Opportunistic Control-based designs

Remembering the City: An Augmented Reality Reconstruction of Memory, Power, and Identity in Ho Chi Minh City through Cartography & Architecture

Cartography and architecture are official channels that facilitate remembrance in Ho Chi Minh City. Maps and buildings serve as sites for actors of memory to manipulate the city\u27s narratives and shape its collective identity. Power enables the production of space and knowledge through sites of memory. The ruling regimes of Ho Chi Minh City have leveraged control over the natural environment and the local population to create new forms of materials that propagate their ideologies and ideals for the city. Alterations to the natural and built environments in the city legitimize the authorities\u27 official narratives for its history and future developments. This project explores the context and subtext of urban memory and its formation, using critical augmented reality to visualize the sites of memory. The design of the supplementary augmented reality application takes into consideration the computational theory behind the technology and the development tools for digital historical narratives. In addition, as this study investigates the complicity of science in promoting colonialism, imperialism, nationalism and uninformed nostalgia within the urban setting, it also critiques the use of a new form of technology, augmented reality, in memory formation and other historical processes. Augmented reality offers unprecedented potentials for history and other disciplines thanks to its accessibility and performance; however, the pitfalls of technology require developers and users to remain aware of the implications and assumptions behind each design

User-centred design of smartphone augmented reality in urban tourism context.

Exposure to new and unfamiliar environments is a necessary part of nearly everyone’s life. Effective communication of location-based information through various locationbased service interfaces (LBSIs) became a key concern for cartographers, geographers, human-computer interaction (HCI) and professional designers alike. Much attention is directed towards Augmented Reality (AR) interfaces. Smartphone AR browsers deliver information about physical objects through spatially registered virtual annotations and can function as an interface to (geo)spatial and attribute data. Such applications have considerable potential for tourism. Recently, the number of studies discussing the optimal placement and layout of AR content increased. Results, however, do not scale well to the domain of urban tourism, because: 1) in any urban destination, many objects can be augmented with information; 2) each object can be a source of a substantial amount of information; 3) the incoming video feed is visually heterogeneous and complex; 4) the target user group is in an unfamiliar environment; 5) tourists have different information needs from urban residents. Adopting a User-Centred Design (UCD) approach, the main aim of this research project was to make a theoretical contribution to design knowledge relevant to effective support for (geo)spatial knowledge acquisition in unfamiliar urban environments. The research activities were divided in four (iterative) stages: (1) theoretical, (2) requirements analysis, (3) design and (4) evaluation. After critical analysis of existing literature on design of AR, the theoretical stage involved development of a theoretical user-centred design framework, capturing current knowledge in several relevant disciplines. In the second stage, user requirements gathering was carried out through a field quasi experiment where tourists were asked to use AR browsers in an unfamiliar for them environment. Qualitative and quantitative data were used to identify key relationships, extend the user-centred design framework and generate hypotheses about effective and efficient design. In the third stage, several design alternatives were developed and used to test the hypotheses through a laboratory-based quantitative study with 90 users. The results indicate that information acquisition through AR browsers is more effective and efficient if at least one element within the AR annotation matches the perceived visual characteristics or inferred non-visual attributes of target physical objects. Finally, in order to ensure that all major constructs and relationships are identified, qualitative evaluation of AR annotations was carried out by HCI and GIS domain-expert users in an unfamiliar urban tourism context. The results show that effective information acquisition in urban tourism context will depend on the visual design and delivered content through AR annotations for both visible and non-visible points of interest. All results were later positioned within existing theory in order to develop a final conceptual user-centred design framework that shifts the perspective towards a more thorough understanding of the overall design space for mobile AR interfaces. The dissertation has theoretical, methodological and practical implications. The main theoretical contribution of this thesis is to Information Systems Design Theory. The developed framework provides knowledge regarding the design of mobile AR. It can be used for hypotheses generation and further empirical evaluations of AR interfaces that facilitate knowledge acquisition in different types of environments and for different user groups. From a methodological point of view, the described userbased studies showcase how a UCD approach could be applied to design and evaluation of novel smartphone interfaces within the travel and tourism domain. Within industry the proposed framework could be used as a frame of reference by designers and developers who are not familiar with knowledge acquisition in urban environments and/or mobile AR interfaces

Understanding VR/AR in marketing & sales for B2B: an explorative study

Abstract. The research explored the impact of immersive reality technologies, particularly VR and AR, in marketing and sales for the B2B sector. Study interests were fuelled by both an industrial hype and vehement investments on these technologies, especially in the last five years. However, the potential of these technologies is still unexplored and widely misunderstood by businesses as the innovations are slowly taking off. Additionally, existing literature showed a need to clearly define various simulated realities in the continuum, including VR and AR, as well as a general misunderstanding of the potential of immersive reality technologies, and a shortage of studies covering holistically different VR/AR applications in marketing, especially for the B2B sector. Therefore, this research aims to bridge the gap between managerial and academic’ understanding by providing a holistic framework explaining the impact of immersive reality technologies in B2B marketing and sales and provide a clear distinction between VR and AR in the Virtuality-reality continuum. This research also aims to assist marketers and managers in embracing these technologies to better meet the needs of future generations. The study adopted a qualitative exploratory approach allowing researchers to gain an in-depth understanding of the topic from an industrial perspective. The study used an abductive thematic analysis approach to analyse empirical results and ten semi-structured interviews with eleven VR/AR providers for primary data collection. Results were structured based on four main themes, namely: VR and AR definitions, customer technology adoption factors, VR/AR impact and applications on B2B marketing, and last, VR/AR impact on sales performance outcomes. This study contributes to the existing literature by proposing a tentative definition for each terminology “VR” and “AR” separately that merges academic perspectives and industry insights. Overall, empirical study suggests that immersive reality technologies can affect both marketing activities and sales performance outcomes for the B2B sector. However, successfully embracing these technologies calls for collaboration to overcome financial, technical and social barriers while also enhancing aspects like the user experience to step out of the still immature VR/AR market. VR and AR together have an impact on marketing for B2B by enhancing customization, non-verbal communication, learning and experiential marketing while also creating memorable experiences that stick in the minds of the consumer. Concerning the customer’s purchasing journey, this study extends existing literature to cover all customer purchasing stages, including the pre-purchase, purchase and post-purchase. Results emphasize the pre-purchase phase as the most impacted by immersive reality technologies, followed by post-purchase and purchase stages, respectively. Finally, this study suggests that the use of VR/AR as sales support tools can yield positive efficiency returns through higher task performance and a reduction in sales-related costs, and positive effectiveness returns through greater customer and social engagement, stronger collaborative business relationships and the enhancement of proactive (sales planning) and reactive (adaptive selling) behaviours in the sales process

Human factors in instructional augmented reality for intravehicular spaceflight activities and How gravity influences the setup of interfaces operated by direct object selection

In human spaceflight, advanced user interfaces are becoming an interesting mean to facilitate human-machine interaction, enhancing and guaranteeing the sequences of intravehicular space operations. The efforts made to ease such operations have shown strong interests in novel human-computer interaction like Augmented Reality (AR). The work presented in this thesis is directed towards a user-driven design for AR-assisted space operations, iteratively solving issues arisen from the problem space, which also includes the consideration of the effect of altered gravity on handling such interfaces.Auch in der bemannten Raumfahrt steigt das Interesse an neuartigen Benutzerschnittstellen, um nicht nur die Mensch-Maschine-Interaktion effektiver zu gestalten, sondern auch um einen korrekten Arbeitsablauf sicherzustellen. In der Vergangenheit wurden wiederholt Anstrengungen unternommen, Innenbordarbeiten mit Hilfe von Augmented Reality (AR) zu erleichtern. Diese Arbeit konzentriert sich auf einen nutzerorientierten AR-Ansatz, welcher zum Ziel hat, die Probleme schrittweise in einem iterativen Designprozess zu lösen. Dies erfordert auch die Berücksichtigung veränderter Schwerkraftbedingungen