Unleashing the Potentials of Immersive Augmented Reality for Software Engineering

In immersive augmented reality (IAR), users can wear a head-mounted display to see computer-generated images superimposed to their view of the world. IAR was shown to be beneficial across several domains, e.g., automotive, medicine, gaming and engineering, with positive impacts on, e.g., collaboration and communication. We think that IAR bears a great potential for software engineering but, as of yet, this research area has been neglected. In this vision paper, we elicit potentials and obstacles for the use of IAR in software engineering. We identify possible areas that can be supported with IAR technology by relating commonly discussed IAR improvements to typical software engineering tasks. We further demonstrate how innovative use of IAR technology may fundamentally improve typical activities of a software engineer through a comprehensive series of usage scenarios outlining practical application. Finally, we reflect on current limitations of IAR technology based on our scenarios and sketch research activities necessary to make our vision a reality. We consider this paper to be relevant to academia and industry alike in guiding the steps to innovative research and applications for IAR in software engineering

Spatial embodied augmented reality: design of AR for spatial productivity applications

Recently there has been a growth in commercial systems for Augmented Reality (AR), such as the Microsoft HoloLens and Magic Leap, which aim to subsume traditionally 2D productivity tasks performed by previous technologies through the use of a 3D User Interface (3DUI). However, there is currently a lack of research into how to design effective AR interfaces and no consensus on how to design general purpose 3DUIs. Augmented reality is an emergent technology with little prior design precedent. As AR becomes more widespread, it becomes apparent that new design paradigms are required to translate potential benefits of these new interfaces. This thesis aims to explore the design of AR productivity software that can exploit the spatial nature of augmented reality to complement the spatial nature of human interaction. To investigate this, three user studies were performed using an email client as a use case scenario. Each study focused on a different aspect in the design of an augmented reality system starting with user elicitation using informance design methods. This progressed to testing of alternative interfaces for three-dimensional document presentation using a formal lab experiment, finally ending with an observation of how users arrange documents in a simulated limited prototype system. The findings of this thesis include: • A new method of user elicitation for augmented reality interface design called “spatial informance design”. • Design recommendations of spatial interface augmentations for email. • Data to support that space can be used to triage email more effectively. • Different layouts of documents in space provide greater or lesser time, accuracy and memorability. • Presentation of document layouts with either an ego or exocentric view alters performance. • Users have a preference to use space over colour to group documents. • Users take advantage of space when a 3rd dimension is available. The conclusion of this thesis is that augmented reality has the potential to improve the user experience over the traditional two-dimensional GUI for knowledge work tasks