The Challenges of Evaluating the Usability of Augmented Reality (AR)

Augmented reality (AR) is a new and emerging technology that could benefit from evaluating its usability to better the user’s experience with the device or application. This is often done through usability testing and heuristic evaluations. However, AR technology presents some challenges when completing these usability evaluations. Practitioners need to keep in mind the hardware limitations of AR devices that may not be present with other computerized technology, consistency of the users’ environment plays a larger role in the AR experience, recognize that a novelty effect may occur and affect subjective scores, and choose heuristic sets that will best evaluate AR applications. Practitioners need to be aware of these challenges and overcome them to accurately assess the usability of these products to gain insights about what should be changed to make the overall experience with the product better

A Comparison of Quantitative and Qualitative Data from a Formative Usability Evaluation of an Augmented Reality Learning Scenario

The proliferation of augmented reality (AR) technologies creates opportunities for the devel-opment of new learning scenarios. More recently, the advances in the design and implementation of desktop AR systems make it possible the deployment of such scenarios in primary and secondary schools. Usability evaluation is a precondition for the pedagogical effectiveness of these new technologies and requires a systematic approach for finding and fixing usability problems. In this paper we present an approach to a formative usability evaluation based on heuristic evaluation and user testing. The basic idea is to compare and integrate quantitative and qualitative measures in order to increase confidence in results and enhance the descriptive power of the usability evaluation report.augmented reality, multimodal interaction, e-learning, formative usability evaluation, user testing, heuristic evaluation

Tutor In-sight: Guiding and Visualizing Students Attention with Mixed Reality Avatar Presentation Tools

Remote conferencing systems are increasingly used to supplement or even replace in-person teaching. However, prevailing conferencing systems restrict the teacher’s representation to a webcam live-stream, hamper the teacher’s use of body-language, and result in students’ decreased sense of co-presence and participation. While Virtual Reality (VR) systems may increase student engagement, the teacher may not have the time or expertise to conduct the lecture in VR. To address this issue and bridge the requirements between students and teachers, we have developed Tutor In-sight, a Mixed Reality (MR) avatar augmented into the student’s workspace based on four design requirements derived from the existing literature, namely: integrated virtual with physical space, improved teacher’s co-presence through avatar, direct attention with auto-generated body language, and usable workfow for teachers. Two user studies were conducted from the perspectives of students and teachers to determine the advantages of Tutor In-sight in comparison to two existing conferencing systems, Zoom (video-based) and Mozilla Hubs (VR-based). The participants of both studies favoured Tutor In-sight. Among others, this main fnding indicates that Tutor Insight satisfed the needs of both teachers and students. In addition, the participants’ feedback was used to empirically determine the four main teacher requirements and the four main student requirements in order to improve the future design of MR educational tools

Framework to Enhance Teaching and Learning in System Analysis and Unified Modelling Language

Cowling, MA ORCiD: 0000-0003-1444-1563; Munoz Carpio, JC ORCiD: 0000-0003-0251-5510Systems Analysis modelling is considered foundational for Information and Communication Technology (ICT) students, with introductory and advanced units included in nearly all ICT and computer science degrees. Yet despite this, novice systems analysts (learners) find modelling and systems thinking quite difficult to learn and master. This makes the process of teaching the fundamentals frustrating and time intensive. This paper will discuss the foundational problems that learners face when learning Systems Analysis modelling. Through a systematic literature review, a framework will be proposed based on the key problems that novice learners experience. In this proposed framework, a sequence of activities has been developed to facilitate understanding of the requirements, solutions and incremental modelling. An example is provided illustrating how the framework could be used to incorporate visualization and gaming elements into a Systems Analysis classroom; therefore, improving motivation and learning. Through this work, a greater understanding of the approach to teaching modelling within the computer science classroom will be provided, as well as a framework to guide future teaching activities

Moving sounds and sonic moves : exploring interaction quality of embodied music mediation technologies through a user-centered perspective

This research project deals with the user-experience related to embodied music mediation technologies. More specifically, adoption and policy problems surrounding new media (art) are considered, which arise from the usability issues that to date pervade new interfaces for musical expression. Since the emergence of new wireless mediators and control devices for musical expression, there is an explicit aspiration of the creative industries and various research centers to embed such technologies into different areas of the cultural industries. The number of applications and their uses have exponentially increased over the last decade. Conversely, many of the applications to date still suffer from severe usability problems, which not only hinder the adoption by the cultural sector, but also make culture participants take a rather cautious, hesitant, or even downright negative stance towards these technologies. Therefore, this thesis takes a vantage point that is in part sociological in nature, yet has a link to cultural studies as well. It combines this with a musicological frame of reference to which it introduces empirical user-oriented approaches, predominantly taken from the field of human-computer-interaction studies. This interdisciplinary strategy is adopted to cope with the complex nature of digital embodied music controlling technologies. Within the Flanders cultural (and creative) industries, opportunities of systems affiliated with embodied interaction are created and examined. This constitutes an epistemological jigsaw that looks into 1) “which stakeholders require what various levels of involvement, what interactive means and what artistic possibilities?”, 2) “the way in which artistic aspirations, cultural prerequisites and operational necessities of (prospective) users can be defined?”, 3) “how functional, artistic and aesthetic requirements can be accommodated?”, and 4) “how quality of use and quality of experience can be achieved, quantified, evaluated and, eventually, improved?”. Within this multi-facetted problem, the eventual aim is to assess the applicability of the foresaid technology, both from a theoretically and empirically sound basis, and to facilitate widening and enhancing the adoption of said technologies. Methodologically, this is achieved by 1) applied experimentation, 2) interview techniques, 3) self-reporting and survey research, 4) usability evaluation of existing devices, and 5) human-computer interaction methods applied – and attuned – to the specific case of embodied music mediation technologies. Within that scope, concepts related to usability, flow, presence, goal assessment and game enjoyment are scrutinized and applied, and both task- and experience-oriented heuristics and metrics are developed and tested. In the first part, covering three chapters, the general context of the thesis is given. In the first chapter, an introduction to the topic is offered and the current problems are enumerated. In the second chapter, a broader theoretical background is presented of the concepts that underpin the project, namely 1) the paradigm of embodiment and its connection to musicology, 2) a state of the arts concerning new interfaces for musical expression, 3) an introduction into HCI-usability and its application domain in systematic musicology, 4) an insight into user-centered digital design procedures, and 5) the challenges brought about by e-culture and digitization for the cultural-creative industries. In the third chapter, the state of the arts concerning the available methodologies related to the thesis’ endeavor is discussed, a set of literature-based design guidelines are enumerated and from this a conceptual model is deduced which is gradually presented throughout the thesis, and fully deployed in the “SoundField”-project (as described in Chapter 9). The following chapters, contained in the second part of the thesis, give a quasi-chronological overview of how methodological concepts have been applied throughout the empirical case studies, aimed specifically at the exploration of the various aspects of the complex status quaestionis. In the fourth chapter, a series of application-based tests, predominantly revolving around interface evaluation, illustrate the complex relation between gestural interfaces and meaningful musical expression, advocating a more user-centered development approach to be adopted. In the fifth chapter, a multi-purpose questionnaire dubbed “What Moves You” is discussed, which aimed at creating a survey of the (prospective) end-users of embodied music mediation technologies. Therefore, it primarily focused on cultural background, musical profile and preferences, views on embodied interaction, literacy of and attitudes towards new technology and participation in digital culture. In the sixth chapter, the ethnographical studies that accompanied the exhibition of two interactive art pieces, entitled "Heart as an Ocean" & "Lament", are discussed. In these studies, the use of interview and questionnaire methodologies together with the presentation and reception of interactive art pieces, are probed. In the seventh chapter, the development of the collaboratively controlled music-game “Sync-In-Team” is presented, in which interface evaluation, presence, game enjoyment and goal assessment are the pivotal topics. In the eighth chapter, two usability studies are considered, that were conducted on prototype systems/interfaces, namely a heuristic evaluation of the “Virtual String” and a usability metrics evaluation on the “Multi-Level Sonification Tool”. The findings of these two studies in conjunction with the exploratory studies performed in association with the interactive art pieces, finally gave rise to the “SoundField”-project, which is recounted in full throughout the ninth chapter. The integrated participatory design and evaluation method, presented in the conceptual model is fully applied over the course of the “SoundField”-project, in which technological opportunities and ecological validity and applicability are investigated through user-informed development of numerous use cases. The third and last part of the thesis renders the final conclusions of this research project. The tenth chapter sets out with an epilogue in which a brief overview is given on how the state of the arts has evolved since the end of the project (as the research ended in 2012, but the research field has obviously moved on), and attempts to consolidate the implications of the research studies with some of the realities of the Flemish cultural-creative industries. Chapter eleven continues by discussing the strengths and weaknesses of the conceptual model throughout the various stages of the project. Also, it comprises the evaluation of the hypotheses, how the assumptions that were made held up, and how the research questions eventually could be assessed. Finally, the twelfth and last chapter concludes with the most important findings of the project. Also, it discusses some of the implications on cultural production, artistic research policy and offers an outlook on future research beyond the scope of the “SoundField” project

The Benefits of Extended Reality for Technical Communication : Utilizing XR for Maintenance Documentation Creation and Delivery

The main goal of this dissertation is to explore the benefits of extended reality for technical communication. Both of these fields offer opportunities and also pose challenges to each other, and this dissertation provides insight into this relationship. The research was initiated by the author’s personal interest in both fields and also human-technology interaction and user needs in general. Even though this is an academic dissertation, it is first and foremost a practitioner’s view of these evolving technologies and their potential uses in industry and, specifically, in industrial maintenance and technical communication. Under the umbrella of extended reality and technical communication, this dissertation focuses on two main themes. The first part studies virtual reality as a technology to facilitate collaboration and digital content creation for technical documentation in industrial companies, and the second part explores the possibilities of augmented reality and smart glasses as a delivery channel for maintenance instructions. The developed concepts were tested by domain experts in user tests. The overall results of testing were positive, and domain experts expressed enthusiasm toward the concepts and technologies in general. The technical documentation process is an inherently collaborative process involving stakeholders from different teams and organizations, and virtual reality was evaluated to have a positive effect on that process, especially in the case of globally scattered teams. The developed tools were also rated positively for digital content creation. Therefore, virtual reality offers many benefits for technical documentation creation, an area where it has not been utilized until now. On the augmented reality side, domain experts were generally enthusiastic about the use of smart glasses even though the technologies are not yet mature enough for field use in industrial maintenance. Furthermore, the results show that content created in the technical communications industry standard, DITA XML, works well when delivered to smart glasses, and the same content can be single sourced to other delivery channels. The use of DITA XML, therefore, eliminates the need to tailor content for each delivery channel separately, and offers an effective way to create and update content for AR applications in industrial companies. This, in turn, can advance the use of AR technologies and related devices in field operations in industrial companies. In conclusion, the findings of this dissertation show that the fields of technical communication and extended reality have a significant amount of synergy. In this dissertation I establish use cases and guidelines for these areas