A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction

Picking up objects requested by a human user is a common task in human-robot interaction. When multiple objects match the user's verbal description, the robot needs to clarify which object the user is referring to before executing the action. Previous research has focused on perceiving user's multimodal behaviour to complement verbal commands or minimising the number of follow up questions to reduce task time. In this paper, we propose a system for reference disambiguation based on visualisation and compare three methods to disambiguate natural language instructions. In a controlled experiment with a YuMi robot, we investigated real-time augmentations of the workspace in three conditions -- mixed reality, augmented reality, and a monitor as the baseline -- using objective measures such as time and accuracy, and subjective measures like engagement, immersion, and display interference. Significant differences were found in accuracy and engagement between the conditions, but no differences were found in task time. Despite the higher error rates in the mixed reality condition, participants found that modality more engaging than the other two, but overall showed preference for the augmented reality condition over the monitor and mixed reality conditions

In-home and remote use of robotic body surrogates by people with profound motor deficits

By controlling robots comparable to the human body, people with profound motor deficits could potentially perform a variety of physical tasks for themselves, improving their quality of life. The extent to which this is achievable has been unclear due to the lack of suitable interfaces by which to control robotic body surrogates and a dearth of studies involving substantial numbers of people with profound motor deficits. We developed a novel, web-based augmented reality interface that enables people with profound motor deficits to remotely control a PR2 mobile manipulator from Willow Garage, which is a human-scale, wheeled robot with two arms. We then conducted two studies to investigate the use of robotic body surrogates. In the first study, 15 novice users with profound motor deficits from across the United States controlled a PR2 in Atlanta, GA to perform a modified Action Research Arm Test (ARAT) and a simulated self-care task. Participants achieved clinically meaningful improvements on the ARAT and 12 of 15 participants (80%) successfully completed the simulated self-care task. Participants agreed that the robotic system was easy to use, was useful, and would provide a meaningful improvement in their lives. In the second study, one expert user with profound motor deficits had free use of a PR2 in his home for seven days. He performed a variety of self-care and household tasks, and also used the robot in novel ways. Taking both studies together, our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates, and that they can gain benefit with only low-level robot autonomy and without invasive interfaces. However, methods to reduce the rate of errors and increase operational speed merit further investigation.Comment: 43 Pages, 13 Figure

Applying the Technology Acceptance Model to Understand Aviation Students’ Perceptions toward Augmented Reality Maintenance Training Instruction

Augmented Reality (AR) Technology, since its inception, has been enhanced significantly by software and hardware developers, and has been widely utilized in various fields such as manufacturing, entertainment, architecture, commerce and education. In recent years, maintenance instructions developed on the basis of AR technology have demonstrated their potential to positively impact maintenance training and technical tasks in aviation. Features of AR maintenance instructions include interactive content, user-friendly operation interface, enhanced visualization and real-time data feeds. The researchers conducted a case study of forty-one aviation maintenance students at a Midwestern university. The purpose of this study was twofold: to evaluate the Technology Acceptance Model as a valid framework for assessing AR implementation in aviation maintenance training; and to evaluate students’ acceptance of AR instructions in aviation maintenance training. On one hand, The Technology Acceptance Model (TAM) can help identify and prove the relationships among variables affecting aviation students’ perceptions and future use of AR instructions, including perceived usefulness, perceived ease of use, attitude toward using, and intention to use. On the other hand, the results may help establish an initial benchmark for aviation students’ acceptance of AR instructions in a maintenance training setting

Augmented reality in support of intelligent manufacturing – A systematic literature review

Industry increasingly moves towards digitally enabled ‘smart factories’ that utilise the internet of things (IoT) to realise intelligent manufacturing concepts like predictive maintenance or extensive machine to machine communication. A core technology to facilitate human integration in such a system is augmented reality (AR), which provides people with an interface to interact with the digital world of a smart factory. While AR is not ready yet for industrial deployment in some areas, it is already used in others. To provide an overview of research activities concerning AR in certain shop floor operations, a total of 96 relevant papers from 2011 to 2018 are reviewed. This paper presents the state of the art, the current challenges, and future directions of manufacturing related AR research through a systematic literature review and a citation network analysis. The results of this review indicate that the context of research concerning AR gets increasingly broader, especially by addressing challenges when implementing AR solutions.No funding was received

Use of Augmented Reality in Human Wayfinding: A Systematic Review

Augmented reality technology has emerged as a promising solution to assist with wayfinding difficulties, bridging the gap between obtaining navigational assistance and maintaining an awareness of one's real-world surroundings. This article presents a systematic review of research literature related to AR navigation technologies. An in-depth analysis of 65 salient studies was conducted, addressing four main research topics: 1) current state-of-the-art of AR navigational assistance technologies, 2) user experiences with these technologies, 3) the effect of AR on human wayfinding performance, and 4) impacts of AR on human navigational cognition. Notably, studies demonstrate that AR can decrease cognitive load and improve cognitive map development, in contrast to traditional guidance modalities. However, findings regarding wayfinding performance and user experience were mixed. Some studies suggest little impact of AR on improving outdoor navigational performance, and certain information modalities may be distracting and ineffective. This article discusses these nuances in detail, supporting the conclusion that AR holds great potential in enhancing wayfinding by providing enriched navigational cues, interactive experiences, and improved situational awareness.Comment: 52 page

A Framework for Mobile Augmented Reality in Urban Maintenance

Mobile handheld devices such as smartphones have become increasingly powerful in modern times. Because of this, there has been a surge in 3D graphics-heavy mobile applications that aim to provide immersive experiences. An example of this phenomenon would be Augmented Reality (AR) applications, which have been increasingly popular and offer a wide array of use-cases. The ability to merge the real world with the virtual world seamlessly using the built-in camera of the smartphone brings a whole new world of possibilities, which makes it interesting to explore how such a technology could be used to solve real-world problems. This dissertation focuses on applying this technology in the field of urban maintenance. To do so, a mobile AR application was developed, designed to be used by urban maintenance workers as a field-assistance tool. Using any standard smartphone camera, the developed system can accurately detect any equipment and augment it with relevant information and step-by-step instructions on how to do any required maintenance jobs. Alongside this mobile application, a desktop application was also developed with the purposes of creating and authoring the data and augmentations that should be displayed during a given job, called Archer. Lastly, this dissertation proposes a novel approach to automatically detect and minimize the amount of points (checkpoints) at which the application will ask the user to perform a new equipment recognition, which are useful in order to maintain tracking stability as the user modifies the real-world object during the course of the job. The experiments and user tests conducted during the final stages of this dissertation demonstrate the accuracy and practicality of the developed systems, proving that they can effectively be used to greatly improve the workflow of urban maintenance workers.Dispositivos móveis tais como os smartphones têm-se tornado cada vez mais poderosos nos tempos modernos. Como tal, tem havido um grande aumento na quantidade de aplicações móveis 3D com o intuito de fornecer experiências imersivas. Um exemplo desse fenómeno são as aplicações de Realidade Aumentada (RA), as quais se têm tornado cada vez mais populares, oferecendo um vasto leque de casos de uso. A habilidade de fundir o mundo real com o mundo virtual através da câmara de um smartphone traz todo um novo mundo de possibilidades, o que torna interessante a exploração de como esta tecnologia pode ser usada para resolver problemas no mundo real. Esta dissertação foca-se na aplicação desta tecnologia na área da manutenção urbana. Nesse sentido, foi desenvolvida uma aplicação móvel de RA projetada para ser usada como uma ferramenta de assistência em campo por trabalhadores de manutenção urbana. Usando qualquer câmara de smartphone, este sistema consegue detetar qualquer equipamento de forma precisa e aumentá-lo digitalmente com informação relevante e instruções passo-a-passo de como fazer qualquer trabalho de manutenção. Juntamente com esta aplicação móvel, também foi desenvolvida uma aplicação para desktop — chamada Archer — com o intuito de criar e validar os dados e os objetos digitais que serão apresentados na aplicação mobile durante o curso de um trabalho de manutenção. Por fim, esta dissertação apresenta uma nova solução para a deteção e minimização automática dos pontos (checkpoints) em que a aplicação móvel deverá pedir ao utilizador para efetuar um novo reconhecimento do equipamento, os quais são úteis para manter um tracking fiável e estável à medida que o utilizador vai modificando o equipamento durante um trabalho. As experiências e testes com utilizadores efetuados na fase final desta dissertação demonstram a precisão e praticidade dos sistemas desenvolvidos, provando que estes podem efetivamente ser usados para melhorar o workflow dos trabalhadores de manutenção urbana

Augmented reality in support of Industry 4.0—Implementation challenges and success factors

Industrial augmented reality (AR) is an integral part of Industry 4.0 concepts, as it enables workers to access digital information and overlay that information with the physical world. While not being broadly adopted in some applications, the compound annual growth rate of the industrial AR market is projected to grow rapidly. Hence, it is important to understand the issues arising from implementation of AR in industry. This study identifies critical success factors and challenges for industrial AR implementation projects, based on an industry survey. The broadly used technology, organisation, environment (TOE) framework is used as a theoretical basis for the quantitative part of the questionnaire. A complementary qualitative part is used to underpin and extend the findings. It is found that, while technological aspects are of importance, organisational issues are more relevant for industry, which has not been reflected to the same extent in literature.University of Cambridg

The threshold of the real: A site for participatory resistance in Blast Theory's Uncle Roy all around you (2003)

This article examines the collision of virtual and real spaces through simultaneous live and online play in Uncle Roy All Around You, and how this disruption of immersion is used to expose the habitual engagements associated with the digital interface. The nature of the participants' immersion and the subsequent reintegration into the real will be explored, before attempting to articulate what defines this piece as politically resistant, through discussion of a self reflexive participation, which undermines what Baudrillard terms the 'simulated response' (Baudrillard 1985/1988 p.216