HOW TO INTERACT WITH AR HEAD MOUNTED DEVICES IN CARE WORK? A STUDY COMPARING HANDHELD TOUCH (HANDS-ON) AND GESTURE (HANDS-FREE) INTERACTION

In this paper, we describe a study investigating augmented reality (AR) to support caregivers. We implemented a system called Care Lenses that supports various care tasks on AR head-mounted devices. For its application, one question was how caregivers could interact with the system while providing care, that is, while using one or both hands for care tasks. Therefore, we compared two mechanisms to interact with the CareLenses (handheld touch similar to touchpads and touchscreens and head gestures). We found that certain head gestures were difficult to apply in practice, but that except from this head gesture support was as usable and useful as handheld touch interaction, although the study participants were much more familiar with the handheld touch control. We conclude that head gestures can be a good means to enable AR support in care, and we provide design considerations to make them more applicable in practice

How to Interact with Augmented Reality Head Mounted Devices in Care Work? A Study Comparing Handheld Touch (Hands-on) and Gesture (Hands-free) Interaction

In this paper, we investigate augmented reality (AR) to support caregivers. We implemented a system called Care Lenses that supported various care tasks on AR head-mounted devices. For its application, one question concerned how caregivers could interact with the system while providing care (i.e., while using one or both hands for care tasks). Therefore, we compared two mechanisms to interact with the Care Lenses (handheld touch similar to touchpads and touchscreens and head gestures). We found that head gestures were difficult to apply in practice, but except for that the head gesture support was as usable and useful as handheld touch interaction, although the study participants were much more familiar with the handheld touch control. We conclude that head gestures can be a good means to enable AR support in care, and we provide design considerations to make them more applicable in practice

Conceitos e métodos para apoio ao desenvolvimento e avaliação de colaboração remota utilizando realidade aumentada

Remote Collaboration using Augmented Reality (AR) shows great potential to establish a common ground in physically distributed scenarios where team-members need to achieve a shared goal. However, most research efforts in this field have been devoted to experiment with the enabling technology and propose methods to support its development. As the field evolves, evaluation and characterization of the collaborative process become an essential, but difficult endeavor, to better understand the contributions of AR. In this thesis, we conducted a critical analysis to identify the main limitations and opportunities of the field, while situating its maturity and proposing a roadmap of important research actions. Next, a human-centered design methodology was adopted, involving industrial partners to probe how AR could support their needs during remote maintenance. These outcomes were combined with literature methods into an AR-prototype and its evaluation was performed through a user study. From this, it became clear the necessity to perform a deep reflection in order to better understand the dimensions that influence and must/should be considered in Collaborative AR. Hence, a conceptual model and a humancentered taxonomy were proposed to foster systematization of perspectives. Based on the model proposed, an evaluation framework for contextualized data gathering and analysis was developed, allowing support the design and performance of distributed evaluations in a more informed and complete manner. To instantiate this vision, the CAPTURE toolkit was created, providing an additional perspective based on selected dimensions of collaboration and pre-defined measurements to obtain “in situ” data about them, which can be analyzed using an integrated visualization dashboard. The toolkit successfully supported evaluations of several team-members during tasks of remote maintenance mediated by AR. Thus, showing its versatility and potential in eliciting a comprehensive characterization of the added value of AR in real-life situations, establishing itself as a generalpurpose solution, potentially applicable to a wider range of collaborative scenarios.Colaboração Remota utilizando Realidade Aumentada (RA) apresenta um enorme potencial para estabelecer um entendimento comum em cenários onde membros de uma equipa fisicamente distribuídos precisam de atingir um objetivo comum. No entanto, a maioria dos esforços de investigação tem-se focado nos aspetos tecnológicos, em fazer experiências e propor métodos para apoiar seu desenvolvimento. À medida que a área evolui, a avaliação e caracterização do processo colaborativo tornam-se um esforço essencial, mas difícil, para compreender as contribuições da RA. Nesta dissertação, realizámos uma análise crítica para identificar as principais limitações e oportunidades da área, ao mesmo tempo em que situámos a sua maturidade e propomos um mapa com direções de investigação importantes. De seguida, foi adotada uma metodologia de Design Centrado no Humano, envolvendo parceiros industriais de forma a compreender como a RA poderia responder às suas necessidades em manutenção remota. Estes resultados foram combinados com métodos da literatura num protótipo de RA e a sua avaliação foi realizada com um caso de estudo. Ficou então clara a necessidade de realizar uma reflexão profunda para melhor compreender as dimensões que influenciam e devem ser consideradas na RA Colaborativa. Foram então propostos um modelo conceptual e uma taxonomia centrada no ser humano para promover a sistematização de perspetivas. Com base no modelo proposto, foi desenvolvido um framework de avaliação para recolha e análise de dados contextualizados, permitindo apoiar o desenho e a realização de avaliações distribuídas de forma mais informada e completa. Para instanciar esta visão, o CAPTURE toolkit foi criado, fornecendo uma perspetiva adicional com base em dimensões de colaboração e medidas predefinidas para obter dados in situ, que podem ser analisados utilizando o painel de visualização integrado. O toolkit permitiu avaliar com sucesso vários colaboradores durante a realização de tarefas de manutenção remota apoiada por RA, permitindo mostrar a sua versatilidade e potencial em obter uma caracterização abrangente do valor acrescentado da RA em situações da vida real. Sendo assim, estabelece-se como uma solução genérica, potencialmente aplicável a uma gama diversificada de cenários colaborativos.Programa Doutoral em Engenharia Informátic

On-Demand Collaboration in Programming

In programming, on-demand assistance occurs when developers seek support for their tasks as needed. Traditionally, this collaboration happens within teams and organizations in which people are familiar with the context of requests and tasks. More recently, this type of collaboration has become ubiquitous outside of teams and organizations, due to the success of paid online crowdsourcing marketplaces (e.g., Upwork) and free online question-answering websites (e.g., Stack Overflow). Thousands of requests are posted on these platforms on a daily basis, and many of them are not addressed in a timely manner for a variety of reasons, including requests that often lack sufficient context and access to relevant artifacts. In consequence, on-demand collaboration often results in suboptimal productivity and unsatisfactory user experiences. This dissertation includes three main parts: First, I explored the challenges developers face when requesting help from or providing assistance to others on demand. I have found seven common types of requests (e.g., seeking code examples) that developers use in various projects when an on-demand agent is available. Compared to studying existing supporting systems, I suggest eight key system features to enable more effective on-demand remote assistance for developers. Second, driven by these findings, I designed and developed two systems: 1) CodeOn, a system that enables more effective task hand-offs (e.g., rich context capturing) between end-user developers and remote helpers than exciting synchronous support systems by allowing asynchronous responses to on-demand requests; and 2) CoCapture, a system that enables interface designers to easily create and then accurately describe UI behavior mockups, including changes they want to propose or questions they want to ask about an aspect of the existing UI. Third, beyond software development assistance, I also studied intelligent assistance for embedded system development (e.g., Arduino) and revealed six challenges (e.g., communication setup remains tedious) that developers have during on-demand collaboration. Through an imaginary study, I propose four design implications to help develop future support systems with embedded system development. This thesis envisions a future in which developers in all kinds of domains can effortlessly make context-rich, on-demand requests at any stage of their development processes, and qualified agents (machine or human) can quickly be notified and orchestrate their efforts to promptly respond to the requests.PHDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/166144/1/yanchenm_1.pd

Stabilized Annotations for Mobile Remote Assistance

Recent mobile technology has provided new opportunities for creating remote assistance systems. However, mobile support systems present a particular challenge: both the camera and display are held by the user, leading to shaky video. When pointing or drawing annotations, this means that the desired target often moves, causing the gesture to lose its intended meaning. To address this problem, this thesis investigates an annotation stabilization technique, which allows annotations to stick to their intended location. I studied two different forms of annotation systems, with both tablets and head-mounted displays. To differentiate my work from the prior research, I considered a number of task factors that might influence system performance in remote assistance scenarios. My analysis suggests that stabilized annotations and head-mounted displays are only beneficial in certain situations. I conclude with reflections on system limitations and potential future work