Using natural user interfaces to support synchronous distributed collaborative work

Synchronous Distributed Collaborative Work (SDCW) occurs when group members work together at the same time from different places together to achieve a common goal. Effective SDCW requires good communication, continuous coordination and shared information among group members. SDCW is possible because of groupware, a class of computer software systems that supports group work. Shared-workspace groupware systems are systems that provide a common workspace that aims to replicate aspects of a physical workspace that is shared among group members in a co-located environment. Shared-workspace groupware systems have failed to provide the same degree of coordination and awareness among distributed group members that exists in co-located groups owing to unintuitive interaction techniques that these systems have incorporated. Natural User Interfaces (NUIs) focus on reusing natural human abilities such as touch, speech, gestures and proximity awareness to allow intuitive human-computer interaction. These interaction techniques could provide solutions to the existing issues of groupware systems by breaking down the barrier between people and technology created by the interaction techniques currently utilised. The aim of this research was to investigate how NUI interaction techniques could be used to effectively support SDCW. An architecture for such a shared-workspace groupware system was proposed and a prototype, called GroupAware, was designed and developed based on this architecture. GroupAware allows multiple users from distributed locations to simultaneously view and annotate text documents, and create graphic designs in a shared workspace. Documents are represented as visual objects that can be manipulated through touch gestures. Group coordination and awareness is maintained through document updates via immediate workspace synchronization, user action tracking via user labels and user availability identification via basic proxemic interaction. Members can effectively communicate via audio and video conferencing. A user study was conducted to evaluate GroupAware and determine whether NUI interaction techniques effectively supported SDCW. Ten groups of three members each participated in the study. High levels of performance, user satisfaction and collaboration demonstrated that GroupAware was an effective groupware system that was easy to learn and use, and effectively supported group work in terms of communication, coordination and information sharing. Participants gave highly positive comments about the system that further supported the results. The successful implementation of GroupAware and the positive results obtained from the user evaluation provides evidence that NUI interaction techniques can effectively support SDCW

Frameworks for enhancing temporal interface behaviour through software architectural design

The work reported in this thesis is concerned with understanding aspects of temporal behaviour. A large part of the thesis is based on analytical studies of temporal properties and interface and architectural concerns. The main areas covered include: i. analysing long-term human processes and the impact of interruptions and delays ii. investigating how infrastructures can be designed to support synchronous fast pace activity iii.design of the Getting-to-Know (GtK) experimental notification server The work is motivated by the failure of many collaborative systems to effectively manage the temporal behaviour at the interface level, as they often assume that the interaction is taking place over fast, reliable local area networks. However, the Web has challenged this assumption and users are faced with frequent network-related delays. The nature of cooperative work increases the importance of timing issues. Collaborative users require both rapid feedback of their own actions and timely feedthrough of other actions. Although it may appear that software architectures are about the internals of system design and not a necessary concern for the user interface, internal details do show up at the surface in non-functional aspects, such as timing. The focus of this work is on understanding the behavioural aspects and how they are influenced by the infrastructure. The thesis has contributed to several areas of research: (a)the study of long-term work processes generated a trigger analysis technique for task decomposition in HCI (b)the analysis of architectures was later applied to investigate architectural options for mobile interfaces (c)the framework for notification servers commenced a design vocabulary in CSCW for the implementation of notification services, with the aim of improving design (d)the impedance matching framework facilitate both goal-directed feedthrough and awareness In particular, (c) and (d) have been exercised in the development of the GtK separable notification server

This is it ! : Indicating and looking in collaborative work at distance

Little is known of the interplay between deixis and eye movements in remote collaboration. This paper presents quantitative results from an experiment where participant pairs had to collaborate at a distance using chat tools that differed in the way messages could be enriched with spatial information from the map in the shared workspace. We studied how the availability of what we defined as an Explicit Referencing mechanism (ER) affected the coordination of the eye movements of the participants. The manipulation of the availability of ER did not produce any significant difference on the gaze coupling. However, we found a primary relation between the pairs recurrence of eye movements and their task performance. Implications for design are discussed

Improving command selection in smart environments by exploiting spatial constancy

With the a steadily increasing number of digital devices, our environments are becoming increasingly smarter: we can now use our tablets to control our TV, access our recipe database while cooking, and remotely turn lights on and off. Currently, this Human-Environment Interaction (HEI) is limited to in-place interfaces, where people have to walk up to a mounted set of switches and buttons, and navigation-based interaction, where people have to navigate on-screen menus, for example on a smart-phone, tablet, or TV screen. Unfortunately, there are numerous scenarios in which neither of these two interaction paradigms provide fast and convenient access to digital artifacts and system commands. People, for example, might not want to touch an interaction device because their hands are dirty from cooking: they want device-free interaction. Or people might not want to have to look at a screen because it would interrupt their current task: they want system-feedback-free interaction. Currently, there is no interaction paradigm for smart environments that allows people for these kinds of interactions. In my dissertation, I introduce Room-based Interaction to solve this problem of HEI. With room-based interaction, people associate digital artifacts and system commands with real-world objects in the environment and point toward these real-world proxy objects for selecting the associated digital artifact. The design of room-based interaction is informed by a theoretical analysis of navigation- and pointing-based selection techniques, where I investigated the cognitive systems involved in executing a selection. An evaluation of room-based interaction in three user studies and a comparison with existing HEI techniques revealed that room-based interaction solves many shortcomings of existing HEI techniques: the use of real-world proxy objects makes it easy for people to learn the interaction technique and to perform accurate pointing gestures, and it allows for system-feedback-free interaction; the use of the environment as flat input space makes selections fast; the use of mid-air full-arm pointing gestures allows for device-free interaction and increases awareness of other’s interactions with the environment. Overall, I present an alternative selection paradigm for smart environments that is superior to existing techniques in many common HEI-scenarios. This new paradigm can make HEI more user-friendly, broaden the use cases of smart environments, and increase their acceptance for the average user

A Review of How Space Affords Socio-Cognitive Processes during Collaboration

This paper reviews the literature about social and cognitive functions of spatial features used when collaborating in both physical and virtual settings. Those concepts come from various fields like social, cognitive as well as environmental psychology or CSCW (Computer Supported Collaborative Work). We briefly summarize the social and cognitive affordances of spatial features like distance, proxemics, co-presence, visibility or activity in the context of physical and virtual space. This review aims at grounding in an explicit framework the way human beings use space to support social interactions. This can be use as a starting point design efficient applications that take spatial context into account

Annotations of maps in collaborative work at a distance

This thesis inquires how map annotations can be used to sustain remote collaboration. Maps condense the interplay of space and communication, solving linguistic references by linking conversational content to the actual places to which it refers. This is a mechanism people are accustomed to. When we are face-to-face, we can point to things around us. However, at a distance, we need to recreate a context that can help disambiguate what we mean. A map can help recreate this context. However other technological solutions are required to allow deictic gestures over a shared map when collaborators are not co-located. This mechanism is here termed Explicit Referencing. Several systems that allow sharing maps annotations are reviewed critically. A taxonomy is then proposed to compare their features. Two filed experiments were conducted to investigate the production of collaborative annotations of maps with mobile devices, looking for the reasons why people might want to produce these notes and how they might do so. Both studies led to very disappointing results. The reasons for this failure are attributed to the lack of a critical mass of users (social network), the lack of useful content, and limited social awareness. More importantly, the study identified a compelling effect of the way messages were organized in the tested application, which caused participants to refrain from engaging in content-driven explorations and synchronous discussions. This last qualitative observation was refined in a controlled experiment where remote participants had to solve a problem collaboratively, using chat tools that differed in the way a user could relate an utterance to a shared map. Results indicated that team performance is improved by the Explicit Referencing mechanisms. However, when this is implemented in a way that is detrimental to the linearity of the conversation, resulting in the visual dispersion or scattering of messages, its use has negative consequences for collaborative work at a distance. Additionally, an analysis of the eye movements of the participants over the map helped to ascertain the interplay of deixis and gaze in collaboration. A primary relation was found between the pair's recurrence of eye movements and their task performance. Finally, this thesis presents an algorithm that detects misunderstandings in collaborative work at a distance. It analyses the movements of collaborators' eyes over the shared map, their utterances containing references to this workspace, and the availability of "remote" deictic gestures. The algorithm associates the distance between the gazes of the emitter and gazes of the receiver of a message with the probability that the recipient did not understand the message

Socio-cognitive functions of space in collaborative settings : a literature review about Space, Cognition and Collaboration

This document reports on a brief literature review about social and cognitive functions of spatial features used when collaborating in both physical and virtual settings. Those concepts come from various fields like social, cognitive as well as environmental psychology or CSCW (Computer Supported Collaborative Work). I briefly summarize the social and cognitive affordances of spatial features like distance, proxemics, co-presence, visibility or activity in the context of physical and virtual space. The way human beings employ those features finally allows to give insights about potential avenues of research

Evaluating groupware usability at the cognitive level of human action

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2010This dissertation explores the importance of the cognitive level of human action in the evaluation and improvement of groupware usability. This research is motivated by the problem that current methods focus on the rational and social levels of human action and yet an increasing number of users relies on computers to fulfil collaborative tasks dominated by perceptual, cognitive, and motor skill. The first contribution of this research is a groupware interface model that leverages existing knowledge on cognitive-level behaviour with single-user interfaces by expanding its application to multi-user interfaces. To do this, I show that the key differences between users interacting with the computer and interacting with other users through the computer can be supported by specialised groupware information flows and input/output devices. The second contribution of this dissertation is a pair of methods for predicting groupware usability at the cognitive level of human action. The first method applies to scenarios of collaboration occurring routinely in shared workspaces. The second aims at capturing the intertwined nature of mixed-focus collaboration, encompassing shared and private workspaces. I use the methods to evaluate and compare the usability of competing designs in four scenarios of collaboration. The methods do not require user testing or functioning prototypes, so they can be integrated into the iterative process of interactive systems design. The third contribution of this research is the evaluation of an attentive electronic brainstorming tool, which implements a novel attentive device that adjusts the delivery of group awareness information according to users’ natural task switching between doing individual work and attending to the group. I present results from a laboratory experiment, which indicate that groups produced 9.6% more ideas when compared to the immediate broadcast of ideas and provide evidence suggesting that the usability improvement was due to the mitigation of information overload.Esta dissertação explora a importância do nível cognitivo da actividade humana, no qual as tarefas demoram segundos a realizar e são tipicamente repetitivas, na avaliação e melhoria da usabilidade de sistemas de trabalho cooperativo suportado por computador, também designados por groupware. Estes sistemas de computadores permitem que grupos de interesse, como amigos e colegas, possam partilhar e organizar actividades de forma flexível e económica, onde o tempo e a distância deixam de ser obstáculos à colaboração. Alguns exemplos de groupware incluem os mensageiros instantâneos, usados por centenas de milhões de pessoas no mundo inteiro, os jogos multi-utilizador, que já atingiram cerca de dezasseis milhões de jogadores, bem como uma gama cada vez mais alargada de aplicações de escritório que estão a ser disponibilizadas na Internet. Com base nesta evidência, uma assumpção desta dissertação é que os sistemas de groupware estão a ficar cada vez mais ubíquos. O problema abordado nesta investigação é que os métodos actuais de avaliação da usabilidade de groupware omitem o nível cognitivo da actividade humana, e, no entanto, as nossas características psicológicas, como a percepção, cognição, e capacidade motora, dominam a execução de tarefas de colaboração rápidas, mas normalmente muito repetitivas. Uma consequência desta situação é que faltam instrumentos aos designers e investigadores de groupware que lhes permitam fazer optimizações de usabilidade de granularidade fina. Isto acontece porque os métodos actuais de avaliação da usabilidade visam tarefas colaborativas de relativa longa duração (que demoram minutos, horas, ou mais, a completar) e, portanto, baseiam-se em abstracções para conter o grau de complexidade da avaliação. Desta forma, as optimizações tendem a abranger vários passos de colaboração de granularidade fina de uma só vez, o que causa problemas porque a usabilidade de sistemas de groupware, como na maioria dos sistemas computacionais, está inerentemente ligada aos detalhes da interface com o utilizador. Estas optimizações, mesmo que de pequena expressão individual, podem acarretar um efeito multiplicador significativo dado o crescente número de utilizadores de groupware, especialmente na Internet. Outra consequência do nível cognitivo da acção humana ser negligenciado das avaliações de usabilidade de groupware é que o design da interface com o utilizador pode estar indevidamente alinhado com as características psicológicas humanas, o que pode fazer com que as tarefas colaborativas exijam uma carga de trabalho que excede as nossas capacidades limitadas de processamento de informação. Aliás, os utilizadores que realizam trabalho em grupo estão particularmente expostos a uma sobrecarga de informação porque têm de acompanhar o que se passa no grupo para além de realizarem trabalho individual, isto é, têm de dividir a atenção entre múltiplos fluxos de informação. Esta carga de trabalho pode penalizar a usabilidade dos sistemas de groupware devido ao aumento da probabilidade dos utilizadores não serem capazes de colaborar adequadamente. Dada esta situação, a minha questão de investigação é: como fazer avaliações ao nível cognitivo da actividade humana para melhorar a usabilidade de tarefas colaborativas realizadas através de sistemas de groupware? As avaliações de usabilidade ao nível cognitivo são bastante conhecidas no contexto das aplicações mono-utilizador, ao ponto de um conjunto de conhecimentos da psicologia aplicada ter sido reunido em modelos de engenharia de desempenho humano que predizem tempos de execução numa gama variada de tarefas de interacção pessoa-máquina. Estes modelos foram já, inclusivamente, aplicados no contexto de trabalho de grupo, mas sempre com a limitação de os utilizadores estarem restringidos a papéis individualistas e de a colaboração ficar de fora dos limites do sistema ou então ser abstraída. Em contraste, nesta dissertação estou interessado em avaliar as tarefas de colaboração realizadas através do sistema de groupware. A primeira contribuição desta investigação é um modelo da interface do groupware, o qual alavanca o conhecimento existente sobre o comportamento humano com interfaces mono-utilizador, baseado em modelos de engenharia que predizem o desempenho humano, através da expansão da sua aplicação a interfaces multi-utilizador. Para fazer isto mostro que as diferenças fundamentais entre os utilizadores interagirem com o computador (para trabalharem individualmente) e interagirem com outros utilizadores através do computador (para colaborar) podem ser suportadas por fluxos de informação e dispositivos de input/output especializados. Este modelo tem como propósito ajudar o designer a organizar o espaço de soluções numa gama alargada de sistemas de groupware. A segunda contribuição desta dissertação é um par de métodos para avaliar a usabilidade de sistemas de groupware ao nível cognitivo da actividade humana. O primeiro método é aplicável a cenários críticos de colaboração que ocorram rotineiramente em espaços de trabalho partilhados e define usabilidade em termos do tempo necessário para executar tarefas colaborativas, tal como estimado pelos modelos de engenharia de desempenho humano. Na dissertação aplico este método para avaliar e comparar a usabilidade de alternativas de design em três casos de colaboração em espaços partilhados. O segundo método visa capturar a natureza complexa e entrecruzada da colaboração que abrange tanto espaços partilhados como privados, bem como capturar os objectivos frequentemente conflituosos dos utilizadores enquanto estão a trabalhar individualmente ou quando estão a interagir com o grupo. Para fazer isto, combino estimativas de tempos de execução de tarefas com contribuições dessas tarefas para a progressão do grupo em direcção a um objectivo comum, em termos de produtividade individual, oportunidades criadas para os outros, e restrições para o trabalho de outros utilizadores. Na dissertação aplico este método a um jogo colaborativo, e mostro que, se para alguma outra coisa mais, este método serve para forçar o designer de groupware a pensar sobre as contrapartidas entre uma interface que permite aos utilizadores enquanto indivíduos serem mais produtivos e outra que permite um melhor desempenho do grupo enquanto um todo. Os dois métodos de avaliação não requerem testes com utilizadores ou a construção de protótipos de groupware para produzirem resultados de usabilidade, o que atesta a sua natureza formativa, e permite a sua integração no processo iterativo de design de sistemas interactivos. A terceira contribuição desta investigação é a avaliação da usabilidade de um sistema de groupware atentivo, que implementa um novo dispositivo de gestão da atenção humana, chamado opportunity seeker , o qual tem como propósito mitigar a sobrecarga de informação em cenários de colaboração síncrona, isto é, em que todos os elementos do grupo estão a trabalhar em simultâneo. O opportunity seeker intercepta e guarda numa memória tampão a informação de estado sobre o grupo e ajusta automaticamente a entrega dessa informação a cada utilizador em função da alternância natural entre este estar a realizar trabalho individual e estar a prestar atenção ao grupo. Na dissertação mostro como este dispositivo pode ser adaptado e instalado numa ferramenta electrónica para geração de ideias, chamada ABTool, e como a fronteira entre os dois estados de atenção pode ser detectada através de actividade no teclado. Para avaliar os efeitos do opportunity seeker na usabilidade da ferramenta ABTool, realizei uma experiência de laboratório em que pedi a grupos de voluntários para submeterem ideias em paralelo o mais rapidamente possível, e recolhi evidência de que quando os grupos estiveram sob a influência do opportunity seeker o número de ideias geradas aumentou em 9.6% em comparação com a condição em que todas as ideias eram imediatamente difundidas por todos os utilizadores. Adicionalmente, levei a cabo uma análise post-hoc que mostra que o opportunity seeker reduziu o número de entregas de ideias em 44.1%, pois combinou as ideias em pequenos lotes, e que isso se traduziu em 54.7% mais tempo para os utilizadores escreverem ideias sem serem interrompidos pela recepção de ideias de outros utilizadores. Nestas condições, os utilizadores foram 18.8% mais rápidos a alternar entre a escrita de uma ideia, o que fizeram em 16.3% menos tempo, e ler novas ideias de outros utilizadores. Estes resultados evidenciam que o opportunity seeker criou condições para mitigar a sobrecarga de informação e mostram que a usabilidade de sistemas de groupware pode ser melhorada através de avaliações focadas nas limitações da capacidade de processamento de informação humana. Com este conjunto de contribuições, mostrei que o nível cognitivo da actividade humana tem um papel determinante na avaliação da usabilidade de sistemas de groupware, complementando os níveis racional e social que têm sido tradicionalmente considerados por outros métodos de avaliação.Portuguese Foundation for Science and Technology(Fundação para a Ciência e Tecnologia), through project PTDC/EIA/67589/2006 and the Multiannual Funding Programme

Virtual Valcamonica: collaborative exploration of prehistoric petroglyphs and their surrounding environment in multi-user virtual reality

In this paper, we present a novel, multi-user, virtual reality environment for the interactive, collaborative 3D analysis of large 3D scans and the technical advancements that were necessary to build it: a multi-view rendering system for large 3D point clouds, a suitable display infrastructure and a suite of collaborative 3D interaction techniques. The cultural heritage site of Valcamonica in Italy with its large collection of prehistoric rock-art served as an exemplary use case for evaluation. The results show that our output-sensitive level-of-detail rendering system is capable of visualizing a 3D dataset with an aggregate size of more than 14 billion points at interactive frame rates. The system design in this exemplar application results from close exchange with a small group of potential users: archaeologists with expertise in rock-art and allows them to explore the prehistoric art and its spatial context with highly realistic appearance. A set of dedicated interaction techniques was developed to facilitate collaborative visual analysis. A multi-display workspace supports the immediate comparison of geographically distributed artifacts. An expert review of the final demonstrator confirmed the potential for added value in rock-art research and the usability of our collaborative interaction techniques