Examining the role of smart TVs and VR HMDs in synchronous at-a-distance media consumption

This article examines synchronous at-a-distance media consumption from two perspectives: How it can be facilitated using existing consumer displays (through TVs combined with smartphones), and imminently available consumer displays (through virtual reality (VR) HMDs combined with RGBD sensing). First, we discuss results from an initial evaluation of a synchronous shared at-a-distance smart TV system, CastAway. Through week-long in-home deployments with five couples, we gain formative insights into the adoption and usage of at-a-distance media consumption and how couples communicated during said consumption. We then examine how the imminent availability and potential adoption of consumer VR HMDs could affect preferences toward how synchronous at-a-distance media consumption is conducted, in a laboratory study of 12 pairs, by enhancing media immersion and supporting embodied telepresence for communication. Finally, we discuss the implications these studies have for the near-future of consumer synchronous at-a-distance media consumption. When combined, these studies begin to explore a design space regarding the varying ways in which at-a-distance media consumption can be supported and experienced (through music, TV content, augmenting existing TV content for immersion, and immersive VR content), what factors might influence usage and adoption and the implications for supporting communication and telepresence during media consumption

Investigating the Impact of Cooperative Communication Mechanics on Player Performance in Portal 2

Cooperative communication mechanics, such as avatar gestures or in-game visual pointers, enable player collaboration directly through gameplay. We currently lack a deeper understanding of how players use cooperative communication mechanics, and whether they can effectively supplement or even supplant traditional voice and chat communication. The present research investigated player communication in Portal 2 by testing the game’s native cooperative communication mechanics for dyads of players in custom test chambers. Following our initial hypothesis, players functioned best when they had access to both cooperative communication mechanics and voice. We found that players preferred voice communication, but perceived cooperative communication mechanics as necessary to coordinate interdependent actions.NMSU Natural Sciences and Engineering Research Council of Canada (NSERC Discovery grant #RGPIN-418622-2012) and the Social Sciences and Humanities Research Council of Canada (SSHRC grant #895-2011-1014 (IMMERSe)Peer-reviewe

Mutual engagement and collocation with shared representations

In this paper we explore the use of shared representations to support creative activities, focussing on collaborative music making. We examine the effect that user interface features of shared representations have on mutual engagement and show that providing shared awareness mechanisms increases mutual engagement. In particular, we show through an empirical study of 78 participants that providing cues to identity and shared pointers increases mutual engagement between participants, but together these features can overwhelm users. We also demonstrate that support for free-form annotation and spatial interaction with shared representations mediates interaction and helps participants to manage their collaborative activity effectively. In this paper we develop several measures mutual engagement and demonstrate their use to assess the design of shared representations. A key contribution of this paper is the development of a measure of collocation of participant interaction which indicates mutual engagement. The findings of the study have implications beyond the domain of collaborative music making and we outline some design guidelines for mutually engaging shared representations

Research for Design of Playful Mobile Services for Social Experiences between Nearby Strangers

Having positive interpersonal interactions is a fundamental human need and source of well-being. While fulfilling this need is usually associated with strong ties, research has shown that meaningful social experiences are not limited to those. This research explores the largely untapped social potential of nearby strangers and ways that mobile services can be designed to take advantage of these social opportunities. Play and playfulness appear to be particularly worthwhile ways to achieve this end: play is meaningful in itself (i.e., does not require an external goal) and takes place outside the context of real life. In addition, playful design tends to make digital services more engaging. This research focuses on playfulness as a design quality and explores the social implications of playful mobile services for nearby strangers. This doctoral thesis asks two research questions: What kind of social experiences emerge between nearby strangers from the use of playful mobile services? How can playful mobile services be designed to encourage social experiences between nearby strangers? The research contributes to the field of human-computer interaction and provides insights into mobile service design through six research articles. Two of the studies charted expected experiences with early-stage mobile application concepts for playful interaction between nearby strangers. One of these concepts was further developed into a fully functional mobile application, and a large-scale, in-the-wild study was arranged to explore the actual social experiences it generated. Two of the studies investigated social experiences between nearby strangers in the context of commercial mobile games. The sixth study explored the design space of playful interactions between nearby strangers through co-design workshops. The playful mobile services investigated in this research were found to induce various behaviors that resulted in social experiences between nearby strangers. Examples of such behaviors are the active exploration of the outside world, community building, communicating and collaborating with strangers, and interacting in crowds. I found that playful and social experiences such as competition, surprise, curiosity, inspiration, and benevolence motivated individuals to use these services

Biometric storyboards: a games user research approach for improving qualitative evaluations of player experience

Developing video games is an iterative and demanding process. It is difficult to achieve the goal of most video games — to be enjoyable, engaging and to create revenue for game developers — because of many hard-to-evaluate factors, such as the different ways players can interact with the game. Understanding how players behave during gameplay is of vital importance to developers and can be uncovered in user tests as part of game development. This can help developers to identify and resolve any potential problem areas before release, leading to a better player experience and possibly higher game review scores and sales. However, traditional user testing methods were developed for function and efficiency oriented applications. Hence, many traditional user testing methods cannot be applied in the same way for video game evaluation. This thesis presents an investigation into the contributions of physiological measurements in user testing within games user research (GUR). GUR specifically studies the interaction between a game and users (players) with the aim to provide feedback for developers to help them to optimise the game design of their title. An evaluation technique called Biometric Storyboards is developed, which visualises the relationships between game events, player feedback and changes in a player’s physiological state. Biometric Storyboards contributes to the field of human-computer interaction and GUR in three important areas: (1) visualising mixedmeasures of player experience, (2) deconstructing game design by analysing game events and pace, (3) incremental improvement of classic user research techniques (such as interviews and physiological measurements). These contributions are described in practical case studies, interviews with game developers and laboratory experiments. The results show this evaluation approach can enable games user researchers to increase the plausibility and persuasiveness of their reports and facilitate developers to better deliver their design goals. Biometric Storyboards is not aimed at replacing existing methods, but to extend them with mixed methods visualisations, to provide powerful tools for games user researchers and developers to better understand and communicate player needs, interactions and experiences. The contributions of this thesis are directly applicable for user researchers and game developers, as well as for researchers in user experience evaluation in entertainment systems

Gaze Awareness in Computer-Mediated Collaborative Physical Tasks

Human eyes play an important role in everyday social interactions. However, the cues provided by eye movements are often missing or difficult to interpret in computer-mediated remote collaboration. Motivated by the increasing availability of gaze-tracking devices in the consumer market and the growing need for improved remotecollaboration systems, this thesis evaluated the value of gaze awareness in a number of video-based remote-collaboration situations. This thesis comprises six publications which enhance our understanding of the everyday use of gaze-tracking technology and the value of shared gaze to remote collaborations in the physical world. The studies focused on a variety of collaborative scenarios involving different camera configurations (stationary, handheld, and head-mounted cameras), display setups (screen-based and projection displays), mobility requirements (stationary and mobile tasks), and task characteristics (pointing and procedural tasks). The aim was to understand the costs and benefits of shared gaze in video-based collaborative physical tasks. The findings suggest that gaze awareness is useful in remote collaboration for physical tasks. Shared gaze enables efficient communication of spatial information, helps viewers to predict task-relevant intentions, and enables improved situational awareness. However, different contextual factors can influence the utility of shared gaze. Shared gaze was more useful when the collaborative task involved communicating pointing information instead of procedural information, the collaborators were mutually aware of the shared gaze, and the quality of gaze-tracking was accurate enough to meet the task requirements. In addition, the results suggest that the collaborators’ roles can also affect the perceived utility of shared gaze. Methodologically, this thesis sets a precedent in shared gaze research by reporting the objective gaze data quality achieved in the studies and also provides tools for other researchers to objectively view gaze data quality in different research phases. The findings of this thesis can contribute towards designing future remote-collaboration systems; towards the vision of pervasive gaze-based interaction; and towards improved validity, repeatability, and comparability of research involving gaze trackers

LAIX-score : a design framework for live audience interaction management systems

This study focuses on computer-supported live audience interaction. In conventional lectures audience interacts explicitly with the performer for example by waving hand and asking question directly or clapping hands. For decades, non digital audience response systems have enabled simple multiple option audience interaction patterns. Modern mobile personal computing devices, digital projectors, wireless networks and real time software platforms enable creation of new kinds of interaction patterns that can significantly increase the amount of audience interaction during events. Audience interaction can make events for example more engaging and productive. This research presents a design framework for computer-supported live audience interaction called the LAIX-score. LAIX stands for Live Audience Interac(X)tion and the “score” refers to the musical notation language. Musical notation has been an inspiration for the development of the framework and illustrates how LAIX-score is intended as generic and practical framework for coordinating live audience interaction similarly as musical notation is generic and practical framework for coordinating musical performances. However, while musical notation is important inspiration, it is not the core reference for the LAIX-score. LAIX-score core references are the live audio mixing and live light control frameworks, which are technologyenabled frameworks for supporting and producing live performances. The LAIX-score framework is composed of five core elements: Interaction activities, interface channels, state control matrix, temporal management of interactions and participant’s identity management. These five core elements compose a concrete and comprehensive framework that can be directly applied in the design of live audience interaction management system and in the development of live audience interaction production practices. The research is a constructive and practice-led in the wild research (Chapter 2) that borrows aspects from design research, artistic research and human-computer interaction research. The LAIX-score framework is based on three core requirements identified during a five years of practice-led domain exploration (Chapter 3). (Requirement 1) Live audience interaction must support different kinds of interaction patterns. Hence, the framework should acknowledge that live audience interaction is more than questions and answers (Q&A) and poll type interaction patterns. (Requirement 2) Live audience interaction must support different roles. Hence, the role configuration in live audience interaction can include several different performer, audience and orchestrator roles. (Requirement 3) Live audience interaction framework must also support different kinds and parallel functions live audience interaction function. Hence, in the same event production live audience interaction may be used for example for audience activation, workshop facilitation, participatory decision making and catalyzing social networking, and these functions may take place concurrently. None of the existing live audience interaction systems satisfy all of the core requirements. This is explained in more detail in Section 4.2. Lack of adequate designs that meets the above mentioned criterias justifies the development of a new design framework. The LAIX-score (Chapter 5) follows a two dimensional matrix type control framework, which is called state control matrix. Also the core references, live audio mixing and live light control (Sections 4.3 –4.5), have similar control framework. Rows in the state control matrix are called as interaction activities. Columns in the state control matrix are interface channels, which is the system equivalent for supporting different roles and user interfaces (requirement 2). The matrix is used for visibility control of the interaction activities. The visibility of interaction activities can be manipulated independently in each interface channel. The matrix form satisfies the three core requirements. The first requirement is satisfied since the matrix format is agnostic to what kind of interactions are controlled in the system. The second requirement is satisfied since the matrix format allows introduction of new roles and there is fundamentally no fixed number for rows. The third requirement is satisfied since multiple interaction activities can be active in any channel and each interaction activity state can be controlled independently. The core framework is implemented as a functional live audience interaction management system called Presemo (version 4) (Chapter 6). The evaluation of the design of Presemo reveals more detailed fivetier structure for the control of interaction activities . The interaction activity control levels in LAIX-score design framework are (1.) creation and deletion, (2.) state control matrix, (3.) interaction pattern specific control, (4.) content management and (5.) presentation management. Presemo is limited implementation of the framework since the basic version supports only four interface channels. Presemo is a commercial level system and it has been utilized in thousands of live audience interaction situations and we have used it to produce more than 100 live audience interaction productions. The research investigates four case studies in more detail (Chapter 7). These four case studies are produced in different environments and this way demonstrate the generic qualities of Presemo and the LAIX-score design framework. One of the case study production focuses on professional event productions, another in application of Presemo in University context, third one focuses on use of live audience interaction in large scale computer-supported workshops and fourth one presents use of live audience interaction techniques in a pervasive adventure designed for K 12 students. The case studies validate the three core requirements and identifies 11 new additional requirements for the LAIX-score matrix. The case studies also reveal a more detailed interface channel structure. The revised LAIX-score design framework divides interface channels in three groups: organizer channels, audience channels and screen channels. Organizer channels combines performer and orchestrator roles, since these are roles that have some kind of control over interaction activities. Audience interface channels can be divided in groups. Screen channels are public channels whereas organizer and audience channels are personal channels. The 11 new requirements are further elaborated as two new core elements of the LAIX-score framework (Chapter 8): temporal management and identity management. Temporal management is divided in three parts; the functional cue list realizes the future temporal management, state control matrix realizes the real time management, and the production log realizes the management of past events. Identity management core element can be visualized as a table that lists all identities on one axis and different identity parameters on another axis. The study has identified six different types of identity attribute categories: identifiers, group membership, access rights, privacy settings, other identity and profile parameters and score attributes used for gamification. Identity attributes and privacy settings are used to manage identity parameters in order to achieve privacy and anonymity, which are important characteristics for most live audience interaction productions. Case studies have shown also that gamification is an important feature for live audience interaction. The core objective of the research is to create a framework for live audience interaction that could be generic and practical. As uch, the study is directly relevant extensive case reference of a live audience interaction system researchers and live audience interaction producers. The framework is adequately described so that any developer can utilize it in their own live audience interaction system designs. Methodologically the research has some areas of improvements mainly due to challenges in organizing data collection in demanding production environments (Section 9.3). These problems are common for in the wild research. The strengths of this research are extensive coverage of the live audience interaction domain and concrete validation of the framework as a production level implemented software system. While we have been developing the LAIX-score framework we have also identified several other research topics for live audience interaction (explained in Section 10.3) that are beyond the scope of the LAIX-score framework. There are for example several issues related to human and organizational factors of live audience interaction that are not covered in the LAIX-score framework, which is designed for the development of the computer system and production practices. These other research topics demonstrate how live audience interaction domain is still emerging domain with many interesting research possibilities. During the study, we have been involved in commercial development of live audience interaction. The business and marketing development (Section 10.4) will most probably be the driving force for the development of new interaction patterns, live audience interaction production formats, professional practices and generally new applications for live audience interaction. The further business and marketing development will define how organizations can adopt live audience interaction techniques and integrate them in to their communication and participation processes. The study proposes that standards organization would start defining protocols for live audience interaction. Details of wider adoption will ultimately define what kind of further research is relevant and feasible in the live audience interaction domain. The five core elements of the LAIX-score are integrated to each other and together they compose a comprehensive framework that can be used as design guideline for generic live audience interaction system (LAIMS). A LAIMS that is based on LAIX-score can host modularly different kinds of interaction patterns (Section 10.2). Modular approach can be also called s interaction agnostic approach. The modular approach may have several implications: modular approach makes development of new interaction patterns easier, support event productions that host different live audience interaction approaches, support sustainable system evolution and establishment of management practices for live audience interaction productions

Conceitos e métodos para apoio ao desenvolvimento e avaliação de colaboração remota utilizando realidade aumentada

Remote Collaboration using Augmented Reality (AR) shows great potential to establish a common ground in physically distributed scenarios where team-members need to achieve a shared goal. However, most research efforts in this field have been devoted to experiment with the enabling technology and propose methods to support its development. As the field evolves, evaluation and characterization of the collaborative process become an essential, but difficult endeavor, to better understand the contributions of AR. In this thesis, we conducted a critical analysis to identify the main limitations and opportunities of the field, while situating its maturity and proposing a roadmap of important research actions. Next, a human-centered design methodology was adopted, involving industrial partners to probe how AR could support their needs during remote maintenance. These outcomes were combined with literature methods into an AR-prototype and its evaluation was performed through a user study. From this, it became clear the necessity to perform a deep reflection in order to better understand the dimensions that influence and must/should be considered in Collaborative AR. Hence, a conceptual model and a humancentered taxonomy were proposed to foster systematization of perspectives. Based on the model proposed, an evaluation framework for contextualized data gathering and analysis was developed, allowing support the design and performance of distributed evaluations in a more informed and complete manner. To instantiate this vision, the CAPTURE toolkit was created, providing an additional perspective based on selected dimensions of collaboration and pre-defined measurements to obtain “in situ” data about them, which can be analyzed using an integrated visualization dashboard. The toolkit successfully supported evaluations of several team-members during tasks of remote maintenance mediated by AR. Thus, showing its versatility and potential in eliciting a comprehensive characterization of the added value of AR in real-life situations, establishing itself as a generalpurpose solution, potentially applicable to a wider range of collaborative scenarios.Colaboração Remota utilizando Realidade Aumentada (RA) apresenta um enorme potencial para estabelecer um entendimento comum em cenários onde membros de uma equipa fisicamente distribuídos precisam de atingir um objetivo comum. No entanto, a maioria dos esforços de investigação tem-se focado nos aspetos tecnológicos, em fazer experiências e propor métodos para apoiar seu desenvolvimento. À medida que a área evolui, a avaliação e caracterização do processo colaborativo tornam-se um esforço essencial, mas difícil, para compreender as contribuições da RA. Nesta dissertação, realizámos uma análise crítica para identificar as principais limitações e oportunidades da área, ao mesmo tempo em que situámos a sua maturidade e propomos um mapa com direções de investigação importantes. De seguida, foi adotada uma metodologia de Design Centrado no Humano, envolvendo parceiros industriais de forma a compreender como a RA poderia responder às suas necessidades em manutenção remota. Estes resultados foram combinados com métodos da literatura num protótipo de RA e a sua avaliação foi realizada com um caso de estudo. Ficou então clara a necessidade de realizar uma reflexão profunda para melhor compreender as dimensões que influenciam e devem ser consideradas na RA Colaborativa. Foram então propostos um modelo conceptual e uma taxonomia centrada no ser humano para promover a sistematização de perspetivas. Com base no modelo proposto, foi desenvolvido um framework de avaliação para recolha e análise de dados contextualizados, permitindo apoiar o desenho e a realização de avaliações distribuídas de forma mais informada e completa. Para instanciar esta visão, o CAPTURE toolkit foi criado, fornecendo uma perspetiva adicional com base em dimensões de colaboração e medidas predefinidas para obter dados in situ, que podem ser analisados utilizando o painel de visualização integrado. O toolkit permitiu avaliar com sucesso vários colaboradores durante a realização de tarefas de manutenção remota apoiada por RA, permitindo mostrar a sua versatilidade e potencial em obter uma caracterização abrangente do valor acrescentado da RA em situações da vida real. Sendo assim, estabelece-se como uma solução genérica, potencialmente aplicável a uma gama diversificada de cenários colaborativos.Programa Doutoral em Engenharia Informátic

An Empirical Investigation of Virtual World Projects and Metaverse Technology Capabilities

Metaverses are immersive three-dimensional virtual worlds (VWs) where people interact with each other and their environment, using the metaphor of the real world but without its physical limitations. Unique technology capabilities of metaverses have the potential to enhance the conduct of virtual projects, but little is known about virtual worlds in this context. Virtual project teams struggle in meeting stated project outcomes due to challenges related to communication, shared understanding, and coordination. One way to address these challenges is to consider the use of emerging technologies, such as metaverses, to minimize the impact on virtual project teams. Applying a theoretical foundation for virtual teams in metaverses that includes both technology capabilities and the social interaction that takes place in the metaverse environment, we conducted an empirical investigation of project teams in a virtual world setting. The study examined the interplay of communication, representation, interaction, and team process tools with behaviors that led to role clarity, shared understanding, and coordination. While each individual technology capability contributed to project execution and outcomes, much of the power of the environment emerged through the interplay of social behaviors and technology capabilities. The results have intriguing implications for how metaverse technology capabilities might provide new ways to address gaps in the current research and practice of virtual project management and virtual teams