Initial perceptions of a casual game to crowdsource facial expressions in the wild

The performance of affective computing systems often depend on the quality of the image databases they are trained on. However, creating good quality training databases is a laborious activity. In this paper, we evaluate BeFaced, a tile matching casual tablet game that enables massive crowdsourcing of facial expressions for the purpose of advancing facial expression analysis. The core aspect of BeFaced is game quality, as increased enjoyment and engagement translates to an increased quantity of varied facial expressions obtained. Hence a pilot user study was performed on 18 university students whereby observational and interview data were obtained during playtests. We found that most users enjoyed the game and were intrigued by the novelty in interacting with the facial expression gameplay mechanic, but also uncovered problems with feedback provision and the dynamic difficulty adjustment mechanism. These findings hence provide invaluable insights for the other researchers/ practitioners working on similar crowdsourcing games with a purpose, as well as for the development of BeFaced

BeFaced: A casual game to crowdsource facial expressions in the wild

Creating good quality image databases for affective computing systems is key to most computer vision research, but is unfortunately costly and time-consuming. This paper describes BeFaced, a tile matching casual tablet game that enables massive crowdsourcing of facial expressions to advance facial expression analysis. BeFaced uses state-of-the-art facial expression tracking technology with dynamic difficulty adjustment to keep the player engaged and hence obtain a large and varied face dataset. CHI attendees will be able to experience a novel game interface that uses the iPad's front camera to track and capture facial expressions as the primary player input, and also investigate how the game design in general enables massive crowdsourcing in an extensible manner

A game to crowdsource data for affective computing

This game submission describes BeFaced, a tile matching casual tablet game that enables massive crowdsourcing of facial expressions to advance affective computing. BeFaced uses state-of-theart facial expression tracking technology with dynamic difficulty adjustment to keep the player engaged and hence obtain a large and varied face dataset. FDG attendees will experience a novel affective game input interface and also investigate how the game design enables massive crowdsourcing in an extensible manner

The platformer experience dataset

Player modeling and estimation of player experience have become very active research fields within affective computing, human computer interaction, and game artificial intelligence in recent years. For advancing our knowledge and understanding on player experience this paper introduces the Platformer Experience Dataset (PED) - the first open-access game experience corpus - that contains multiple modalities of user data of Super Mario Bros players. The open-access database aims to be used for player experience capture through context-based (i.e. game content), behavioral and visual recordings of platform game players. In addition, the database contains demographical data of the players and self-reported annotations of experience in two forms: ratings and ranks. PED opens up the way to desktop and console games that use video from webcameras and visual sensors and offer possibilities for holistic player experience modeling approaches that can, in turn, yield richer game personalization.peer-reviewe

Learning facial-expression models with crowdsourcing

The computational power is increasing day by day. Despite that, there are some tasks that are still difficult or even impossible for a computer to perform. For example, while identifying a facial expression is easy for a human, for a computer it is an area in development. To tackle this and similar issues, crowdsourcing has grown as a way to use human computation in a large scale. Crowdsourcing is a novel approach to collect labels in a fast and cheap manner, by sourcing the labels from the crowds. However, these labels lack reliability since annotators are not guaranteed to have any expertise in the field. This fact has led to a new research area where we must create or adapt annotation models to handle these weaklylabeled data. Current techniques explore the annotators’ expertise and the task difficulty as variables that influences labels’ correction. Other specific aspects are also considered by noisy-labels analysis techniques. The main contribution of this thesis is the process to collect reliable crowdsourcing labels for a facial expressions dataset. This process consists in two steps: first, we design our crowdsourcing tasks to collect annotators labels; next, we infer the true label from the collected labels by applying state-of-art crowdsourcing algorithms. At the same time, a facial expression dataset is created, containing 40.000 images and respective labels. At the end, we publish the resulting dataset

Affective Game Computing: A Survey

This paper surveys the current state of the art in affective computing principles, methods and tools as applied to games. We review this emerging field, namely affective game computing, through the lens of the four core phases of the affective loop: game affect elicitation, game affect sensing, game affect detection and game affect adaptation. In addition, we provide a taxonomy of terms, methods and approaches used across the four phases of the affective game loop and situate the field within this taxonomy. We continue with a comprehensive review of available affect data collection methods with regards to gaming interfaces, sensors, annotation protocols, and available corpora. The paper concludes with a discussion on the current limitations of affective game computing and our vision for the most promising future research directions in the field

A Design Exploration of Affective Gaming

Physiological sensing has been a prominent fixture in games user research (GUR) since the late 1990s, when researchers began to explore its potential to enhance and understand experience within digital game play. Since these early days, it has been widely argued that “affective gaming”—in which gameplay is influenced by a player’s emotional state—can enhance player experience by integrating physiological sensors into play. In this thesis, I conduct a design exploration of the field of affective gaming by first, systematically exploring the field and creating a framework (the affective game loop) to classify existing literature; and second by presenting two design probes, to probe and explore the design space of affective games contextualized within the affective game loop: In the Same Boat and Commons Sense. The systematic review explored this unique design space of affective gaming, opening up future avenues for exploration. The affective game loop was created as a way to classify the physiological signals and sensors most commonly used in prior literature within the context of how they are mapped into the gameplay itself. Findings suggest that the physiological input mappings can be more action-based (e.g., affecting mechanics in the game such as the movement of the character) or more context-based (e.g., affecting things like environmental or difficulty variables in the game). Findings also suggested that while the field has been around for decades, there is still yet to be any commercial successes, so does physiological interaction really heighten player experience? This question instigated the design of the two probes, exploring ways to implement these mappings and effectively heighten player experience. In the Same Boat (Design Probe One) is an embodied mirroring game designed to promote an intimate interaction, using players’ breathing rate and facial expressions to control movement of a canoe down a river. Findings suggest that playing In the Same Boat fostered the development of affiliation between the players, and that while embodied controls were less intuitive, people enjoyed them more, indicating the potential of embodied controls to foster social closeness in synchronized play over a distance. Commons Sense (Design Probe Two) is a communication modality intended to heighten audience engagement and effectively capture and communicate the audience experience, using a webcam-based heart rate detection software that takes an average of each spectator’s heart rate as input to affect in-game variables such as lighting and sound design, and game difficulty. Findings suggest that Commons Sense successfully facilitated the communication of audience response in an online entertainment context—where these social cues and signals are inherently diminished. In addition, Commons Sense is a communication modality that can both enhance a play experience while offering a novel way to communicate. Overall, findings from this design exploration shows that affective games offer a novel way to deliver a rich gameplay experience for the player

Doing Illness: Cancer Narratives in Digital Media

Through three case studies, the article explores how digital media have been used in recent years to depict and comprehend experiences of cancer. It first investigates the illness blog, specifically Swedish journalist and musician Kristian Gidlund’s immensely popular blog In My Body, in which he, from 2011 to 2013, shared the narrative of his struggle with an aggressive, incurable, and ultimately deadly stomach cancer. It continues by discussing Italian engineer, artist, and hacker Salvatore Iaconesi’s digital open-source project La Cura – The Cure (2012), which has great relevance from both the digital and the medical humanities perspectives in the way Iaconesi uses his personal narrative of brain cancer to encourage people to join his struggle to find a cure. Finally, it analyzes the American couple Ryan and Amy Green’s videogame That Dragon, Cancer (2016). A game differing significantly from video and computer games in general and from other games taking cancer as their subject by letting the player enter the role of caregiver to a small child dying of cancer. Expanding on Lisa Diedrich’s theoretical concept of “doing illness”, the article emphasizes the performative dimension of narrating illness in digital media, considering how these authors and creators negotiate with narrative, cultural, and medial scripts when portraying their cancer experiences. It highlights the interactive and participatory dimension of doing illness in digital media, by exploring how the blog, open-source project, and videogame both invite and limit the audience’s opportunities to interact and participate with the illness narrative conveyed

Towards a crowdsourced solution for the authoring bottleneck in interactive narratives

Interactive Storytelling research has produced a wealth of technologies that can be employed to create personalised narrative experiences, in which the audience takes a participating rather than observing role. But so far this technology has not led to the production of large scale playable interactive story experiences that realise the ambitions of the field. One main reason for this state of affairs is the difficulty of authoring interactive stories, a task that requires describing a huge amount of story building blocks in a machine friendly fashion. This is not only technically and conceptually more challenging than traditional narrative authoring but also a scalability problem. This thesis examines the authoring bottleneck through a case study and a literature survey and advocates a solution based on crowdsourcing. Prior work has already shown that combining a large number of example stories collected from crowd workers with a system that merges these contributions into a single interactive story can be an effective way to reduce the authorial burden. As a refinement of such an approach, this thesis introduces the novel concept of Crowd Task Adaptation. It argues that in order to maximise the usefulness of the collected stories, a system should dynamically and intelligently analyse the corpus of collected stories and based on this analysis modify the tasks handed out to crowd workers. Two authoring systems, ENIGMA and CROSCAT, which show two radically different approaches of using the Crowd Task Adaptation paradigm have been implemented and are described in this thesis. While ENIGMA adapts tasks through a realtime dialog between crowd workers and the system that is based on what has been learned from previously collected stories, CROSCAT modifies the backstory given to crowd workers in order to optimise the distribution of branching points in the tree structure that combines all collected stories. Two experimental studies of crowdsourced authoring are also presented. They lead to guidelines on how to employ crowdsourced authoring effectively, but more importantly the results of one of the studies demonstrate the effectiveness of the Crowd Task Adaptation approach