Tangible user interfaces and social interaction in children with autism

Tangible User Interfaces (TUIs) offer the potential for new modes of social interaction for children with Autism Spectrum Conditions (ASC). Familiar objects that are embedded with digital technology may help children with autism understand the actions of others by providing feedback that is logical and predictable. Objects that move, playback sound or create sound – thus repeating programmed effects – offer an exciting way for children to investigate objects and their effects. This thesis presents three studies of children with autism interacting with objects augmented with digital technology. Study one looked at Topobo, a construction toy augmented with kinetic memory. Children played with Topobo in groups of three of either Typically Developing (TD) or ASC children. The children were given a construction task, and were also allowed to play with the construction sets with no task. Topobo in the task condition showed an overall significant effect for more onlooker, cooperative, parallel, and less solitary behaviour. For ASC children significantly less solitary and more parallel behaviour was recorded than other play states. In study two, an Augmented Knights Castle (AKC) playset was presented to children with ASC. The task condition was extended to allow children to configure the playset with sound. A significant effect in a small sample was found for configuration of the AKC, leading to less solitary behaviour, and more cooperative behaviour. Compared to non-digital play, the AKC showed reduction of solitary behaviour because of augmentation. Qualitative analysis showed further differences in learning phase, user content, behaviour oriented to other children, and system responsiveness. Tangible musical blocks (‘d-touch’) in study three focused on the task. TD and ASC children were presented with a guided/non-guided task in pairs, to isolate effects of augmentation. Significant effects were found for an increase in cooperative symbolic play in the guided condition, and more solitary functional play was found in the unguided condition. Qualitative analysis highlighted differences in understanding blocks and block representation, exploratory and expressive play, understanding of shared space and understanding of the system. These studies suggest that the structure of the task conducted with TUIs may be an important factor for children’s use. When the task is undefined, play tends to lose structure and the benefits of TUIs decline. Tangible technology needs to be used in an appropriately structured manner with close coupling (the distance between digital housing and digital effect), and works best when objects are presented in familiar form

Exploring expressive augmented reality: The fing AR puppet system for social pretend play

We present “FingAR Puppet”, an Augmented Reality (AR) system enhancing social pretend play by young children. Un-like goal-oriented AR systems that augment reality with in-formative instructions, FingAR Puppet helps children asso-ciate expressive interpretations with immediate reality. Em-pirical results show that FingAR Puppet promotes reasoning about emotional states, communication and divergent think-ing during social pretend play for children 4-6 years old. We suggest that this study opens an interesting space for future AR systems to support complex cognitive and social devel-opment in early childhood. We also identify broader implica-tions from using theories of cognitive development to guide the design of tangible and augmented interactions

InfoInternet for Education in the Global South: A Study of Applications Enabled by Free Information-only Internet Access in Technologically Disadvantaged Areas (authors' version)

This paper summarises our work on studying educational applications enabled by the introduction of a new information layer called InfoInternet. This is an initiative to facilitate affordable access to internet based information in communities with network scarcity or economic problems from the Global South. InfoInternet develops both networking solutions as well as business and social models, together with actors like mobile operators and government organisations. In this paper we identify and describe characteristics of educational applications, their specific users, and learning environment. We are interested in applications that make the adoption of Internet faster, cheaper, and wider in such communities. When developing new applications (or adopting existing ones) for such constrained environments, this work acts as initial guidelines prior to field studies.Comment: 16 pages, 1 figure, under review for a journal since March 201

Tangible Multimedia: A Case Study for Bringing Tangibility into Multimedia Learning

AbstractMultimedia augmented with tangible objects is an area that has not been explored. Current multimedia systems lack the natural elements that allow young children to learn tangibly and intuitively. In view of this, we propose a research to merge tangible objects with multimedia for preschoolers, and propose to term it as “tangible multimedia”. To evaluate the feasibility of such multimedia prior to actual research, a prototype named TangiLearn has been developed for a case study. This paper is the report of what we discovered during the study. The study concluded that TangiLearn enhanced the preschoolers’ enjoyment and learning performance

Nostalgia videogames as playable game criticism

The aim of this paper is to consider the emergence of nostalgia videogames in the context of playable game criticism. Mirroring the development of the nostalgia film in cinema, an increasing number of developers are creating videogames that are evocative of past gaming forms, designs, and styles. The primary focus of this paper is to explore the extent to which these nostalgia videogames could be considered games-on-games: games that offer a critical view on game design and development, framed by the nostalgia and cultural memory of both gamers and game developers. Theories of pastiche and parody as applied to literature, film, and art are used to form a basis for the examination of recent nostalgia videogames, all of which demonstrate a degree of reflection on the videogame medium

PokéStops and Other Spaces of Our Lives: Co-Creation of Social Spatiality Through Consumption of Pokémon GO

The tremendous Pokémon GO craze got the world by surprise in July 2016. Within a couple of days this location-based mobile game gathered a record-breaking number of downloads and transformed the urban environment into a game board by driving people outside their homes to catch imaginary creatures. Combining the physical reality and imagination, this ‘pervasive’ game can be thought of as a new instance of digital virtual consumption (DVC) that could have an enhanced potential to simulate consumer fantasies. While there is a decade of research on how the urban environment can be experienced differently through location-aware technologies, most of it is outdated. Furthermore, little attention has been paid to the full spectrum of social spaces following Lefebvre’s (1991) triad that extends beyond behavior in the observable environment. The aim of this research is to help fill those gaps in understanding how consumers participate in the co-creation of social spaces through the next-generation location-based mobile games as a part of their daily lives. Moreover, the goal is to examine how the interface between the ‘real’ and the ‘imaginary’ is being experienced, and what kind of a role gamification plays in the equation. The research problem is approached through qualitative research methods leaning on the philosophical assumptions of interpretivism and phenomenology, that view the world as socially constructed and only understood through one’s ‘being’ in it. The research lies within the field of Consumer Culture Theory (CCT) that explores the dynamic interplay of consumer behavior, environment and culture through contextual, symbolic and experiential aspects of consumption. Consequently, the aim is to describe the phenomenon rather than to find an explanation for it. The research identifies three main themes that together describe the experience of co-creating social spaces through game play. These are labelled as I) trusting the community to fill in the blanks, II) negotiation between spaces of play and duty, and III) making the most of both worlds. Together they draw a picture of a constant struggle between the player community and the game over agency in creating the experience; transformation of spaces and places through the practice of ‘mapping’; un-controlled switches between spaces of nostalgia and responsibility due to loss of anonymity in the physical reality; and developing alternative game interfaces to optimally integrate the game into players’ everyday spaces. Furthermore, the findings imply an emergence of a new form of gamification, where the core ‘service’ being gamified is, in fact, an individual life as a whole

Family memories in the home: contrasting physical and digital mementos

We carried out fieldwork to characterise and compare physical and digital mementos in the home. Physical mementos are highly valued, heterogeneous and support different types of recollection. Contrary to expectations, we found physical mementos are not purely representational, and can involve appropriating common objects and more idiosyncratic forms. In contrast, digital mementos were initially perceived as less valuable, although participants later reconsidered this. Digital mementos were somewhat limited in function and expression, largely involving representational photos and videos, and infrequently accessed. We explain these digital limitations and conclude with design guidelines for digital mementos, including better techniques for accessing and integrating these into everyday life, allowing them to acquire the symbolic associations and lasting value that characterise their physical counterparts

Towards a framework for investigating tangible environments for learning

External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning

The internet of toys: a posthuman and multimodal analysis of connected play

Background: The study that is reported in this paper focuses on an exploration of the role and nature of play in young children’s use of toys that connect physical and digital domains. Purpose: The purpose of the paper is to explore the nature of the connections that are made in play that transverses physical and virtual domains. The paper draws on post-human theory in order to explain some of the complexity of the play that occurs in these contexts. Research Design: The research took place in the UK and the overall study consisted of four distinct stages: (i) A survey of 2000 parents of children aged 0-5, focusing on children’s access to and use of tablet apps; (ii) Case studies of pre-school children’s use of apps in six families (iii) Observations of children aged 3-5 in a school using apps (iv) Content and multimodal analysis of apps. The focus of this paper is on (ii), although some of the survey data from the first stage of the study are also shared in order to provide context. Data Collection and Analysis: The focus for this paper is the play of a threeyear old girl, Amy. In addition to ethnographic data constructed over a 2- month period (field notes, interviews, photographs and films), Amy’s mother collected data between the researchers’ visit by making films of her daughter’s use of apps. Amy also collected data herself by wearing a GoPro Chestcam. The data that informs the analysis in this paper is a film created by Amy (11.05 minutes) and a video filmed by Amy’s mother (5.21 minutes). Data were both inductively analyzed using multimodal (inter)action analysis and deductively analysed, utilizing a posthumanist approach. Findings: Amy’s play connected digital and non-digital components in complex ways. An app and related physical object that typify the Internet of Toys provided opportunities for Amy’s play to take place across physical and digital domains, and the inorganic objects embedded in the electronic toy and related app were an important element of this play, shaping Amy’s responses at times. However, Amy’s play was not always determined by the design of the electronic objects and she demonstrated agency within play episodes. There were multiple connections made across a variety of domains/ dimensions, which added to the complexity of the play. Conclusions/Recommendations: Young children’s play increasingly connects digital and non-digital domains and post-humanist theories can enhance understanding of how connections across these time/spaces are made

Avatar actors

In this text I wish to discuss, as well as illustrate through pictorial examples, how the Live Visuals of three dimensional online virtual worlds may be leading us into participatory and collaborative Play states during which we appear to become the creators as well as the actors of what may also be described as our own real-time cinematic output. One of the most compelling of these stages may be three dimensional, online virtual worlds in which avatars create and enact their own tales and conceptions, effectively bringing forth live, participatory cinema through Play