Narrating by doing: a bridging concept for understanding and informing the design of tangible interfaces for storytelling

We present and discuss the concept ‘narrating by doing’ as the process of creating narratives by performing different embodied actions with tangible interfaces for storytelling. We characterize it as a ‘bridging concept’ that can facilitate exchange between theory and design, informing research and design of TUIs for storytelling targeting young children. The concept builds on theories of cognition, learning and narration, specifically drawing upon the following perspectives: Constructionism, Socio-Constructivism, Embodied Cognition, Narratology and The Narrative Practice Hypothesis. Building upon these theoretical foundations, we identify and discuss four ‘design articulations’ (i.e. important parameters that express the qualities of the concept), namely communicative situation, narrative function of the tangible objects, collaborative and embodied actions and the narrator’s position. Finally, we add evidence to the concept and discuss its productiveness by presenting a set of considerations to inform the design of tangible interfaces for storytelling.This work has been financed by national funds through the Portuguese Foundation for Science and Technology (FCT) and by the European Regional Development Fund through the Competitiveness and Internationalization Operational Program under the reference POCI/01/0145/FEDER/032580. This work is funded by CIEd—Research Centre on Education, Institute of Education, University of Minho, projects UIDB/01661/2020 and UIDP/01661/2020, through national funds of FCT/MCTES-PT

Towards a framework for investigating tangible environments for learning

External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning

Embodiment and designing learning environments

There is increasing recognition amongst learning sciences researchers of the critical role that the body plays in thinking and reasoning across contexts and across disciplines. This workshop brings ideas of embodied learning and embodied cognition to the design of instructional environments that engage learners in new ways of moving within, and acting upon, the physical world. Using data and artifacts from participants' research and designs as a starting point, this workshop focuses on strategies for how to effectively leverage embodiment in learning activities in both technology and non-technology environments. Methodologies for studying/assessing the body's role in learning are also addressed

Issues and techniques for collaborative music making on multi-touch surfaces

A range of systems exist for collaborative music making on multi-touch surfaces. Some of them have been highly successful, but currently there is no systematic way of designing them, to maximise collaboration for a particular user group. We are particularly interested in systems that will engage novices and experts. We designed a simple application in an initial attempt to clearly analyse some of the issues. Our application allows groups of users to express themselves in collaborative music making using pre-composed materials. User studies were video recorded and analysed using two techniques derived from Grounded Theory and Content Analysis. A questionnaire was also conducted and evaluated. Findings suggest that the application affords engaging interaction. Enhancements for collaborative music making on multi-touch surfaces are discussed. Finally, future work on the prototype is proposed to maximise engagement

Developing a Pedagogical Framework for Designing a Multisensory Serious Gaming Environment

The importance of multisensory interaction for learning has increased with improved understanding of children’s sensory development, and a flourishing interest in embodied cognition. The potential to foster new forms of multisensory interaction through various sensor, mobile and haptic technologies is promising in providing new ways for young children to engage with key mathematical concepts. However, designing effective learning environments for real world classrooms is challenging, and requires a pedagogically, rather than technologically, driven approach to design. This paper describes initial work underpinning the development of a pedagogical framework, intended to inform the design of a multisensory serious gaming environment. It identifies the theoretical basis of the framework, illustrates how this informs teaching strategies, and outlines key technology research driven perspectives and considerations important for informing design. An initial table mapping mathematical concepts to design, a framework of considerations for design, and a process model of how the framework will continue to be developed across the design process are provided

Emotional Biosensing: Exploring Critical Alternatives

Emotional biosensing is rising in daily life: Data and categories claim to know how people feel and suggest what they should do about it, while CSCW explores new biosensing possibilities. Prevalent approaches to emotional biosensing are too limited, focusing on the individual, optimization, and normative categorization. Conceptual shifts can help explore alternatives: toward materiality, from representation toward performativity, inter-action to intra-action, shifting biopolitics, and shifting affect/desire. We contribute (1) synthesizing wide-ranging conceptual lenses, providing analysis connecting them to emotional biosensing design, (2) analyzing selected design exemplars to apply these lenses to design research, and (3) offering our own recommendations for designers and design researchers. In particular we suggest humility in knowledge claims with emotional biosensing, prioritizing care and affirmation over self- improvement, and exploring alternative desires. We call for critically questioning and generatively re- imagining the role of data in configuring sensing, feeling, ‘the good life,’ and everyday experience

Transfer Scenarios: Grounding Innovation with Marginal Practices

Transfer scenarios is a method developed to support the design of innovative interactive technology. Such a method should help the designer to come up with inventive ideas, and at the same time provide grounding in real human needs. In transfer scenarios, we use marginal practices to encourage a changed mindset throughout the design process. A marginal practice consists of individuals who share an activity that they find meaningful. We regard these individuals not as end-users, but as valuable input in the design process. We applied this method when designing novel applications for autonomous embodied agents, e.g. robots. Owners of unusual pets, such as snakes and spiders, were interviewed - not with the intention to design robot pets, but to determine underlying needs and interests of their practice. The results were then used to design a set of applications for more general users, including a dynamic living-room wall and a set of communicating hobby robots

ICS Materials. Towards a re-Interpretation of material qualities through interactive, connected, and smart materials.

The domain of materials for design is changing under the influence of an increased technological advancement, miniaturization and democratization. Materials are becoming connected, augmented, computational, interactive, active, responsive, and dynamic. These are ICS Materials, an acronym that stands for Interactive, Connected and Smart. While labs around the world are experimenting with these new materials, there is the need to reflect on their potentials and impact on design. This paper is a first step in this direction: to interpret and describe the qualities of ICS materials, considering their experiential pattern, their expressive sensorial dimension, and their aesthetic of interaction. Through case studies, we analyse and classify these emerging ICS Materials and identified common characteristics, and challenges, e.g. the ability to change over time or their programmability by the designers and users. On that basis, we argue there is the need to reframe and redesign existing models to describe ICS materials, making their qualities emerge