The effect of representation location on interaction in a tangible learning environment

Drawing on the 'representation' TUI framework [21], this paper reports a study that investigated the concept of 'representation location' and its effect on interaction and learning. A reacTIVision-based tangible interface was designed and developed to support children learning about the behaviour of light. Children aged eleven years worked with the environment in groups of three. Findings suggest that different representation locations lend themselves to different levels of abstraction and engender different forms and levels of activity, particularly with respect to speed of dynamics and differences in group awareness. Furthermore, the studies illustrated interaction effects according to different physical correspondence metaphors used, particularly with respect to combining familiar physical objects with digital--based table-top representation. The implications of these findings for learning are discussed

Resonating Experiences of Self and Others enabled by a Tangible Somaesthetic Design

Digitalization is penetrating every aspect of everyday life including a human's heart beating, which can easily be sensed by wearable sensors and displayed for others to see, feel, and potentially "bodily resonate" with. Previous work in studying human interactions and interaction designs with physiological data, such as a heart's pulse rate, have argued that feeding it back to the users may, for example support users' mindfulness and self-awareness during various everyday activities and ultimately support their wellbeing. Inspired by Somaesthetics as a discipline, which focuses on an appreciation of the living body's role in all our experiences, we designed and explored mobile tangible heart beat displays, which enable rich forms of bodily experiencing oneself and others in social proximity. In this paper, we first report on the design process of tangible heart displays and then present results of a field study with 30 pairs of participants. Participants were asked to use the tangible heart displays during watching movies together and report their experience in three different heart display conditions (i.e., displaying their own heart beat, their partner's heart beat, and watching a movie without a heart display). We found, for example that participants reported significant effects in experiencing sensory immersion when they felt their own heart beats compared to the condition without any heart beat display, and that feeling their partner's heart beats resulted in significant effects on social experience. We refer to resonance theory to discuss the results, highlighting the potential of how ubiquitous technology could utilize physiological data to provide resonance in a modern society facing social acceleration.Comment: 18 page

Humanizing Scholarship: Going Public Via Multimodality

“Humanizing Scholarship” offers a look into the ways that multimodality can be used to make the scholarly conversations had within the academy more accessible to members of the public. This thesis acknowledges and echoes the responsibility academics have to bridging the gap between their research and the people who so often serve as the basis for the ethnographic work being done in academia. My project does two things: First, it brings together some of the conversations surrounding multimodality and public scholarship. Second, it offers some first-hand models of multimodal compositions—the short films Don Armando and The Adjunct, and the screenplay for Huevos Revueltos— which attempt to translate academic conversations through a narrative lens offered by the mediums of screenplays and short films. It is my hope that these models can help future scholars see the value in multimodality and the critical humanizing factor it can offer their research

Teachers’ perspectives on the intertwining of tangible and digital modes of activity with a drawing robot for geometry

The GeomBot is a drawing robot that combines the well-known strengths and opportunities offered by Scratch with those of Papert’s original robotic drawing-turtle. In this study we look at the GeomBot as a physical programmable artifact around which action research with a group of teachers and a researcher was carried out with the aim of designing, implementing and discussing geometry activities for primary school classes. The aim of this article is to investigate teachers’ positioning and perspectives with respect to the activities and the educational environment emerging around the Geombot in the action research. The action research meetings between September 2018 and June 2019 included nine primary school teachers from seven different Italian schools, who met regularly with the first author. After the design and experimentation sessions, in June 2019, the teachers shared their experiences during a final meeting and group discussion guided by the first author. The data collected from the teachers were analyzed using cultural categories from the Semiotic Systems of Cultural Signification, theorized by the Theory of Objectification, to identify the most significant features defining the teachers’ perspectives and identity. The seven emergent features cover teachers’ positioning with respect to: forms of rationality and language and languages: sensuous cognition and the use of ideal and material semiotic resources; accepted teaching practices, problems and situations; mathematical knowledge; the conception of the student; social interaction and forms of rationality; ethical issues; technology

Interactive Polymedia Pixel and Protocol for Collaborative creative content generation on urban digital media displays

This research is an investigation into a creative and technical 'pixel' element that may facilitate Urban Digital Media, a field that inhabits the intersection between architecture, information and culture in the arena of technology and building. It asks how contemporary requirements of public space in our everyday life, such as adaptability, new modes of communication and transformative environments that offer flexibility for future needs and uses, can be addressed by a new form of public display, assembled through the use of an advanced pixel, described as an interactive Polymedia Pixel with situated media device protocol. The weakness of many current media facades for building-scale interactive installation environments lies in the dearth of quality creative content and unresponsiveness in terms of potential human factors, richness of locative situation and contextual interaction (Sauer, 2004). Media facades have evolved from simple 2D visual displays to 3D voxel arrays for depicting static and moving images with a spatial depth dimension (Haeusler, 2009). As a subsequent step in this development, the research investigates a display that reacts to the need for empathetic and responsive urban digital media; integrates multiple modalities; smart energy-saving; and collaborative community engagement. The Polymedia Pixel, which is presented in its research and development in this paper, contributes to the evolution of building-scale interactive installation environments. The paper firstly discusses the attributes of the Polymedia Pixel in order to address the above mentioned weaknesses of public displays. In responding to these necessities, the prototype of the developed Polymedia Pixel with its technology is outlined. The Polymedia Pixel reserach aims to addres

An Integrating Framework for Mixed Systems

International audienceTechnological advances in hardware manufacturing led to an extended range of possibilities for designing physical-digital objects involved in a mixed system. Mixed systems can take various forms and include augmented reality, augmented virtuality, and tangible systems. In this very dynamic context, it is difficult to compare existing mixed systems and to systematically explore the design space. Addressing this design problem, this chapter presents a unified point of view on mixed systems by focusing on mixed objects involved in interaction, i.e. hybrid physical-digital objects straddling physical and digital worlds. Our integrating framework is made of two complementary facets of a mixed object: we define intrinsic characteristics of an object as well as extrinsic characteristics of an object by considering its role in the interaction. Such characteristics of an object are useful for comparing existing mixed systems at a fine-grain level. The taxonomic power of these characteristics is discussed in the context of existing mixed systems from the literature. Their generative power is illustrated by considering a system, Roam, which we designed and developed

Enfrentando os desafios do design de textos multimodais para promover a pedagogia das multiliteracias

This article aims to add to the understanding of the challenges involved in designing digital texts to promote multiliteracies pedagogy. A multiliteracies approach calls for multimodal meaning-making and cultural diversity to be integrated into new school curriculum content, and accordingly, we analyse an interactive children’s story app, named Mobeybou in Brazil. The re-search question addressed was: what can we learn about the design of multimodal texts aimed at promoting intercultural learning from the design of this story app? The app incorporates tangible and digital storytelling materials to promote intercultural skills among young children, focusing specifically on Brazil. Mobeybou in Brazil was studied to characterize the design of its multimodal representations of meaning, using categories from the grammar of storytelling and multimodal meaning-making, particularly those concerned with representing the experiential diversity and personal positioning of the app users. The findings provide evidence of the complexity involved in designing multimodal texts to meet the challenges of promoting multiliteracies pedagogy, highlighting the urgent need to narrow the interface between research undertaken in education, semiotics, and digital media design. The article concludes by identifying the study’s limitations and some future developments.Este artigo pretende contribuir para a compreensão dos desafios colocados pelo design de textos digitais especificamente destinados a promover a pedagogia das multiliteracias. Esta abordagem pedagógica defende que a compreensão e produção de significados multimodais e a diversidade cultural são conteúdos incontornáveis no novo currículo escolar. Apresentamos o estudo de uma story app multimodal interativa, intitulada Mobeybou no Brasil, desenvolvida para dar resposta à seguinte questão de investigação: o que se pode aprender sobre o design de textos multimodais destinados a promover a aprendizagem intercultural a partir do design desta story app? Na nossa análise, utilizamos categorias da gramática da narrativa e da representação multi-modal incidentes na representação da diversidade experiencial e do posicionamento pessoal dos utilizadores. Os resultados sugerem a complexidade envolvida no design de textos multimodais que respondem aos desafios da pedagogia das multiliteracias, salientando a necessidade urgen-te do estreitamento da colaboração entre a investigação realizada nas áreas da educação, da se-miótica e do design de média digitais. Por fim, identificamos as limitações do estudo, apontando alguns desenvolvimentos futuros.This research is funded by the FCT within the scope of the project PTDC/CEDEDG/ 0736/2021, by CIEd – Research Centre on Education, Institute of Education, University of Minho, projects UIDB/01661/2020 and UIDP/01661/2020, through national funds of FCT/MCTES-PT, and by the LARSyS - FCT Plurianual funding 2020-202