SyMAPse: Augmented Interactive Maps for Subjective Expression

International audienceInteractive geographic maps are widely available nowadays. Yet, these maps remain mostly limited to standard interaction contexts and only allow a small panel of expression means to novice users. For example, most available interactive maps only enable users to explore content but not to contribute to the content. On the other hand, the " Smart city " process enlightens the importance of providing ways of expression to the citizens as they are directly affected by urban issues. We introduce a spatial augmented reality map, in which a virtual map is projected on a physical piece of paper. This system, developed for a museum, has two main uses for the users: expressing themselves about their city, and exploring the multiple visions of that city. In a preliminary study we compared interaction techniques for this system based on multi-touch, tangible and spatial modalities for three common map functions: zooming, repositioning the map excerpt, and changing the basemap. Based on the results from this study we designed a new version of this prototype and present future working directions.Les cartes géographiques interactives sont largement disponibles de nos jours. Cependant, ces cartes sont pourtant limitées à des contextes d'interaction standards et n'autorisent qu'un panel limité de moyens d'expression pour les utilisateurs novices. Par exemple, la plupart des cartes interactives disponibles permettent seulement aux utilisateurs d'explorer du contenu mais rarement d'en ajouter. À côté de cela, le processus "Villes Intelligentes" met l'accent sur l'importance de fournir des façons de s'exprimer aux citoyens car ils sont directement concernés par les questions urbaines. Nous introduisons ici un système de réalité augmentée grâce auquel une carte virtuelle est projetée sur une feuille de papier physique. Cet système, développé pour un musée, a deux utilisations pour les utilisateurs: s'exprimer sur leur ville et explorer les multiples visions de cette même ville. Dans une étude préliminaire, nous avons comparé plusieurs techniques d'interactions pour ce système, basées sur des modalités multi-touch, tangibles et spatiales, pour trois fonctionnalités communes pour les cartes : le zoom, le repositionnement du centre de la carte, and le changement de base de carte. En nous appuyant sur cette étude, nous avons développé une nouvelle version de ce prototype et nous présentons plusieurs directions pour de futurs travaux

SyMAPse: Design and Evaluation of an Augmented Reality Map

Interactive geographic maps are today widely available, but remain mostly limited to standard interaction contexts. We introduce SyMAPse [3], a spatial augmented reality map, in which a virtual map is projected onto a physical piece of paper. In a preliminary study we compared interaction techniques based on multi-touch, tangible and spatial modalities for three common map functions: zooming, panning, and changing the basemap. Our results suggest that object-based and spatial interaction may be advantageous over multi-touch in our augmented reality setup. We are currently investigating the rich interaction possibilities provided by this augmented reality setup

Exploring input modalities for interacting with augmented paper maps

International audienceInteractive geographic maps are today widely available, but remain mostly limited to standard interaction contexts. We introduce a spatial augmented reality map, in which a virtual map is projected on a physical piece of paper. In a preliminary study we compared interaction techniques based on multi-touch, tangible and spatial modalities for three common map functions: zooming, panning, and changing the basemap. Our results suggest that object-based and spatial interaction may be advantageous over multi-touch in our augmented reality setup

Exploration input modalities for interacting with augmented paper maps

National audienceInteractive geographic maps are today widely available, but remain mostly limited to standard interaction contexts. We introduce a spatial augmented reality map, in which a virtual map is projected on a physical piece of paper. In a preliminary study we compared interaction techniques based on multi-touch, tangible and spatial modalities for three common map functions: zooming, panning, and changing the basemap. Our results suggest that object-based and spatial interaction may be advantageous over multi-touch in our augmented reality setup

Embodied Interaction in Virtual Reality for Grounding Mathematics

Mathematics is a useful skill to learn, even for students who do not wish to become mathematicians. One of the goals of mathematics is to gain understanding of the patterns of the world, in order to model and predict future outcomes. Recent years shed light on the importance of mathematics, for example to understand and take action during the COVID-19 pandemic or the general climate crisis. However, most students find mathematics useless and disconnected from the real world, and several countries are witnessing a worrisome decline in mathematics ability. This means, we argue, that we should challenge our assumptions about how to teach mathematics, specifically as the way we teach mathematics currently does not enable students to understand it well enough to transfer their skills to other classes, nor outside of the classroom setting. Researchers and educators tackling this issue have argued that, although mathematics is often considered as a Platonic ideal that cannot directly be sensed or manipulated, mathematics rather is a social and malleable process that arises from our sensorimotor experiences of the world. For example, arithmetic can be seen as manipulation of object collections. Similarly, the concept of infinity can be related to our way of expressing something too large for our senses to perceive, while continuity may arise from sequences of elements too small for us to sense. Simply put, mathematics is embodied, and, although not all experts agree on the implications of this theory, there is undeniable evidence that our bodies play an important role in learning mathematics. As a consequence, when designing learning activities, we need to facilitate gesture production, and support sense-making of bodily actions. Meanwhile, Virtual Reality (VR), a technology heavily focused on bodily movement and manipulations, became more affordable and widespread. Using a wireless Head-Mounted Display (HMD), VR can immerse learners' sensory channels into another world, digitally manufactured. Using hand-tracking technologies, VR seems particularly suitable to support embodied learning activities. With VR, we could create the MathLand imagined by Papert, a world of mathematics where learners can explore and manipulate mathematical objects. But is VR truly the solution to support the transition towards embodied learning of mathematics? Although the idea of using VR to implement embodied learning activities seems promising, VR interaction research does not leverage actual theories of embodied learning. Specifically, we identified three main challenges in this field. First, there is a lack of empirical studies evaluating the effect of VR embodied learning activities. Second, only a handful of studies address higher education specifically. Third, there are no design guidelines for embodied learning activities in VR focusing specifically on interaction. Considering these challenges, our work addresses the following research question: How to design embodied interaction to support embodied sense-making of mathematics? We ground our work in three meanings of embodiment: Embodied cognition relates to the role of learners' bodies in cognition, embodied interaction relates to the role of users' bodies in interaction, and avatar embodiment relates to the perception of a digital body as one's own. Within this framework, we address the challenges with four contributions. Specifically, we address three levels of focus: (1) the avatar level, (2) the interaction level, and (3) the context level. Our last contribution focuses on (4) learners and the bodily actions they perform in sense-making activities. (1) Looking at embodied interaction at the level of the avatar, we propose "Digital Gloves", a novel input mechanism that supports embodied interaction and reduces split-attention effect by co-locating input and display on the users' hands. Through two user studies, we demonstrate the potential of our mechanism for more intuitive, enjoyable, and effective gaming and learning experiences. We offer recommendations to best design activities using our mechanism as well as suggestions for future applications. (2) Focusing on the interaction level, we offer an empirical evaluation of the impact of the degree and type of embodiment on usability and learning outcomes. To do so, we designed an embodied activity to learn about derivatives. Our results reveal that although the degree of embodiment only impacts the duration of the activity, the type of embodiment impacts manipulations and learning outcomes. We offer an explanation of these results in terms of mathematical meanings highlighted by different types of embodiment, and conclude with design recommendations for VR embodied learning activities. (3) In our third project, we focus on the context of the interaction. We conceptualize embodiment as a form of concreteness, and demonstrate the grounding affordances of embodied concreteness. To do so, we designed an embodied activity in VR to teach graph theory to bachelor students. Our activity builds on embodied metaphors by representing graphs as water flow systems. Our results show that students using our activity feel that the activity highlights the relevance of the topic best, and also feel better prepared for the subsequent lecture. Moreover, unlike an activity focusing on manipulation only, our activity does not impair transfer abilities. (4) In our last project, we focus on the learner and explore the design space of embodied interaction for sense-making. We explore two contexts. First, we look into an intuition-building activity where learners are directed towards specific bodily actions. Second, we look into an intuition-probing activity where learners spontaneously perform bodily actions. Specifically, we look at the role of individual differences and we aggregate the results from both studies to offer general design recommendations as well as directions for future research on sense-making embodied interaction in VR. Our work shows that, although VR is a powerful tool to ground abstract mathematics, our interaction design decisions impact how people manipulate the virtual elements and the resulting learning outcomes. Moreover, our work opens a novel avenue of research by highlighting the importance of considering in-VR embodied learning assessments

Resources for Co-Designing Games with Children

As part of a co-design project, we designed and assembled a kit of resources for co-designing games with children. This kit is intended for researchers or designers who wish to co-design computer games with children. Moreover, our kit is specifically focused on raising interest and understanding of game design among the participants, and has been tested with girls aged 11-12. In this document, we describe each of the resources, their goal, and how to use them. We anchor our resources in game design process, with an emphasis on learning

Three Perspectives on Embodied Learning in Virtual Reality: Opportunities for Interaction Design

With the fast evolution of Virtual Reality (VR) technology, new prospects opened for embodied learning. Learners can now manipulate digital representations of abstract concepts and make sense of them through sensorimotor stimulation. However, in research, embodiment is explored from several perspectives, which, we argue, should be considered within a same framework. In this paper, we describe three major perspectives relevant for embodied learning in VR: embodied cognition, embodied interaction, and avatar embodiment. We organize these perspectives within one common interdisciplinary framework, and discuss resulting design opportunities for VR embodied learning interactions. Specifically, we show that embodied interaction does not necessarily support embodied cognition, and that breaking recommendations of avatar embodiment can actually support meaning-making. We believe our work offers novel avenues for future research and will foster interesting conversations in the HCI community

Grounding Graph Theory in Embodied Concreteness with Virtual Reality

Abstract mathematics can be difficult to grasp, in part because it relies on symbols and formalisms that are powerful yet meaningless to novices unless grounded in concreteness. Although a wide corpus of research focuses on concreteness in mathematics education, the notion of concreteness can be apprehended in various ways and it is not yet clear which specific aspects of concreteness help the learners. In this paper, we explore embodiment as a form of concreteness to ground abstract mathematics. First, we designed and evaluated an embodied learning activity on graph theory. Through a user study with 89 participants, we then compared three approaches: abstraction, manipulated concreteness, and embodied concreteness. Our results show that, compared to abstraction, both forms of concreteness increase learners’ perceived attention, confidence, and satisfaction. However, only embodied concreteness increases perceived relevance and grounding. Moreover, unlike manipulated concreteness, embodied concreteness does not impair learning outcomes nor transfer abilities

A Creative Game Design and Programming App: The detailed report

We present a game creation app for tablets that builds on the popularity of video games while focusing attention on creativity and problem solving. With our app, users design and build a custom game by first drawing game characters and objects on paper with markers and crayons. The drawings are scanned into the app using the tablet's camera, and the scanned sprites are used to design levels. An event-based visual programming language allows users to program the logic of their game. In the spirit of creative play, users can test their game at any point or jump between the design and programming phases in order to realize their imagination. We evaluate our app with a user study to understand how gender and the use of self-made drawings influence the type of games users create as well as their state of flow during the creation process. Our results show that letting users draw their own game elements can lead to higher engagement. Regarding gender differences, we observe that girls tend to spend more time programming and less time testing compared to boys. We also show that our app can help girls gain self-confidence

A Creative Game Design and Programming App

We present a game creation app for tablets that builds on the popularity of video games while focusing attention on creativity and problem solving. With our app, users design and build a game by first drawing characters and objects on paper with markers and crayons, and then automatically integrate them with our app. An event-based visual programming language allows to program the game logic. In the spirit of creative play, users can jump at any point between the design, programming and test phases in order to realize their imagination. We evaluate our app with a user study to understand how gender and the use of self-made drawings influence the type of games users create and their state of flow during the process. Our results show that letting users draw their own game elements can lead to higher engagement. We also show that girls tend to spend more time programming and less time testing compared to boys, and that our app can help girls gain self-confidence