GAINE - tanGible Augmented INteraction for Edutainment

Interactive tabletops are gaining an increasing interest since they provide a more natural interaction with digital contents and allow the interaction of multiple users at a time promoting face-to-face collaboration, information sharing and the raise of social experiences. Given the potentialities offered by these devices, several entertainment-edutainment applications based on interactive tabletops have been successfully developed in different areas, from medical therapy support to children’s collaborative learning, interactive storytelling and cultural heritage. However, the development of such applications often requires complex technical and implementation skills. Taking this into consideration, in this paper we present GAINE (tanGible Augmented INteraction for Edutainment), a software framework aimed at the rapid prototyping and development of interactive tabletop games. GAINE offers developers context specific high-level constructs and a simple scripting language that simplifies the implementation task. The framework is portable on different operating systems and offers independence from the underlying hardware. Two practical case studies are thoroughly discussed to show how GAINE can simplify the development of interactive tabletop applications in the entertainment and edutainment contexts

GAINE - A Portable Framework for the Development of Edutainment Applications Based on Multitouch and Tangible Interaction

In the last few years, Multitouch and Tangible User Interfaces have emerged as a powerful tool to integrate interactive surfaces and responsive spaces that embody digital information. Besides providing a natural interaction with digital contents, they allow the interaction of multiple users at the same time, thus promoting collaborative activities and information sharing. In particular, these characteristics have opened new exploration possibilities in the edutainment context, as witnessed by the many applications successfully developed in different areas, from children’s collaborative learning to interactive storytelling, cultural heritage and medical therapy support. However, due to the availability of different multitouch and tangible interaction technologies and of different target computing platforms, the development and deployment of such applications can be challenging. To this end, in this paper we present GAINE (tanGible Augmented INteraction for Edutainment), a software framework that enables rapid prototyping and development of tangible augmented applications for edutainment purposes. GAINE has two main features. First, it offers developers high-level context specific constructs that significantly reduces the implementation burden. Second, the framework is portable on different operating systems and offers independence from the underlying hardware and tracking technology. In this paper, we also discuss several case studies to show the effectiveness of GAINE in simplifying the development of entertainment and edutainment applications based on multitouch and tangible interaction

Virtual Heritage: new technologies for edutainment

Cultural heritage represents an enormous amount of information and knowledge. Accessing this treasure chest allows not only to discover the legacy of physical and intangible attributes of the past but also to provide a better understanding of the present. Museums and cultural institutions have to face the problem of providing access to and communicating these cultural contents to a wide and assorted audience, meeting the expectations and interests of the reference end-users and relying on the most appropriate tools available. Given the large amount of existing tangible and intangible heritage, artistic, historical and cultural contents, what can be done to preserve and properly disseminate their heritage significance? How can these items be disseminated in the proper way to the public, taking into account their enormous heterogeneity? Answering this question requires to deal as well with another aspect of the problem: the evolution of culture, literacy and society during the last decades of 20th century. To reflect such transformations, this period witnessed a shift in the museum’s focus from the aesthetic value of museum artifacts to the historical and artistic information they encompass, and a change into the museums’ role from a mere "container" of cultural objects to a "narrative space" able to explain, describe, and revive the historical material in order to attract and entertain visitors. These developments require creating novel exhibits, able to tell stories about the objects and enabling visitors to construct semantic meanings around them. The objective that museums presently pursue is reflected by the concept of Edutainment, Education + Entertainment. Nowadays, visitors are not satisfied with ‘learning something’, but would rather engage in an ‘experience of learning’, or ‘learning for fun’, being active actors and players in their own cultural experience. As a result, institutions are faced with several new problems, like the need to communicate with people from different age groups and different cultural backgrounds, the change in people attitude due to the massive and unexpected diffusion of technology into everyday life, the need to design the visit by a personal point of view, leading to a high level of customization that allows visitors to shape their path according to their characteristics and interests. In order to cope with these issues, I investigated several approaches. In particular, I focused on Virtual Learning Environments (VLE): real-time interactive virtual environments where visitors can experience a journey through time and space, being immersed into the original historical, cultural and artistic context of the work of arts on display. VLE can strongly help archivists and exhibit designers, allowing to create new interesting and captivating ways to present cultural materials. In this dissertation I will tackle many of the different dimensions related to the creation of a cultural virtual experience. During my research project, the entire pipeline involved into the development and deployment of VLE has been investigated. The approach followed was to analyze in details the main sub-problems to face, in order to better focus on specific issues. Therefore, I first analyzed different approaches to an effective recreation of the historical and cultural context of heritage contents, which is ultimately aimed at an effective transfer of knowledge to the end-users. In particular, I identified the enhancement of the users’ sense of presence in VLE as one of the main tools to reach this objective. Presence is generally expressed as the perception of 'being there', i.e. the subjective belief of users that they are in a certain place, even if they know that the experience is mediated by the computer. Presence is related to the number of senses involved by the VLE and to the quality of the sensorial stimuli. But in a cultural scenario, this is not sufficient as the cultural presence plays a relevant role. Cultural presence is not just a feeling of 'being there' but of being - not only physically, but also socially, culturally - 'there and then'. In other words, the VLE must be able to transfer not only the appearance, but also all the significance and characteristics of the context that makes it a place and both the environment and the context become tools capable of transferring the cultural significance of a historic place. The attention that users pay to the mediated environment is another aspect that contributes to presence. Attention is related to users’ focalization and concentration and to their interests. Thus, in order to improve the involvement and capture the attention of users, I investigated in my work the adoption of narratives and storytelling experiences, which can help people making sense of history and culture, and of gamification approaches, which explore the use of game thinking and game mechanics in cultural contexts, thus engaging users while disseminating cultural contents and, why not?, letting them have fun during this process. Another dimension related to the effectiveness of any VLE is also the quality of the user experience (UX). User interaction, with both the virtual environment and its digital contents, is one of the main elements affecting UX. With respect to this I focused on one of the most recent and promising approaches: the natural interaction, which is based on the idea that persons need to interact with technology in the same way they are used to interact with the real world in everyday life. Then, I focused on the problem of presenting, displaying and communicating contents. VLE represent an ideal presentation layer, being multiplatform hypermedia applications where users are free to interact with the virtual reconstructions by choosing their own visiting path. Cultural items, embedded into the environment, can be accessed by users according to their own curiosity and interests, with the support of narrative structures, which can guide them through the exploration of the virtual spaces, and conceptual maps, which help building meaningful connections between cultural items. Thus, VLE environments can even be seen as visual interfaces to DBs of cultural contents. Users can navigate the VE as if they were browsing the DB contents, exploiting both text-based queries and visual-based queries, provided by the re-contextualization of the objects into their original spaces, whose virtual exploration can provide new insights on specific elements and improve the awareness of relationships between objects in the database. Finally, I have explored the mobile dimension, which became absolutely relevant in the last period. Nowadays, off-the-shelf consumer devices as smartphones and tablets guarantees amazing computing capabilities, support for rich multimedia contents, geo-localization and high network bandwidth. Thus, mobile devices can support users in mobility and detect the user context, thus allowing to develop a plethora of location-based services, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits and cultural or tourist sites according to visitors’ personal interest and curiosity

Investigation and development of a tangible technology framework for highly complex and abstract concepts

The ubiquitous integration of computer-supported learning tools within the educational domain has led educators to continuously seek effective technological platforms for teaching and learning. Overcoming the inherent limitations of traditional educational approaches, interactive and tangible computing platforms have consequently garnered increased interest in the pursuit of embedding active learning pedagogies within curricula. However, whilst Tangible User Interface (TUI) systems have been successfully developed to edutain children in various research contexts, TUI architectures have seen limited deployment towards more advanced educational pursuits. Thus, in contrast to current domain research, this study investigates the effectiveness and suitability of adopting TUI systems for enhancing the learning experience of abstract and complex computational science and technology-based concepts within higher educational institutions (HEI)s. Based on the proposal of a contextually apt TUI architecture, the research describes the design and development of eight distinct TUI frameworks embodying innovate interactive paradigms through tabletop peripherals, graphical design factors, and active tangible manipulatives. These computationally coupled design elements are evaluated through summative and formative experimental methodologies for their ability to aid in the effective teaching and learning of diverse threshold concepts experienced in computational science. In addition, through the design and adoption of a technology acceptance model for educational technology (TAM4Edu), the suitability of TUI frameworks in HEI education is empirically evaluated across a myriad of determinants for modelling students’ behavioural intention. In light of the statistically significant results obtained in both academic knowledge gain (μ = 25.8%) and student satisfaction (μ = 12.7%), the study outlines the affordances provided through TUI design for various constituents of active learning theories and modalities. Thus, based on an empirical and pedagogical analyses, a set of design guidelines is defined within this research to direct the effective development of TUI design elements for teaching and learning abstract threshold concepts in HEI adaptations

NIDABA : plataforma digital para produção de recursos educacionais inclusivos baseados em mesa tangível

A utilização de ferramentas tecnológicas na educação é bastante disseminada no Brasil e no mundo, porém, quando envolve educação inclusiva, especialmente com crianças com deficiência intelectual ou autismo, existem poucas soluções e recursos tecnológicos disponíveis. O uso de interfaces tangíveis para fins educacionais apresenta consideráveis vantagens e pode proporcionar um incremento no engajamento e motivação dos estudantes e se revelando um recurso de tecnologia assistiva adequado para a educação. Um dos principais problemas encontrados para a utilização de mesas tangíveis no ambiente educacional é a ausência de softwares para a sua programação e uso que atendam às necessidades educacionais e que sejam fáceis de utilizar, especialmente por educadores que não dominam a lógica de programação. A presente tese tem por objetivo criar uma plataforma digital para produção de recursos educacionais baseados em mesa tangível, que atendam à diversidade para a educação inclusiva. Para isso, um levantamento de necessidades pedagógicas e as questões de acessibilidade para a interação tangível no contexto inclusivo são analisadas e é desenvolvido um editor intuitivo e de fácil utilização para criação de recursos educacionais, juntamente com um player para executar as aplicações em mesas tangíveis e que permite a simulação em computadores, tablets e smartphones. Uma avaliação da usabilidade do editor e do player com as aplicações para mesa tangível e a viabilidade e utilidade da proposta é realizada e é apresentada uma especificação para a construção e montagem de uma mesa de baixo custo. Assim, este estudo apresenta uma proposta de uso da mesa tangível na educação inclusiva, juntamente com uma ferramenta de autoria para a construção de aplicativos e a plataforma necessária para sua execução, desenvolvidos especificamente para este propósito. Esta plataforma pode ser utilizada como um instrumento de mediação tecnológica, de forma interativa e intuitiva por educadores e estudantes, para o desenvolvimento e utilização de atividades pedagógicas num contexto inclusivo. A plataforma desenvolvida e o editor reúnem, numa única ferramenta, diversas características importantes apresentadas em trabalhos relacionados e apresenta, ainda, vários recursos inovadores para a produção de aplicações para mesa tangível.Use of technological tools in education is widespread in Brazil and in the world, however when it involves inclusive education, especially with children with intellectual disabilities or autism, there are few technological solutions and resources available. Tangible interfaces use for educational purposes has considerable advantages and can provide an increase in student engagement and motivation, and have proved to be suitable for assistive technology resources for education. One of the main problems found for the use of tangible tabletops in the educational environment is the absence of software for their programming and use that meet educational needs and that are easy to use, especially by teachers who do not master the programming logic. This thesis aims to create a digital platform for the production of educational resources based on a tangible tabletop, to contemplate diversity for inclusive education. For this, a survey of pedagogical needs and accessibility issues for tangible interaction in the inclusive context are analyzed and an intuitive and user-friendly editor is developed for creating educational resources, along with a player to run the applications on tangible tabletop and that allows simulation on computers, tablets and smartphones. An evaluation of the editor and player usability with the tangible tabletop applications and the feasibility and usefulness of the proposal is carried out and is presented a specification for the construction and assembly of a low-cost table. Thus, this study presents a proposal for using the tangible table in inclusive education, together with an authoring tool for building applications and the platform needed for its execution, developed specifically for this purpose. This platform can be used as an instrument of technological mediation, in an interactive and intuitive way by teachers and students, for the development and use of pedagogical activities in an inclusive context. The developed platform and the editor bring together in a single tool, several important features presented in related works and also presents several innovative resources for the production of applications to a tangible tabletop

Putting manipulation, grasping and sensing in educational APPs for children

La tesi ha come obiettivo di valorizzare pratiche psico-pedagogiche classiche utilizzando le nuove opportunità fornite dalla tecnologia e dagli strumenti digitali sempre più ubiquitari nella nostra società. La modalità individuata per connettere l’ambito digitale e quello fisico è attraverso lo sviluppo della multisensorialità e dunque la riscoperta dei sensi come l’olfatto, il gusto e il tatto nelle pratiche educative per bambini dai 6 agli 8 anni (primi tre anni della scuola primaria). Per ottenere questo obiettivo si sono sviluppate metodologie e prototipi (sia hardware, che software – APP) fruibili dai bambini della scuola primaria. In primo luogo è stato ideato, concettualizzato e sviluppato il modello dell’Hyper Activity Book (HAB), che rappresenta la porzione modellistica fondamentale a cui ispirarsi per l’implementazione di prototipi in un settore così delicato quanto cruciale, dove troppo spesso le applicazioni si sono fermate a miglioramenti puntuali e non sistemici. Il modello dell’Hyper Activity Book ha l’ambizione di formare uno schema entro cui poter applicare in maniera organica i principi del Learning by Doing in stretta comunicazione con il concetto degli ambienti didattici ibridi e delle interfacce-utente di stampo tangibile e dei moduli di Intelligenza Artificiale. A corredo del modello dell’HAB e sulla base dell’analisi dello stato dell’arte condotto, sono state prodotte delle linee guida per la progettazione di Ambienti Ibridi. La ricerca ha permesso lo sviluppo di due hardware, che hanno rappresentato un’evoluzione e un miglioramento incrementale rispetto alle precedenti realizzazioni. Si è sviluppato un algoritmo per il riconoscimento di oggetti fisici su schermi touchscreen grazie al loro posizionamento sugli stessi. Il prototipo, denominato TriPOD, contiene l’algoritmo che è in grado di riconoscere gli oggetti fisici sulla base della disposizione delle protrusioni capacitive (i pin capacitivi) in maniera univoca, collegando l’identificativo generato con un oggetto tangibile. Questo prototipo è funzionante e sebbene abbia necessità di divenire ancora più robusto in termini tecnici, è uno sviluppo interessante per le applicazioni nel settore dell’educazione e della riabilitazione, in grado di aprire nuovi scenari applicativi. L’altro prototipo hardware, sviluppato nel percorso dottorale è rappresentato dall’Activity Board 1.0. La tavoletta attiva che sfrutta la tecnologia RFID è il miglioramento di precedenti versioni, attraverso un complesso e lungo lavoro di riaggiornamenti e miglioramenti incrementali. La nuova versione ha la possibilità di connettersi via Wi-Fi e contiene tutti i materiali al suo interno, fornendo una versione “portatile” e immediatamente fruibile in ogni contesto. La tavoletta ha raggiunto un ottimo grado di robustezza, tanto da permettere sperimentazioni senza bug di sistema e rimuovendo la maggior parte di problematiche tecniche. Un risultato è stato anche quello di ridurre drasticamente il costo del sistema, attraverso un’attenta analisi delle componentistiche sul mercato, con l’obiettivo di mantenere gli standard progettuali definiti dal modello HAB e i requisiti funzionali. Ulteriore risultato del progetto è stato lo sviluppo di APP educative che sfruttano i principi della multisensorialità. Con queste APP educative si intende estendere il campo rispetto alla manualità che è stata centrale nei progetti precedenti (vedi BlockMagic). L’estensione nei confronti dell’udito, del tatto, del gusto e dell’olfatto hanno aperto nuovi scenari di opportunità per tutto il gruppo di ricerca, proponendo delle modalità di interazione ancora più aderenti alle pratiche psico-pedagogiche a cui si ispire. Le APP educative si sono centrate sul metodo ibrido, di coniugazione tra mondo fisico e reale in campo didattico, creando applicazioni per l’educazione musicale, per l’educazione olfattiva e della conoscenza del mondo, per l’immersione nei processi di storytelling, per l’apprendimento dell’anatomia, per l’apprendimento delle lingue, per l’apprendimento dell’ortografia e per l’apprendimento dei rudimenti della matematica. In questa tesi sono stati presentati solo alcune delle APP educative sviluppate in questi tre anni. A valle del processo è stata svolta una sperimentazione per verificare l’efficacia e l’accettabilità degli strumenti nei confronti di due modelli già utilizzati nelle classi e negli ambiti informati ossia lo strumento cartaceo e il tablet. Per quanto riguarda l’accettabilità, gli strumenti multisensoriali sono emersi come i preferiti per il gruppo di partecipanti (121 bambini appartenenti a due scuole). Per quanto riguarda l’efficacia, si è effettuato uno studio sulla memorizzazione di storie. I dati mostrano come le storie eseguite con il libro e con i materiali multisensoriali permettono un buon grado di memorizzazione mentre con il tablet il ricordo degli avvenimenti scende radicalmente

´Now I Get It!´ : Developing a Real-World Design Solution for Understanding Equation-Solving Concepts

Strong conceptual understanding contributes to mathematics learning. Manipulatives (i.e., hands-on learning tools that allow for mathematical concept exploration through different senses) can facilitate students' understanding of mathematical concepts when used meaningfully. However, a body of research has demonstrated that although teachers have considered manipulatives pedagogically beneficial, when it comes to everyday classroom practice, they often prefer traditional teacher-centred and paper-and-pencil instruction. This doctoral research aims to develop a manipulative and its appropriate use to promote not only students' understanding of mathematical concepts, but also classroom adoption. Solving one-variable linear equations in primary school classrooms was used as a case study. An educational design research (EDR) approach was used throughout three phases of a 6-year enquiry: initial research, concept development, and design development. Phase 1 (initial research) was undertaken to gain a theoretical and contextual understanding and investigate existing manipulatives. In Phase 2 (concept development), four manipulative concepts were generated based on the Phase 1 findings. Each concept was then evaluated in terms of its pedagogical benefits and compatibility with school and classroom practice. During Phase 3 (design development), informed by the Phase 2 findings, a design solution (i.e., a tangible manipulative allowing physical input and providing digital output, student worksheets, teacher guides, and class activities) was developed. The developed design solution was then implemented and evaluated in classrooms. Empirical research was conducted in Finnish comprehensive schools. Altogether, 18 teachers, 98 primary school students, and 65 lower secondary school students took part in different phases of the research. The data were collected using mixed methods, including class interventions, paper-based tests, thinking aloud, questionnaires, and interviews. Qualitative and quantitative data collected from various methods and data sources were simultaneously analysed and then compared and combined to holistically understand the research results. Together, multiple iterations (of investigation, design, and assessment) resulted in practical and theoretical outcomes. The research-based design solution, which promotes students' understanding of equation-solving concepts and classroom practice, is the practical outcome of this research to directly improve educational practice. Additionally, the research contributes to three types of theories: domain theories, design frameworks, and design methodologies. The first theoretical outcome is a domain theory yielding two types of knowledge, that is, context and outcomes knowledge. The context knowledge describes the challenges and opportunities of using manipulatives in mathematics classrooms, as well as strengths and limitations of existing manipulatives. The outcomes knowledge describes outcomes of implementing the design solution: the developed tangible manipulative accompanied by the instructional materials enhanced students' understanding of equation-solving concepts through discovery learning, social interaction, and multimodal expression of mathematical thinking; the manipulative is likely to be adopted in the classroom because of its pedagogical benefits and compatibility with school and classroom practice. The second theoretical outcome is a design Jramework for real-world educational technologies. Content, pedagogy, practice, and technology should be taken into consideration when designing real- world educational technologies to ensure their educational benefits, utilisation, adoption, and feasibility. The third theoretical outcome is a design methodo/ogy built on firsthand experience from undertaking this EDR. The guidelines for conducting EDR guides how to embrace opportunities and overcome challenges that may emerge. This research contributes to a link between research and practice in mathematics education. It provides researchers with knowledge of how multimodal interaction with manipulatives enhances mathematics learning and guidelines for conducting EDR. It guides educational designers to take various aspects into consideration when designing educational technologies to improve real-world practice. Moreover, this research also has practical implications. First, it encourages teacher educators to prepare pre- and in-service teachers for successful incorporation of manipulatives in their mathematics classrooms. Second, it guides practitioners on how to support their students to benefit from manipulatives. Third, it urges schools to support the acquisition and utilisation of manipulatives. Finally, it calls on school curricula to encourage the use of manipulatives in the mathematics classroom to promote students' conceptual understanding

GAINE - tanGible Augmented INteraction for Edutainment

Interactive tabletops are gaining an increasing interest since they provide a more natural interaction with digital contents and allow the interaction of multiple users at a time promoting face-to-face collaboration, information sharing and the raise of social experiences. Given the potentialities offered by these devices, several entertainment-edutainment applications based on interactive tabletops have been successfully developed in different areas, from medical therapy support to children's collaborative learning, interactive storytelling and cultural heritage. However, the development of such applications often requires complex technical and implementation skills. Taking this into consideration, in this paper we present GAINE (tanGible Augmented INteraction for Edutainment), a software framework aimed at the rapid prototyping and development of interactive tabletop games. GAINE offers developers context specific high-level constructs and a simple scripting language that simplifies the implementation task. The framework is portable on different operating systems and offers independence from the underlying hardware. Two practical case studies are thoroughly discussed to show how GAINE can simplify the development of interactive tabletop applications in the entertainment and edutainment context