Developing a Pedagogical Framework for Designing a Multisensory Serious Gaming Environment

The importance of multisensory interaction for learning has increased with improved understanding of children’s sensory development, and a flourishing interest in embodied cognition. The potential to foster new forms of multisensory interaction through various sensor, mobile and haptic technologies is promising in providing new ways for young children to engage with key mathematical concepts. However, designing effective learning environments for real world classrooms is challenging, and requires a pedagogically, rather than technologically, driven approach to design. This paper describes initial work underpinning the development of a pedagogical framework, intended to inform the design of a multisensory serious gaming environment. It identifies the theoretical basis of the framework, illustrates how this informs teaching strategies, and outlines key technology research driven perspectives and considerations important for informing design. An initial table mapping mathematical concepts to design, a framework of considerations for design, and a process model of how the framework will continue to be developed across the design process are provided

WeDRAW: using multisensory serious games to explore concepts in primary mathematics

WeDRAW aims to mediate learning of primary school mathematical concepts, such as geometry and arithmetic, through the design, development and evaluation of multisensory serious games, using a combination of sensory interactive technologies. Working closely with schools, using participatory design techniques, the WeDRAW system will be embedded into the school curricula, and configurable by teachers. Besides application to typically developing children, a major goal is to examine this multisensory approach with visually impaired and dyslexic children

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD

Creating Bridges: The Role of Exploratory Design Research in an Intelligent Tutoring System Project

Designers of Intelligent Tutoring Systems (ITS) have long been interested in delivering personalised teaching to individual students, typically by ensuring that the student receives content appropriate to their skills and knowledge. Nonetheless, a more holistic view on what constitutes good teaching practice has challenged whether this approach to user modelling is sufficient. Teaching is not only defined by what is taught, but also by how it is taught. In this paper, we demonstrate that exploratory design research can support this view by generating a more inclusive set of user attributes for purposes of user modelling. Through a case study, we show that design research for user modelling can function as a boundary object serving three important roles, that underpin more specifically the design of user modelling and more broadly ITS design. First, design research can establish common ground by encapsulating domain knowledge in an accessible form. This can support diverse project stakeholders to make decisions on what is to be modelled. Second, design research can reveal a wide range of teaching and learning perspectives that in turn introduce transparency to the decision-making process of user modelling and provoke a sense of criticality and accountability amongst project stakeholders. Third, design research can build new bridges between the design of the technology and the user model that underpins it. To this end, user attributes deemed important, yet too complex or cumbersome to develop, can become design principles in the context of the overall ITS design

Multisensory interactive technologies for primary education: From science to technology

While technology is increasingly used in the classroom, we observe at the same time that making teachers and students accept it is more difficult than expected. In this work, we focus on multisensory technologies and we argue that the intersection between current challenges in pedagogical practices and recent scientific evidence opens novel opportunities for these technologies to bring a significant benefit to the learning process. In our view, multisensory technologies are ideal for effectively supporting an embodied and enactive pedagogical approach exploiting the best-suited sensory modality to teach a concept at school. This represents a great opportunity for designing technologies, which are both grounded on robust scientific evidence and tailored to the actual needs of teachers and students. Based on our experience in technology-enhanced learning projects, we propose six golden rules we deem important for catching this opportunity and fully exploiting it

Educational E-book For Children With and Without Developmental Disorders

In the last decade, Autism Spectrum Disorder (ASD) prevalence rate has signifcantly increased, which consequently led to the expansion of research and expenditure in the feld,predominantly focusing on searching for the cause. In a typical classroom scenario, working with children with ASD very often requires 1:1 teacher to child ratio, which makes it very expensive and difcult to implement. Serious games have been utilised as a medium for teaching various developmental skills, such as social interaction, speech, motor skills development, emotion recognition, and other basic concepts. Designing serious games for ASD population difers from other games and even other serious games signifcantly. It requires a holistic approach with extensive knowledge and expertise from felds other than computer science, such as psychology, sociology and cognitive science. However, once harnessed correctly, such games can be used by children with ASD on their own time, with or without supervision and they can be educational. In addition, they can adjust the appropriate pace while at the same time providing feedback in form of reinforcement and correction. Applying the rules of science of learning and teaching, one can design games that are educational for all types of learners, including children with ASD. In this paper, two independent user studies have been conducted, demonstrating how serious gaming and e-learning principles can be harnessed in order to intervene, develop or strengthen pivotal developmental skills, like learning novel vocabulary, counting, identifying numbers and colours, and responding to inference questions. We have tested the educational e-book with children diagnosed with ASD and with typically developing children to assess skill acquisition in native language for children with ASD and in English, a foreign language, for typically developing children to demonstrate the educational aspect of the game for all types of learners. We showed that the same e-book in two languages can be used for teaching diferent types of learners through a fun and engaging medium

An interactive E-book with an educational game for children with developmental disorders: a pilot user study

Children diagnosed with Autism spectrum disorder (ASD), as one of the most complex neurodevelopmental disabilities, are characterized by different brain and functioning development, distinct interaction with the environment and different learning patterns, language and social skills impairments, and repetitive auto-stimulating restricting behaviors. It has been shown that computer-assisted intervention is much more attention captivating and interesting to children compared with a classic approach to teaching, allowing for faster acquisition of skills. This makes these tools and the technology highly suitable for teaching children with autism basic developmental skills. In addition, interactive electronic books showed positive outcomes for comprehension and information acquisition in children with ASD, while decreasing inappropriate children behavior in the classroom. In this paper a pilot user study on an e-book with an embedded educational game for children with developmental disorders was presented. The results show that the e-book can be efficiently used for teaching children with ASD basic developmental skills and that the learned skills can be efficiently transfered to new media and environments. The framework will provide preschool children with and without disabilities with appropriate educational software, to build up their early cognitive abilities and school readiness skills, and promote incorporating technology as part of the educational and pedagogical process in school

Mulsemedia in Telecommunication and Networking Education: A Novel Teaching Approach that Improves the Learning Process

The advent and increased use of new technologies, such as innovative mulsemedia and multi-modal content distribution mechanisms, have brought new challenges and diverse opportunities for technology enhanced learning (TEL). NEWTON is a Horizon 2020 European project that revolutionizes the educational process through innovative TEL methodologies and tools, integrated in a pan-European STEM-related learning network platform. This article focuses on one of these novel TEL methodologies (i.e., mulsemedia) and presents how NEWTON enables mulsemedia- enhanced teaching and learning of STEM subjects, with a particular focus on telecommunication and networking related modules. The article also discusses the very promising results of NEWTON case studies carried out with engineering students across two different universities in Spain and Ireland, respectively. The case studies focused on analyzing the impact on the learning process of the mulsemedia-enhanced teaching in the context of telecommunication and networking modules. The main conclusion of the article is that mulsemedia-enhanced education significantly increases students' learning experience and improves their knowledge gain

Innovating Pedagogy 2020: Open University Innovation Report 8

This series of reports explores new forms of teaching, learning and assessment for an interactive world, to guide teachers and policy makers in productive innovation. This eighth report, produced by The Open University in collaboration with the National Institute for Digital Learning (NIDL) in Ireland, describes ten innovations that have the potential to influence education in the coming years