MeIPeAS: An Intelligent Virtual Tutor for Mexican Elementary Schoolchildren

Artículo en revista indizada publicado en Research in Computing ScienceIt is known that virtual tutors have a wide range of functionalities, which have been little exploited and applied in the educational field at the primary level. However, these functionalities allow to offer mechanisms of interaction with students through an interactive dialogue by using text to speech, and even more sophisticated, the recognition and understanding of natural language or speech. In this paper, a personalized virtual tutor for the primary education scenario in Mexico is presented. This virtual tutor is called Mexican Intelligent Pedagogical Agent for Schoolchildren (MeIPeAS) and was created to be used as a pedagogical support mechanism offering a unique attraction for current and future generations of schoolchildren in Mexico. The virtual tutor has been validated in practice in public primary schools of the municipalities of the State of Mexico in Mexico. This validation is to analyze the impact of the user experience from the obtained results having relevant information about the reinforcement of topics taught within the classroom

The Effect of Story Narrative in Multimedia Learning

abstract: ELearning, distance learning, has been a fast-developing topic in educational area. In 1999, Mayer put forward “Cognitive Theory of Multimedia learning” (Moreno, & Mayer, 1999). The theory consisted of several principles. One of the principles, Modality Principle describes that when learners are presented with spoken words, their performance are better than that with on-screen texts (Mayer, R., Dow, & Mayer, S. 2003; Moreno, & Mayer, 1999).It gave an implication that learners performance can be affected by modality of learning materials. A very common tool in education in literature and language is narrative. This way of storytelling has received success in practical use. The advantages of using narrative includes (a) inherent format advantage such as simple structure and familiar language and ideas, (b) motivating learners, (c) facilitate listening, (d) oral ability and (e)provide schema for comparison in comprehension. Although this storytelling method has been widely used in literature, language and even moral education, few studies focused it on science and technology area. The study aims to test the effect of narrative effect in multimedia setting with science topic. A script-based story was applied. The multimedia settings include a virtual human with synthetic speech, and animation on a solar cell lesson. The experiment design is a randomized alternative- treatments design, in which participants are requested to watch a video with pedagogical agent in story format or not. Participants were collected from Amazon Mechanical Turk. Result of transfer score and retention score showed that no significant difference between narrative and non-narrative condition. Discussion was put forward for future study.Dissertation/ThesisMasters Thesis Engineering 201

Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home

Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data with kids going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction tasks in the math learning domain. To recognize kids' speech in realistic home environments, we investigate several ASR systems, including the commercial Google Cloud and the latest open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children for math learning in authentic homes.Comment: Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA) at ACL 202

In the mind and in the technology: the vicarious presence of the teacher in pupil’s learning of science in collaborative group activity at the interactive whiteboard

publication-status: Publishedtypes: ArticleThe focus of research into the use of the interactive whiteboard (IWB) in the classroom has been largely in relation to teacher–pupil interaction, with very little consideration of its possible use as a tool for pupils’ collaborative endeavour. This paper is based upon an ESRC-funded project,1 which considers how pupils use the interactive whiteboard when working together on science-related activities. It provides an analysis of video and other data from science lessons in UK Years 4 and 5 primary classrooms (pupils aged 8–10 years). Concentrating on a series of lessons constructed by three (out of 12) of the project teachers, together with their written and spoken commentaries, it takes each set of lessons as a case for study and comparison. This paper focuses in particular on the nature of the ‘vicarious presence’ of the teacher evident in the group interactions at the board. We address the following questions: How is the teacher’s vicarious presence evident in the work of pupils at the interactive whiteboard? How does this presence influence the behaviour of pupils engaged in science activities? In this account, we suggest that the teacher remotely mediates the activity of the pupils at the board in two specific and interlinked ways. Firstly, the vicarious presence of the teacher seems to be in the minds of pupils, enabling them to appropriate and use introduced rules and procedures, in this case in relation to group talk. Secondly, it is in the ways in which the constructed task environment on the IWB guides and mediates the pupils’ actions, enabling them to connect with, interpret and act upon the teacher intentions for the task. Here, the teacher’s vicarious presence is in the technology. We conclude that the IWB can provide both a tool and an environment that can encourage the creation of a shared dialogic space within which co-constructed knowledge building can take place. However, this only occurs where there is active support from the teacher for collaborative, dialogic activity in the classroom and where the teacher is able to devise tasks that use board affordances to promote active learning and pupil agency

Associated Students Annual Report 2015-2016

Creating Change, Broadening Perspectiveshttps://scholarworks.sjsu.edu/associatedstudents/1008/thumbnail.jp

The Last Decade of HCI Research on Children and Voice-based Conversational Agents

Voice-based Conversational Agents (CAs) are increasingly being used by children. Through a review of 38 research papers, this work maps trends, themes, and methods of empirical research on children and CAs in HCI research over the last decade. A thematic analysis of the research found that work in this domain focuses on seven key topics: ascribing human-like qualities to CAs, CAs’ support of children’s learning, the use and role of CAs in the home and family context, CAs’ support of children’s play, children’s storytelling with CA, issues concerning the collection of information revealed by CAs, and CAs designed for children with differing abilities. Based on our findings, we identify the needs to account for children's intersectional identities and linguistic and cultural diversity and theories from multiple disciples in the design of CAs, develop heuristics for child-centric interaction with CAs, to investigate implications of CAs on social cognition and interpersonal relationships, and to examine and design for multi-party interactions with CAs for different domains and contexts

Affective learning: improving engagement and enhancing learning with affect-aware feedback

This paper describes the design and ecologically valid evaluation of a learner model that lies at the heart of an intelligent learning environment called iTalk2Learn. A core objective of the learner model is to adapt formative feedback based on students’ affective states. Types of adaptation include what type of formative feedback should be provided and how it should be presented. Two Bayesian networks trained with data gathered in a series of Wizard-of-Oz studies are used for the adaptation process. This paper reports results from a quasi-experimental evaluation, in authentic classroom settings, which compared a version of iTalk2Learn that adapted feedback based on students’ affective states as they were talking aloud with the system (the affect condition) with one that provided feedback based only on the students’ performance (the non-affect condition). Our results suggest that affect-aware support contributes to reducing boredom and off-task behavior, and may have an effect on learning. We discuss the internal and ecological validity of the study, in light of pedagogical considerations that informed the design of the two conditions. Overall, the results of the study have implications both for the design of educational technology and for classroom approaches to teaching, because they highlight the important role that affect-aware modelling plays in the adaptive delivery of formative feedback to support learning