The Multimodal Tutor: Adaptive Feedback from Multimodal Experiences

This doctoral thesis describes the journey of ideation, prototyping and empirical testing of the Multimodal Tutor, a system designed for providing digital feedback that supports psychomotor skills acquisition using learning and multimodal data capturing. The feedback is given in real-time with machine-driven assessment of the learner's task execution. The predictions are tailored by supervised machine learning models trained with human annotated samples. The main contributions of this thesis are: a literature survey on multimodal data for learning, a conceptual model (the Multimodal Learning Analytics Model), a technological framework (the Multimodal Pipeline), a data annotation tool (the Visual Inspection Tool) and a case study in Cardiopulmonary Resuscitation training (CPR Tutor). The CPR Tutor generates real-time, adaptive feedback using kinematic and myographic data and neural networks

Computational model of negotiation skills in virtual artificial agents

Negotiation skills represent crucial abilities for engaging in effective social interactions in formal and informal settings. Serious games, intelligent systems and virtual agents can provide solid tools upon which one-to-one training and assessment can be reliably made available. The aim of the present work is to fill the gap between the recent growing interest towards soft skills, and the lack of a robust and modern methodology for supporting their investigation. A computational model for the development of Enact, a 3D virtual intelligent platform for training and testing negotiation skills, will be presented. The serious game allows users to interact with simulated peers in scenarios depicting daily life situations and receive a psychological assessment and adaptive training reflecting their negotiation abilities. To pursue this goal, this work has gone through different research stages, each with a unique methodology, results and discussion described in its specific section. In the first phase, the platform was designed to operationalize the examined negotiation theory, developed and assessed. The negotiation styles considered, consistently with previous findings, have been found not to correlate with personality traits, coping strategies and perceived self-efficacy. The serious game has been widely tested for its usability and underwent two development and release stages aimed at improving its accuracy, usability and likeability. The variables measured by the platform have been found to predict in all cases at least two of the negotiation styles considered. Concerning the user feedback, the game has been judged as useful, more pleasant than the traditional test, and the perceived time spent on the game resulted significantly lower than the real time spent. In the second stage of this research, the game scenarios were used to collect a dataset of documents containing natural language negotiations between users and the virtual agents. The dataset was used to assess the correlations between the personal pronouns' use and the negotiation styles. Results showed that more engaged styles generally used pronouns with a significantly higher frequency than less engaged styles. Styles with a high concern for self showed a higher frequency of singular personal pronouns while styles with a high concern for others used significantly more relational pronouns. The corpus of documents was also used to perform multiclass classification on the negotiation styles using machine learning. Both linear (SVM) and non-linear models (MNB, CNN) performed reliably with a state-of-the-art accuracy

Explicit Feedback Within Game-based Training: Examining The Influence Of Source Modality Effects On Interaction

This research aims to enhance Simulation-Based Training (SBT) applications to support training events in the absence of live instruction. The overarching purpose is to explore available tools for integrating intelligent tutoring communications in game-based learning platforms and to examine theory-based techniques for delivering explicit feedback in such environments. The primary tool influencing the design of this research was the Generalized Intelligent Framework for Tutoring (GIFT), a modular domain-independent architecture that provides the tools and methods to author, deliver, and evaluate intelligent tutoring technologies within any training platform. Influenced by research surrounding Social Cognitive Theory and Cognitive Load Theory, the resulting experiment tested varying approaches for utilizing an Embodied Pedagogical Agent (EPA) to function as a tutor during interaction in a game-based environment. Conditions were authored to assess the tradeoffs between embedding an EPA directly in a game, embedding an EPA in GIFT’s browser-based Tutor-User Interface (TUI), or using audio prompts alone with no social grounding. The resulting data supports the application of using an EPA embedded in GIFT’s TUI to provide explicit feedback during a game-based learning event. Analyses revealed conditions with an EPA situated in the TUI to be as effective as embedding the agent directly in the game environment. This inference is based on evidence showing reliable differences across conditions on the metrics of performance and self-reported mental demand and feedback usefulness items. This research provides source modality tradeoffs linked to tactics for relaying training relevant explicit information to a user based on real-time performance in a game

A Review of the Open Educational Resources (OER) Movement: Achievements, Challenges, and New Opportunities

Examines the state of the foundation's efforts to improve educational opportunities worldwide through universal access to and use of high-quality academic content

Expanding evidence approaches for learning in a digital world

Executive Summary: Relatively low-cost digital technology is ubiquitous in daily life and work. The Web is a vast source of information, communication, and connection opportunities available to anyone with Internet access. Most professionals and many students have a mobile device in their pocket with more computing power than early supercomputers. These technological advances hold great potential for improving educational outcomes, but by themselves hardware and networks will not improve learning. Decades of research show that high-quality learning resources and sound implementations are needed as well.The learning sciences have found that today’s technologies offer powerful capabilities for creating high-quality learning resources, such as capabilities for visualization, simulation, games, interactivity, intelligent tutoring, collaboration, assessment, and feedback. Further, digital learning resources enable rapid cycles of iterative improvement, and improvements to resources can be instantly distributed over the Internet. In addition, digital technologies are attracting exciting new talent, both from other industries and from the teacher workforce itself, into the production of digital learning resources. Yet even with so many reasons to expect dramatic progress, something more—better use of evidence— is needed to support the creation, implementation, and continuous enhancement of high-quality learning resources in ways that improve student outcomes

The student-produced electronic portfolio in craft education

The authors studied primary school students’ experiences of using an electronic portfolio in their craft education over four years. A stimulated recall interview was applied to collect user experiences and qualitative content analysis to analyse the collected data. The results indicate that the electronic portfolio was experienced as a multipurpose tool to support learning. It makes the learning process visible and in that way helps focus on and improves the quality of learning. © ISLS.Peer reviewe

Towards Failure-Based Instructional Design: A Phenomenological Study of the Perceptions of Drone Pilots About the Use of Simulations to Promote Failure-Based Learning

Simulations have become increasingly popular in many contexts, particularly for performance optimization, testing, and safety (Aldrich, 2003). By nature, simulations immerse the learner in an environment that is an approximate imitation of the situation or process to be learned (Baek, 2009). In the literature, there is a lack of qualitative research on the perceptions of learners regarding the use of failure-based learning in simulations. The idea of learning through failure experiences is not a new concept, yet, to date, no instructional design models have discussed how to employ failure strategically within education (Tawfik, Rong, & Choi, 2015). This study utilized Tawfik et al.’s (2015) unified model of failure and learning systems design to create a drone flight simulation designed to focus on safely operating a drone while capturing high-quality aerial videography. Data collection included semi-structured interviews with 16 licensed drone pilots. This study illuminates the pilots’ perceptions and understanding about employing a failure-based learning model in a drone flight training simulation. Key findings from a thematic analysis of the interviews were that learners find value in experiencing and learning from failure and that the failure experiences led to increased self-confidence and intrinsic motivation