Gesture Assessment of Teachers in an Immersive Rehearsal Environment

Interactive training environments typically include feedback mechanisms designed to help trainees improve their performance through either guided- or self-reflection. When the training system deals with human-to-human communications, as one would find in a teacher, counselor, enterprise culture or cross-cultural trainer, such feedback needs to focus on all aspects of human communication. This means that, in addition to verbal communication, nonverbal messages must be captured and analyzed for semantic meaning. The goal of this dissertation is to employ machine-learning algorithms that semi-automate and, where supported, automate event tagging in training systems developed to improve human-to-human interaction. The specific context in which we prototype and validate these models is the TeachLivE teacher rehearsal environment developed at the University of Central Florida. The choice of this environment was governed by its availability, large user population, extensibility and existing reflection tools found within the AMITIES framework underlying the TeachLivE system. Our contribution includes accuracy improvement of the existing data-driven gesture recognition utility from Microsoft; called Visual Gesture Builder. Using this proposed methodology and tracking sensors, we created a gesture database and used it for the implementation of our proposed online gesture recognition and feedback application. We also investigated multiple methods of feedback provision, including visual and haptics. The results from the conducted user studies indicate the positive impact of the proposed feedback applications and informed body language in teaching competency. In this dissertation, we describe the context in which the algorithms have been developed, the importance of recognizing nonverbal communication in this context, the means of providing semi- and fully-automated feedback associated with nonverbal messaging, and a series of preliminary studies developed to inform the research. Furthermore, we outline future research directions on new case studies, and multimodal annotation and analysis, in order to understand the synchrony of acoustic features and gestures in teaching context

The Experience of Physical and Social Presence in a Virtual Learning Environment as Impacted by the Affordance of Movement Enabled by Motion Tracking

This research synthesizes existing research findings that social presence (sense of connection with others) and physical presence (sense of being there) increase learning outcomes in Virtual Learning Environments (VLEs) with findings that traditional motion tracking of participants wearing head mounted displays in virtual reality increases both physical and social presence. This information suggests that motion tracking in mixed reality VLEs has a positive impact on social presence and on physical presence. For this study, the affordance of free movement among virtual objects is enabled by Microsoft Kinect tracking of the user\u27s position that is translated to movement of the virtual camera to simulate user movement and proximity to elements of the virtual environment. This study used a mixed method, multimodal approach including qualitative, subjective, objective, and physiological data to measure social and physical presence. The testbed for this research was TLE TeachLivE™, a mixed reality classroom populated with virtual students. The subjective measures are 1) modified Witmer and Singer Questionnaire and 2) Social Presence Instrument (Bailenson, 2002b). The objective measure is a literature based Social Presence Behavioral Coding sheet used to record frequency of occurrences of factors of social presence. Finally, the physiological measure is heart rate as recorded by the MIO Alpha. The primary contribution of this study was that the hypotheses that the affordance of movement in a mixed reality classroom has a positive impact on user perception and experience of a) physical presence and b) social presence in a VLE were supported. This hypothesis was supported in all three measures. The secondary contribution of this research is the literature based Social Presence Behavioral Coding. The final contribution of this research is a research framework that integrates subjective, objective, and physiological measures of social presence in one study. This approach can be applied to various user experience research studies of various VLEs. Finally, in addition to general alignment of the physiological, objective, and subjective measures, there were anecdotal instances of factors of social presence occurring simultaneously with increased heart rate

Interactive Virtual Training: Implementation for Early Career Teachers to Practice Classroom Behavior Management

Teachers that are equipped with the skills to manage and prevent disruptive behaviors increase the potential for their students to achieve academically and socially. Student success increases when prevention strategies and effective classroom behavior management (CBM) are implemented in the classroom. However, teachers with less than 5 years of experience, early career teachers (ECTs), are ill equipped to handle disruptive students. ECTs describe disruptive behaviors as a major factor for stress given their limited training in CBM. As a result, disruptive behaviors are reported by ECTs as one of the main reasons for leaving the field. Virtual training environments (VTEs) combined with advances in virtual social agents can support the training of CBM. Although VTEs for teachers already exist, requirements to guide future research and development of similar training systems have not been defined. We propose a set of six requirements for VTEs for teachers. Our requirements were established from a survey of the literature and from iterative lifecycle activities to build our own VTE for teachers. We present different evaluations of our VTE using methodologies and metrics we developed to assess whether all requirements were met. Our VTE simulates interactions with virtual animated students based on real classroom situations to help ECTs practice their CBM. We enhanced our classroom simulator to further explore two aspects of our requirements: interaction devices and emotional virtual agents. Interactions devices were explored by comparing the effect of immersive technologies on users\u27 experience (UX) such as presence, co-presence, engagement and believability. We adapted our VTE originally built for desktop computer, to be compatible with two immersive VR platforms. Results show that our VTE generates high levels of UX across all VR platforms. Furthermore, we enhanced our virtual students to display emotions using facial expressions as current studies do not address whether emotional virtual agents provide the same level of UX across different VR platforms. We assessed the effects of VR platforms and display of emotions on UX. Our analysis shows that facial expressions have greater impact when using a desktop computer. We propose future work on immersive VTEs using emotional virtual agents

Exploring Preservice Teachers\u27 Affective Response to Disruptive Student Behavior in an Immersive Simulation Classroom

This mixed methods study investigated changes in preservice teachers\u27 affective response to disruptive student behavior within the TeachLivE, immersive simulation classroom. Preservice teachers completed two simulation teaching sessions, during which they were exposed to five different disruptive student behavior events in each. All teaching sessions were recorded and post-processed using iMotions Affectiva Affdex software to collect data on preservice teachers\u27 emotion expression and valence during their teaching experiences. At the end of each teaching session, participants completed a self-report survey on their level of stress. Simulated teaching sessions were followed-up with video stimulated recall sessions where participants reflected on their feelings during the simulation. The goal of this research was to examine changes in preservice teachers\u27 affective response to stress, with repeated exposure to disruptive student behavior, to determine if it had a desensitization effect, potentially increasing emotional regulation ability and decreasing negative emotional responses. Descriptive statistics were used to examine differences in emotional valence by disruptive student events and teaching sessions. Paired samples t-tests were conducted to examine if mean differences existed in self-reported stress within and between teaching sessions. Additional qualitative analysis of video stimulated recall sessions was conducted using thematic analysis. Analysis revealed minimal difference in preservice teachers\u27 positive or negative emotional valence in response to disruptive student behavior events within and between teaching sessions. There was a statistically significant change in self-reported stress from the first simulated teaching session to the second. Analysis of video stimulated recall reflections revealed themes of cognitive dissonance, behavior-induced stress, and difficulty with virtual behavior management

Exploring Interactions between Adult English Learners and Their TeachLivE Digital Character Peers

Peer interaction is an important part of language learning. The results of previous studies showed advantages and disadvantages of peer interaction between native speakers and English learners. Using the educational simulation platform TeachLivE as the interaction platform, this qualitative descriptive case study explored the possibilities of bridging the gap between classroom language practice and real-life second language communication. The study recruited eight participants for a task-based interaction project. Using conversation analysis, the study revealed details in communication between adult English learners and their digital character peers. During the interactions, the participants were actively involved and the most frequent communication patterns were collaborative and cooperative. The analysis of interviews of participants and the interactor explored the factors that influenced the communication patterns between the young adult English learners and their digital peers. The results showed that the communication experience with digital characters was authentic and the virtual platform was critical to build the confidence of English learners\u27 language use. Moreover, the multiple digital characters manipulated by interaction protocols were also helpful to create a scaffolding effect for practicing oral communication for the English learners

Training Peer Mentors in Reinforcement and Modeling with Prompting (RaMP): Using Mixed Reality Simulation to Encourage Social Reciprocity Skills in Adolescents with Autism Spectrum Disorder

Children diagnosed with autism spectrum disorder (ASD) can face many challenges, including limited ability in social reciprocity, nonverbal communication, and restrictive and repetitive behaviors and interests as a result of the disorder (APA, 2013). This hinders their ability to build and maintain friendships in the same manner as their typically developing peers and affects their interactions, speech and language abilities, and emotional regulation (Whitman & DeWitt, 2011). Peer Mentors (PMs) are trained to support the development of these skills as part of the Peer Mediated Instruction and Intervention (PMII) evidence-based practice (EBP). However, and explicit training program is not always evident nor is the opportunity for purposeful practice prior to working with the children with ASD. Further, a gap in literature exists regarding training adolescent PMs. The purpose of this study was to determine if when PMs receive explicit instruction in the reinforcement and modeling with prompting (RaMP) training program they become more adequately prepared to encourage social reciprocity skills from adolescents with ASD. The additional mixed reality simulation (MRS) platform provided the practice component. A multiple baseline single subject design was implemented to determine that the proposed RaMP training program was in fact effective across three PM participants. The study supports the need for an explicit training program for PMs. Each PM not only increased in their fidelity of the RaMP skills, but they also maintained and were able to generalize the skills taught even after the support was withdrawn

Using Simulation For Law Enforcement De-escalation Training

Law enforcement needs simulation practice tailored to de-escalation skills. Law enforcement officers spend most of their training time practicing how to use force appropriately and very little time practicing how to avoid using force. There is little research into the best ways for law enforcement to reduce their use of force and the best ways to provide training to individuals to coach them to avoid using force. There are many training programs promoting de-escalation techniques, but there is little research into the effectiveness of these programs beyond the ability of individuals to perform the skills that are trained. There is little to show that using these skills reduces crime, reduces the need for incarceration, reduces the level of violence, or improves the communities where they are used. The scope of this project addresses a small part of this gap by examining different ways simulators can be used to provide practice in the skills that are taught. This dissertation contributes to the field of simulation by demonstrating how virtual reality can address deficits in law enforcement training. It does so by studying which techniques are most appropriate in some scenarios and how to better train officers to use them. This project looks at different ways of allowing police officers to practice de-escalation skills to see if these have any bearing on an officer\u27s approach to de-escalation and if the officer responds positively to the practice. This research does not attempt to take the next step of measuring the use of these skills outside the training environment. The results indicate active-duty officers have a positive response to any attempt to practice or promote de-escalation and are especially positive about the potential for training in realistic, situationally appropriate virtual environments

Research-Informed Teaching in a Global Pandemic: "Opening up" Schools to Research

The teacher-research agenda has become a significant consideration for policy and professional development in a number of countries. Encouraging research-based teacher education programmes remains an important goal, where teachers are able to effectively utilize educational research as part of their work in school settings and to reflect on and enhance their professional development. In the last decade, teacher research has grown in importance across the three i’s of the teacher learning continuum: initial, induction and in-service teacher education. This has been brought into even starker relief with the global spread of COVID-19, and the enforced and emergency, wholesale move to digital education. Now, perhaps more than ever, teachers need the perspective and support of research-led practice, particularly in how to effectively use Internet technologies to mediate and enhance learning, teaching and assessment online, and new blended modalities for education that must be physically distant. The aim of this paper is to present a number of professional development open educational systems which exist or are currently being developed to support teachers internationally, to engage with, use and do research. Exemplification of the opening up of research to schools and teachers is provided in the chapter through reference to the European Union-funded Erasmus + project, BRIST: Building Research Infrastructures for School Teachers. BRIST is developing technology to coordinate and support teacher-research at a European level

Designing personalised, authentic and collaborative learning with mobile devices: Confronting the challenges of remote teaching during a pandemic.

This article offers teachers a digital pedagogical framework, research-inspired and underpinned by socio-cultural theory, to guide the design of personalised, authentic and collaborative learning scenarios for students using mobile devices in remote learning settings during this pandemic. It provides a series of freely available online resources underpinned by our framework, including a mobile learning toolkit, a professional learning app, and robust, validated surveys for evaluating tasks. Finally, it presents a set of evidence-based principles for effective innovative teaching with mobile devices