Gesture Assessment of Teachers in an Immersive Rehearsal Environment

Interactive training environments typically include feedback mechanisms designed to help trainees improve their performance through either guided- or self-reflection. When the training system deals with human-to-human communications, as one would find in a teacher, counselor, enterprise culture or cross-cultural trainer, such feedback needs to focus on all aspects of human communication. This means that, in addition to verbal communication, nonverbal messages must be captured and analyzed for semantic meaning. The goal of this dissertation is to employ machine-learning algorithms that semi-automate and, where supported, automate event tagging in training systems developed to improve human-to-human interaction. The specific context in which we prototype and validate these models is the TeachLivE teacher rehearsal environment developed at the University of Central Florida. The choice of this environment was governed by its availability, large user population, extensibility and existing reflection tools found within the AMITIES framework underlying the TeachLivE system. Our contribution includes accuracy improvement of the existing data-driven gesture recognition utility from Microsoft; called Visual Gesture Builder. Using this proposed methodology and tracking sensors, we created a gesture database and used it for the implementation of our proposed online gesture recognition and feedback application. We also investigated multiple methods of feedback provision, including visual and haptics. The results from the conducted user studies indicate the positive impact of the proposed feedback applications and informed body language in teaching competency. In this dissertation, we describe the context in which the algorithms have been developed, the importance of recognizing nonverbal communication in this context, the means of providing semi- and fully-automated feedback associated with nonverbal messaging, and a series of preliminary studies developed to inform the research. Furthermore, we outline future research directions on new case studies, and multimodal annotation and analysis, in order to understand the synchrony of acoustic features and gestures in teaching context

A Case Study To Track Teacher Gestures And Performance In A Virtual Learning Environment

As part of normal interpersonal communication, people send and receive messages with their body, especially with their hands. Gestures play an important role in teacher-student classroom interactions. In the domain of education, many research projects have focused on the study of such gestures either in real classrooms or in tutorial settings with experienced teachers. Novice teachers especially need to understand the messages they are sending through nonverbal communication as this can have a major effect on their ability to manage behaviors and deliver content. Such learning should optimally occur before experiencing the real classroom. To assist in this process, we have developed a virtual classroom environment-TeachLivE-and used it for teacher practice, reflection and assessment. This paper investigates the way teachers use gestures in the virtual classroom settings of TeachLivE. Biology and algebra teachers were evaluated in our study. Analysis of video recordings from real and virtual environment seems to indicate that algebra teachers gesture significantly more often than biology teachers. These results have implications for providing useful feedback to participant teachers