Communication skills training intervention based on automated recognition of nonverbal signals

There have been promising studies that show a potential of providing social signal feedback to improve communication skills. However, these studies have primarily focused on unimodal methods of feedback. In addition to this, studies do not assess whether skills are maintained after a given time. With a sample size of 22 this paper investigates whether multimodal social signal feedback is an efective method of improving communication in the context of media interviews. A pre-post experimental evaluation of media skills training intervention is presented which compares standard feedback with augmented feedback based on automated recognition of multimodal social signals. Results revealed signifcantly diferent training efects between the two conditions. However, the initial experiment study failed to show signifcant diferences in human judgement of performance. A 6-month follow-up study revealed human judgement ratings were higher for the experiment group. This study suggests that augmented selective multimodal social signal feedback is an efective method for communication skills training.Defence, Science and Defence Lab; Engineering and Physical Science Research Council

Utilizing Asynchronous Online Modules to Educate Preservice Teachers to Address Bullying Behaviors for Elementary Students with Autism Spectrum Disorder

Due to social and communication deficits, individuals with Autism Spectrum Disorder (ASD) are targeted for school bullying more than other populations. With an increased number of individuals with ASD being served in general education classrooms, teachers are at the front line of defense for this vulnerable population. Many teachers and preservice teachers lack experience and self-confidence when dealing with situations related to individuals with ASD in their inclusive classrooms. While research on educating preservice teachers to deal with bullying behaviors is limited, the importance of providing high-quality teacher preparation programs for upcoming teachers remains at the upmost importance. The purpose of this study was to examine the effects of three online modules on preservice teachers\u27 ability to identify and respond appropriately to situations of bullying involving individuals with ASD in their classroom. This study utilized an experimental group design to determine the impact of the modules for participants in the treatment group compared to participants in the control group who received a prerecorded lecture on the topic of bullying in schools. The researcher found this intervention to have a positive change between the pre-test and the post-test for participants receiving the modules; however, the results were not statistically significant

Gesture Assessment of Teachers in an Immersive Rehearsal Environment

Interactive training environments typically include feedback mechanisms designed to help trainees improve their performance through either guided- or self-reflection. When the training system deals with human-to-human communications, as one would find in a teacher, counselor, enterprise culture or cross-cultural trainer, such feedback needs to focus on all aspects of human communication. This means that, in addition to verbal communication, nonverbal messages must be captured and analyzed for semantic meaning. The goal of this dissertation is to employ machine-learning algorithms that semi-automate and, where supported, automate event tagging in training systems developed to improve human-to-human interaction. The specific context in which we prototype and validate these models is the TeachLivE teacher rehearsal environment developed at the University of Central Florida. The choice of this environment was governed by its availability, large user population, extensibility and existing reflection tools found within the AMITIES framework underlying the TeachLivE system. Our contribution includes accuracy improvement of the existing data-driven gesture recognition utility from Microsoft; called Visual Gesture Builder. Using this proposed methodology and tracking sensors, we created a gesture database and used it for the implementation of our proposed online gesture recognition and feedback application. We also investigated multiple methods of feedback provision, including visual and haptics. The results from the conducted user studies indicate the positive impact of the proposed feedback applications and informed body language in teaching competency. In this dissertation, we describe the context in which the algorithms have been developed, the importance of recognizing nonverbal communication in this context, the means of providing semi- and fully-automated feedback associated with nonverbal messaging, and a series of preliminary studies developed to inform the research. Furthermore, we outline future research directions on new case studies, and multimodal annotation and analysis, in order to understand the synchrony of acoustic features and gestures in teaching context

Improving Social Communication Skills Using Kinesics Feedback

Interactive training environments typically include feedback mechanisms designed to help trainees improve their performance through guided or self-reflection. When the training system deals with human-to-human communications, as one would find in a teacher, counselor or cross-cultural trainer, such feedback needs to focus on all aspects of human communication. This means that, in addition to verbal communication, nonverbal messages (kinesics in particular) must be captured and analyzed for semantic meaning. The goal of this research is to introduce interactive training models developed to improve human-to-human interaction. The specific context in which we prototype and validate these models is the TeachLivE™ teacher rehearsal environment developed at the University of Central Florida. We implemented an online gesture recognition application on top of the Microsoft Kinect software development kit with multiple feedback channels including visual and haptics. In a study of twelve participants rehearsing a teaching session in TeachLivE, we found that the online gesture recognition tool and its associated feedback method are effective and non-intrusive approaches for the purpose of communication-skill training. The algorithms employed, the results, and the implications for other interactive contexts are discussed in this paper