An Exploratory Assessment Of Small Group Performance Leveraging Motion Dynamics With Optical Flow

Understanding team behaviors and dynamics are important to better understand and foster better teamwork. The goal of this master\u27s thesis was to contribute to understanding and assessing teamwork in small group research, by analyzing motion dynamics and team performance with non-contact sensing and computational assessment. This thesis\u27s goal is to conduct an exploratory analysis of motion dynamics on teamwork data to understand current limitations in data gathering approaches and provide a methodology to automatically categorize, label, and code team metrics from multi-modal data. We created a coding schema that analyzed different teamwork datasets. We then produced a taxonomy of the metrics from the literature that classify teamwork behaviors and performance. These metrics were grouped on whether they measured communication dynamics or movement dynamics. The review showed movement dynamics in small group research is a potential area to apply more robust computational sensing and detection approaches. To enhance and demonstrate the importance of motion dynamics, we analyzed video and transcript data on a publicly available multi-modal dataset. We determined areas for future study where movement dynamics are potentially correlated to team behaviors and performance. We processed the video data into movement dynamic time series data using an optical flow approach to track and measure motion from the data. Audio data was measured by speaking turns, words used, and keywords used, which were defined as our communication dynamics. Our exploratory analysis demonstrated a correlation between the group performance score using communication dynamics metrics, along with movement dynamics metrics. This assessment provided insights for sensing data capture strategies and computational analysis for future small group research studies

Affect Analysis and Membership Recognition in Group Settings

PhD ThesisEmotions play an important role in our day-to-day life in various ways, including, but not limited to, how we humans communicate and behave. Machines can interact with humans more naturally and intelligently if they are able to recognise and understand humans’ emotions and express their own emotions. To achieve this goal, in the past two decades, researchers have been paying a lot of attention to the analysis of affective states, which has been studied extensively across various fields, such as neuroscience, psychology, cognitive science, and computer science. Most of the existing works focus on affect analysis in individual settings, where there is one person in an image or in a video. However, in the real world, people are very often with others, or interact in group settings. In this thesis, we will focus on affect analysis in group settings. Affect analysis in group settings is different from that in individual settings and provides more challenges due to dynamic interactions between the group members, various occlusions among people in the scene, and the complex context, e.g., who people are with, where people are staying and the mutual influences among people in the group. Because of these challenges, there are still a number of open issues that need further investigation in order to advance the state of the art, and explore the methodologies for affect analysis in group settings. These open topics include but are not limited to (1) is it possible to transfer the methods used for the affect recognition of a person in individual settings to the affect recognition of each individual in group settings? (2) is it possible to recognise the affect of one individual using the expressed behaviours of another member in the same group (i.e., cross-subject affect recognition)? (3) can non-verbal behaviours be used for the recognition of contextual information in group settings? In this thesis, we investigate the affect analysis in group settings and propose methods to explore the aforementioned research questions step by step. Firstly, we propose a method for individual affect recognition in both individual and group videos, which is also used for social context prediction, i.e., whether a person is alone or within a group. Secondly, we introduce a novel framework for cross-subject affect analysis in group videos. Specifically, we analyse the correlation of the affect among group members and investigate the automatic recognition of the affect of one subject using the behaviours expressed by another subject in the same group or in a different group. Furthermore, we propose methods for contextual information prediction in group settings, i.e., group membership recognition - to recognise which group of the person belongs. Comprehensive experiments are conducted using two datasets that one contains individual videos and one contains group videos. The experimental results show that (1) the methods used for affect recognition of a person in individual settings can be transferred to group settings; (2) the affect of one subject in a group can be better predicted using the expressive behaviours of another subject within the same group than using that of a subject from a different group; and (3) contextual information (i.e., whether a person is staying alone or within a group, and group membership) can be predicted successfully using non-verbal behaviours