Multimodality in Group Communication Research

Team interactions are often multisensory, requiring members to pick up on verbal, visual, spatial and body language cues. Multimodal research, research that captures multiple modes of communication such as audio and visual signals, is therefore integral to understanding these multisensory group communication processes. This type of research has gained traction in biomedical engineering and neuroscience, but it is unclear the extent to which communication and management researchers conduct multimodal research. Our study finds that despite its' utility, multimodal research is underutilized in the communication and management literature's. This paper then covers introductory guidelines for creating new multimodal research including considerations for sensors, data integration and ethical considerations.Comment: 27 pages, 3 figure

An Exploratory Assessment Of Small Group Performance Leveraging Motion Dynamics With Optical Flow

Understanding team behaviors and dynamics are important to better understand and foster better teamwork. The goal of this master\u27s thesis was to contribute to understanding and assessing teamwork in small group research, by analyzing motion dynamics and team performance with non-contact sensing and computational assessment. This thesis\u27s goal is to conduct an exploratory analysis of motion dynamics on teamwork data to understand current limitations in data gathering approaches and provide a methodology to automatically categorize, label, and code team metrics from multi-modal data. We created a coding schema that analyzed different teamwork datasets. We then produced a taxonomy of the metrics from the literature that classify teamwork behaviors and performance. These metrics were grouped on whether they measured communication dynamics or movement dynamics. The review showed movement dynamics in small group research is a potential area to apply more robust computational sensing and detection approaches. To enhance and demonstrate the importance of motion dynamics, we analyzed video and transcript data on a publicly available multi-modal dataset. We determined areas for future study where movement dynamics are potentially correlated to team behaviors and performance. We processed the video data into movement dynamic time series data using an optical flow approach to track and measure motion from the data. Audio data was measured by speaking turns, words used, and keywords used, which were defined as our communication dynamics. Our exploratory analysis demonstrated a correlation between the group performance score using communication dynamics metrics, along with movement dynamics metrics. This assessment provided insights for sensing data capture strategies and computational analysis for future small group research studies