Recognizing Teamwork Activity In Observations Of Embodied Agents

This thesis presents contributions to the theory and practice of team activity recognition. A particular focus of our work was to improve our ability to collect and label representative samples, thus making the team activity recognition more efficient. A second focus of our work is improving the robustness of the recognition process in the presence of noisy and distorted data. The main contributions of this thesis are as follows: We developed a software tool, the Teamwork Scenario Editor (TSE), for the acquisition, segmentation and labeling of teamwork data. Using the TSE we acquired a corpus of labeled team actions both from synthetic and real world sources. We developed an approach through which representations of idealized team actions can be acquired in form of Hidden Markov Models which are trained using a small set of representative examples segmented and labeled with the TSE. We developed set of team-oriented feature functions, which extract discrete features from the high-dimensional continuous data. The features were chosen such that they mimic the features used by humans when recognizing teamwork actions. We developed a technique to recognize the likely roles played by agents in teams even before the team action was recognized. Through experimental studies we show that the feature functions and role recognition module significantly increase the recognition accuracy, while allowing arbitrary shuffled inputs and noisy data

Role-Based Teamwork Activity Recognition In Observations Of Embodied Agent Actions

Recognizing team actions in the behavior of embodied agents has many practical applications and had seen significant progress in recent years. One approach with proven results is based on HMM-based recognition of spatio-temporal patterns in the behavior of the agents. While it had been shown to work on real-world datasets, this approach was found to be brittle. In this paper we present two contributions which together can significantly increase the robustness of teamwork activity recognition. First we introduce a technique to reduce high dimensional continuous input data to a set of discrete features, which capture the essential components of the team actions. Second, we prefix the actual team action recognition with a role recognition module, which allows us to present the recognizer with arbitrarily shuffled input, and still obtain high recognition rates. We validate the improved accuracy and robustness of the team action recognizer on datasets derived from captured real world data. Copyright © 2008, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved