Automated pattern analysis in gesture research : similarity measuring in 3D motion capture models of communicative action

The question of how to model similarity between gestures plays an important role in current studies in the domain of human communication. Most research into recurrent patterns in co-verbal gestures – manual communicative movements emerging spontaneously during conversation – is driven by qualitative analyses relying on observational comparisons between gestures. Due to the fact that these kinds of gestures are not bound to well-formedness conditions, however, we propose a quantitative approach consisting of a distance-based similarity model for gestures recorded and represented in motion capture data streams. To this end, we model gestures by flexible feature representations, namely gesture signatures, which are then compared via signature-based distance functions such as the Earth Mover's Distance and the Signature Quadratic Form Distance. Experiments on real conversational motion capture data evidence the appropriateness of the proposed approaches in terms of their accuracy and efficiency. Our contribution to gesture similarity research and gesture data analysis allows for new quantitative methods of identifying patterns of gestural movements in human face-to-face interaction, i.e., in complex multimodal data sets

Automated pattern analysis in gesture research : similarity measuring in 3D motion capture models of communicative action

The question of how to model similarity between gestures plays an important role in current studies in the domain of human communication. Most research into recurrent patterns in co-verbal gestures – manual communicative movements emerging spontaneously during conversation – is driven by qualitative analyses relying on observational comparisons between gestures. Due to the fact that these kinds of gestures are not bound to well-formedness conditions, however, we propose a quantitative approach consisting of a distance-based similarity model for gestures recorded and represented in motion capture data streams. To this end, we model gestures by flexible feature representations, namely gesture signatures, which are then compared via signature-based distance functions such as the Earth Mover's Distance and the Signature Quadratic Form Distance. Experiments on real conversational motion capture data evidence the appropriateness of the proposed approaches in terms of their accuracy and efficiency. Our contribution to gesture similarity research and gesture data analysis allows for new quantitative methods of identifying patterns of gestural movements in human face-to-face interaction, i.e., in complex multimodal data sets

Distance based similarity models for content based multimedia retrieval

Concomitant with the digital information age, an increasing amount of multimedia data is generated, processed, and finally stored in very large multimedia data collections. The expansion of the internet and the spread of mobile devices allow users the utilization of multimedia data everywhere. Multimedia data collections tend to grow continuously and are thus no longer manually manageable by humans. As a result, multimedia retrieval approaches that allow efficient information access to massive multimedia data collections become immensely important. These approaches support users in searching multimedia data collections in a content-based way based on a similarity model. A similarity model defines the similarity between multimedia data objects and is the core of each multimedia retrieval approach. This thesis investigates distance-based similarity models in the scope of content-based multimedia retrieval. After an introduction to content-based multimedia retrieval, the first part deals with the fundamentals of modeling and comparing contents of multimedia data. This is complemented by an explanation of different query types and query processing approaches. A novel distance-based similarity model, namely the Signature Quadratic Form Distance, is developed in the second part of this thesis. The theoretical and empirical properties are investigated and an extension of this model to continuous feature representations is proposed. Finally, different techniques for efficient similarity query processing are studied and evaluated

Distance based similarity models for content based multimedia retrieval

Concomitant with the digital information age, an increasing amount of multimedia data is generated, processed, and finally stored in very large multimedia data collections. The expansion of the internet and the spread of mobile devices allow users the utilization of multimedia data everywhere. Multimedia data collections tend to grow continuously and are thus no longer manually manageable by humans. As a result, multimedia retrieval approaches that allow efficient information access to massive multimedia data collections become immensely important. These approaches support users in searching multimedia data collections in a content-based way based on a similarity model. A similarity model defines the similarity between multimedia data objects and is the core of each multimedia retrieval approach. This thesis investigates distance-based similarity models in the scope of content-based multimedia retrieval. After an introduction to content-based multimedia retrieval, the first part deals with the fundamentals of modeling and comparing contents of multimedia data. This is complemented by an explanation of different query types and query processing approaches. A novel distance-based similarity model, namely the Signature Quadratic Form Distance, is developed in the second part of this thesis. The theoretical and empirical properties are investigated and an extension of this model to continuous feature representations is proposed. Finally, different techniques for efficient similarity query processing are studied and evaluated