HuComTech Multimodal Corpus Annotation

L

Labeling of Gestures in SmartKom - The Coding System

The SmartKom project is concerned with the development of an intelligent computer-user interface that allows almost natural communication and gesture input. For the training of the gesture analyzer, data is collected in so called Wizard-of-Oz experiments. Recordings of subjects are made and labeled off-line with respect to the gestures that were used. This article is concerned with the coding of these gestures. The presented concept is the first step in the development of a practical gesture coding system specifically designed for the description of communicative and non-communicative gestures that typically show up in human-machine dialogues. After a short overview over the development process and the special requirements for the project, the labels will be described in detail. We will conclude with a short outline of open points and differences to the well-known taxonomy for gestures of Ekman [1]. Keywords: Human-Machine interaction, annotation of corpora, multimodal dialogue systems, gesture coding system