Development of the User-State Conventions for the Multimodal Corpus in SmartKom

This contribution deals with the problems and solutions of finding procedures for the labeling of a multimodal data corpus that is created within the SmartKom project. The goal of the SmartKom project is the development of an intelligent computer-user interface that should allow almost natural communication with an adaptive and selfexplanatory machine. The system does not only accept input in form of natural speech but also in form of gestures. Additionally the facial expression and prosody of speech is analyzed. To train recognizers and to explore how users interact with the system, data is collected in so-called Wizard-of-Oz experiments. Speech is transliterated and gestures as well as user-states 2 are labeled. At the start of the project only the speech transliteration conventions existed. Therefore conventions and procedures to label the video data had to be newly created. In this contribution we will describe the development process of the User-State Labeling Conventions 3 as an example for our strategy of "functional labeling". The development and structure of the gesture labeling is described in detail in Steininger et al. [1]. The transliteration conventions can be found in Beringer et al. [2]. The special problem of combining the information of the different labeling steps and the transliteration is discussed in Schiel et al. at this workshop

Integration of multi-modal data and annotations into a simple extendable form: the extension of the BAS Partitur Format

Multi-modal resources typically consist of very different data in terms of content and format. This paper discusses a practical solution for the integration of different physical signals as well as associated symbolic data into a common framework. There are ongoing efforts like for instance the ISLE project to develop guidelines and best-of-practice for the standardized representation of such data collections. Since these efforts have not yet converged into a widely accepted concept, we suggest as a starting point to use two different already existing frameworks that can be easily combined for this purpose: The QuickTime format for the handling of synchronized multi-modal signals and the (extended) BAS Partitur Format for the handling of all symbolic data. We can show that with this simple approach it is already possible to integrate the rather complex data streams of the SmartKom Corpus into an easy-to-use format that will be distributed via the Bavarian Archive for Speech Signals (BAS) starting in July 2002

Integration of Multi-modal Data and Annotations into a Simple Extendable Form: the Extension of the BAS Partitur Format

Multi-modal resources typically consist of very different data in terms of content and format. This paper discusses a practical solution for the integration of different physical signals as well as associated symbolic data into a common framework. There are ongoing efforts like for instance the ISLE project to develop guidelines and best-of-practice for the standardized representation of such data collections. Since these efforts have not yet converged into a widely accepted concept, we suggest as a starting point to use two different already existing frameworks that can be easily combined for this purpose: The QuickTime format for the handling of synchronized multi-modal signals and the (extended) BAS Partitur Format for the handling of all symbolic data. We can show that with this simple approach it is already possible to integrate the rather complex data streams of the SmartKom Corpus into an easy-to-use format that will be distributed via the Bavarian Archive for Speech Signals (BAS) starting in July 2002

Vicarious learning and case-based teaching: developing health science students' clinical reasoning skills

Developing expertise in clinical reasoning and diagnosis poses an enormous challenge to health science students. In a three-phase project, we identified the difficulties that health science students in initial training have with clinical reasoning, designed and implemented an online vicarious learning system to address those difficulties, and evaluated the outcomes in terms of learners' experiences, diagnostic skill development and use of professional language