A multi-modal dance corpus for research into real-time interaction between humans in online virtual environments

We present a new, freely available, multimodal corpus for research into, amongst other areas, real-time realistic interaction between humans in online virtual environments. The specific corpus scenario focuses on an online dance class application scenario where students, with avatars driven by whatever 3D capture technology are locally available to them, can learn choerographies with teacher guidance in an online virtual ballet studio. As the data corpus is focused on this scenario, it consists of student/teacher dance choreographies concurrently captured at two different sites using a variety of media modalities, including synchronised audio rigs, multiple cameras, wearable inertial measurement devices and depth sensors. In the corpus, each of the several dancers perform a number of fixed choreographies, which are both graded according to a number of specific evaluation criteria. In addition, ground-truth dance choreography annotations are provided. Furthermore, for unsynchronised sensor modalities, the corpus also includes distinctive events for data stream synchronisation. Although the data corpus is tailored specifically for an online dance class application scenario, the data is free to download and used for any research and development purposes

Rapid prototyping as a faculty-wide activity: An innovative approach to the redesign of courses and instructional medthods at the University of Twente

At the Faculty of Educational Science and Technology (Toegepaste Onderwijskunde, T.O.) of the University of Twente a revolutionary process of institutional change is occurring. Under the banner of C@MPUS+, we have made a commitment to blend the best of our old values of good teaching and an attractive campus life with new didactics and advanced technologies so that we can extend our already unique curriculum and instructional practice over distance and time, as well as enrich it

Virtual Institutes: Between Immersion and Communication

In the two expressions "virtual reality" and "virtual community", the term "virtual" has different meanings. A virtual reality is a depiction or, more generally speaking, a sensuous representation of reality that allows - mainly by means of interactivity - to experience various features of reality without actually being in contact with the reality depicted. Therefore, any interactive depiction that is able to imitate reality to such an extent that a high degree of sensory-motor immersion becomes possible is called a virtual reality (Heim 1998, 6f). Since reality is always much more complex than its depiction and full of unpredictable surprises, hardly ever a user has doubts about the difference between the depiction and the thing depicted. Nevertheless, there are good reasons for preferring the imitation to the reality: at least, the imitation is usually not as dangerous as reality sometimes turns out to be. Accordingly, quite different platforms for virtual institutes may be used emphasizing either the immersion aspect or the communication aspect. The decision for a platform depends on the goals pursued with the institute: text-based chat systems allow virtual communities to flourish, single-user VRML scenes convey a highly immersive 3D impression to its users. This is particularly true for virtual institutes realized as a 3D environment, as well as for corresponding virtual communities since 3D environments are adequate for certain tasks only. As an overall framework for the evaluation it is helpful to distinguish three major application areas: research, presentation, and communicative work. The Virtual Institute for Image Science (VIB), which we would like to describe in the following (3) as a case study, is almost exclusively designed for the third task: communicative working. It intends to provide a working space persons can share for joint projects despite being physically separated. Before describing the VIB in more detail we would like to give an overview of virtual institutes between the poles of realistic immersion and of communication in a community (2). The discussion of the case study leads to some more general considerations about the balance virtual institutes must find along that bi-polar dimension (4). In the concluding remarks we focus on the technical tools for virtual communities in 3D presently available

User quality of experience of mulsemedia applications

User Quality of Experience (QoE) is of fundamental importance in multimedia applications and has been extensively studied for decades. However, user QoE in the context of the emerging multiple-sensorial media (mulsemedia) services, which involve different media components than the traditional multimedia applications, have not been comprehensively studied. This article presents the results of subjective tests which have investigated user perception of mulsemedia content. In particular, the impact of intensity of certain mulsemedia components including haptic and airflow on user-perceived experience are studied. Results demonstrate that by making use of mulsemedia the overall user enjoyment levels increased by up to 77%

The specification and design of an interactive virtual environment for use in teacher training

In this paper, we examine the rationale behind the specification and design of an interactive, virtual environment, optimized for particular task-based learning activities and the dissemination of information. The software we describe represents a typical British primary school, for use in training Information and Communications Technology (ICT)co-ordinators at primary level. By documenting our ongoing evaluation of both this resource and the technologies used in its implementation, we provide a detailed description of the production process of a prototype piece of software. This highlights the importance of pedagogy, new technologies and project management, and should be of particular interest to multimedia designers and academics preparing to develop innovative learning applications

Desiderata for an Every Citizen Interface to the National Information Infrastructure: Challenges for NLP

In this paper, I provide desiderata for an interface that would enable ordinary people to properly access the capabilities of the NII. I identify some of the technologies that will be needed to achieve these desiderata, and discuss current and future research directions that could lead to the development of such technologies. In particular, I focus on the ways in which theory and techniques from natural language processing could contribute to future interfaces to the NII. Introduction The evolving national information infrastructure (NII) has made available a vast array of on-line services and networked information resources in a variety of forms (text, speech, graphics, images, video). At the same time, advances in computing and telecommunications technology have made it possible for an increasing number of households to own (or lease or use) powerful personal computers that are connected to this resource. Accompanying this progress is the expectation that people will be able to more..

Multimedia search without visual analysis: the value of linguistic and contextual information

This paper addresses the focus of this special issue by analyzing the potential contribution of linguistic content and other non-image aspects to the processing of audiovisual data. It summarizes the various ways in which linguistic content analysis contributes to enhancing the semantic annotation of multimedia content, and, as a consequence, to improving the effectiveness of conceptual media access tools. A number of techniques are presented, including the time-alignment of textual resources, audio and speech processing, content reduction and reasoning tools, and the exploitation of surface features

Synote: weaving media fragments and linked data

While end users could easily share and tag the multimedia resources online, the searching and reusing of the inside content of multimedia, such as a certain area within an image or a ten minutes segment within a one-hour video, is still difficult. Linked data is a promising way to interlink media fragments with other resources. Many applications in Web 2.0 have generated large amount of external annotations linked to media fragments. In this paper, we use Synote as the target application to discuss how media fragments could be published together with external annotations following linked data principles. Our design solves the dereferencing, describing and interlinking methods problems in interlinking multimedia. We also implement a model to let Google index media fragments which improves media fragments' online presence. The evaluation shows that our design can successfully publish media fragments and annotations for both semantic Web agents and traditional search engines. Publishing media fragments using the design we describe in this paper will lead to better indexing of multimedia resources and their consequent findabilit