Multimodal person recognition for human-vehicle interaction

Next-generation vehicles will undoubtedly feature biometric person recognition as part of an effort to improve the driving experience. Today's technology prevents such systems from operating satisfactorily under adverse conditions. A proposed framework for achieving person recognition successfully combines different biometric modalities, borne out in two case studies

Multiple Media Interfaces for Music Therapy

This article describes interfaces (and the supporting technological infrastructure) to create audiovisual instruments for use in music therapy. In considering how the multidimensional nature of sound requires multidimensional input control, we propose a model to help designers manage the complex mapping between input devices and multiple media software. We also itemize a research agenda

Relating Objective and Subjective Performance Measures for AAM-based Visual Speech Synthesizers

We compare two approaches for synthesizing visual speech using Active Appearance Models (AAMs): one that utilizes acoustic features as input, and one that utilizes a phonetic transcription as input. Both synthesizers are trained using the same data and the performance is measured using both objective and subjective testing. We investigate the impact of likely sources of error in the synthesized visual speech by introducing typical errors into real visual speech sequences and subjectively measuring the perceived degradation. When only a small region (e.g. a single syllable) of ground-truth visual speech is incorrect we find that the subjective score for the entire sequence is subjectively lower than sequences generated by our synthesizers. This observation motivates further consideration of an often ignored issue, which is to what extent are subjective measures correlated with objective measures of performance? Significantly, we find that the most commonly used objective measures of performance are not necessarily the best indicator of viewer perception of quality. We empirically evaluate alternatives and show that the cost of a dynamic time warp of synthesized visual speech parameters to the respective ground-truth parameters is a better indicator of subjective quality

A case study of intuition and design: Building a tool for parents of premature babies and the nursing staff who care for them

The paper presents a research-based study project conveyed by Media Lab Helsinki in 2007–2008. During the process, the design team constructed a unique audiovisual tool that provides emotional support in coping with the challenges of a premature birth. The purpose of the two-hour DVD, with its three separate parts, is to help family, intensive care staff, and other associated health care personnel to better connect with each other and the tiny babies. As the first audiovisual material, which aims to explain premature babies’ communication and interaction, the end product has proven to be of significant value to Finnish neonatal care and other associated fields. The project was realized in cooperation with the Neonatal Intensive Care Unit in Children’s Hospital, HUCH (Helsinki University Central Hospital), and the design team worked in a network of shared expertise. In the case study the authors used co-design and self-reflection methods to monitor graduate students working with a society-level challenge. The focus was upon the utilisation of intuition in innovations, problem solving, and project management. During the design process, the authors observed the working theory: intuition is the basis of successful decision-making, which leads to innovative solutions. Throughout the design process, the team collected systematically feedback, which was later utilised as a corner stone in the case study. The authors believe that relying on intuition, the design team ended up addressing many hidden issues of the clinic and parenthood of premature babies’. This paper tries to unfold the evolutionary working approach used during the design process, presents highlights of the case study research as well as illuminates the intuitive way of working from designers’ point of view. Keywords: Design; Intuition; Case Study; Shared Expertise; Innovations; Digital Media; Evolutionary Process</p

King's speech: pronounce a foreign language with style

Computer assisted pronunciation training requires strategies that capture the attention of the learners and guide them along the learning pathway. In this paper, we introduce an immersive storytelling scenario for creating appropriate learning conditions. The proposed learning interaction is orchestrated by a spoken karaoke. We motivate the concept of the spoken karaoke and describe our design. Driven by the requirements of the proposed scenario, we suggest a modular architecture designed for immersive learning applications. We present our prototype system and our approach for the processing of spoken and visual interaction modalities. Finally, we discuss how technological challenges can be addressed in order to enable the learner's self-evaluation

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research