Continuous Interaction with a Virtual Human

Attentive Speaking and Active Listening require that a Virtual Human be capable of simultaneous perception/interpretation and production of communicative behavior. A Virtual Human should be able to signal its attitude and attention while it is listening to its interaction partner, and be able to attend to its interaction partner while it is speaking – and modify its communicative behavior on-the-fly based on what it perceives from its partner. This report presents the results of a four week summer project that was part of eNTERFACE’10. The project resulted in progress on several aspects of continuous interaction such as scheduling and interrupting multimodal behavior, automatic classification of listener responses, generation of response eliciting behavior, and models for appropriate reactions to listener responses. A pilot user study was conducted with ten participants. In addition, the project yielded a number of deliverables that are released for public access

Generic dialogue modeling for multi-application dialogue systems

We present a novel approach to developing interfaces for multi-application dialogue systems. The targeted interfaces allow transparent switching between a large number of applications within one system. The approach, based on the Rapid Dialogue Prototyping Methodology (RDPM) and the Vector Space model techniques from Information Retrieval, is composed of three main steps: (1) producing finalized dia logue models for applications using the RDPM, (2) designing an application interaction hierarchy, and (3) navigating between the applications based on the user's application of interest

Compost: a server for multilingual text-to-speech synthesis

This article presents a server which offers multilingual text-to-speech synthesis to connected clients . It describes the COMPOST environment which bas been created to give a complete answer ta this challenging idea : voice output will be fully integrated in man-machine dialog systems when hardware and software architectures will offer flexible and powerfull communication tools between text-to-speech systems and dialog systems . COMPOST consists of a specialised programing language for text-tospeech synthesis. Each text-to-speech system describes the different steps which convert a running text towards its acoustic andlor visual synthesis by means of a main program called a scenario . Each scenario is then compiled and may be downloaded into the working environment of the server . A normalised communication interface enables a real-time dialog between the server and the attached clients i.e . for using a tree-structure as input (synthesis by concepts), changing the voice quality, the language or the spelling mode . The ideas presented in this article are enlighted by concrete examples extracted from a text-to-speech system for French developped using COMPOST.Cet article présente la notion de serveur de synthèse et décrit l'environnement COMPOST qui a été créé pour répondre aux concepts que cette notion développe: l'intégration du dialogue vocal dans les interfaces homme-machine ne sera opérationnel que lorsque les architectures matérielles et logicielles offriront aux concepteurs de systèmes de synthèse et aux spécialistes du génie logiciel un moyen souple de communication. COMPOST propose aux premiers un langage de programmation de systèmes de synthèse multilingues et multi-entrées et aux derniers une infrastructure logicielle basée sur la notion de serveur-clien

Vocal Interactivity in-and-between Humans, Animals, and Robots

Almost all animals exploit vocal signals for a range of ecologically motivated purposes: detecting predators/prey and marking territory, expressing emotions, establishing social relations, and sharing information. Whether it is a bird raising an alarm, a whale calling to potential partners, a dog responding to human commands, a parent reading a story with a child, or a business-person accessing stock prices using Siri, vocalization provides a valuable communication channel through which behavior may be coordinated and controlled, and information may be distributed and acquired. Indeed, the ubiquity of vocal interaction has led to research across an extremely diverse array of fields, from assessing animal welfare, to understanding the precursors of human language, to developing voice-based human–machine interaction. Opportunities for cross-fertilization between these fields abound; for example, using artificial cognitive agents to investigate contemporary theories of language grounding, using machine learning to analyze different habitats or adding vocal expressivity to the next generation of language-enabled autonomous social agents. However, much of the research is conducted within well-defined disciplinary boundaries, and many fundamental issues remain. This paper attempts to redress the balance by presenting a comparative review of vocal interaction within-and-between humans, animals, and artificial agents (such as robots), and it identifies a rich set of open research questions that may benefit from an interdisciplinary analysis

A Conversational Academic Assistant for the Interaction in Virtual Worlds

Proceedings of: Forth International Workshop on User-Centric Technologies and applications (CONTEXTS 2010). Valencia, 07-10 September , 2010.The current interest and extension of social networking are rapidly introducing a large number of applications that originate new communication and interaction forms among their users. Social networks and virtual worlds, thus represent a perfect environment for interacting with applications that use multimodal information and are able to adapt to the specific characteristics and preferences of each user. As an example of this application, in this paper we present an example of the integration of conversational agents in social networks, describing the development of a conversational avatar that provides academic information in the virtual world of Second Life. For its implementation techniques from Speech Technologies and Natural Language Processing have been used to allow a more natural interaction with the system using voice.Funded by projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, SINPROB, CAM MADRINET S-0505/TIC/0255, and DPS2008-07029-C02-02.Publicad