On the dynamic adaptation of language models based on dialogue information

We present an approach to adapt dynamically the language models (LMs) used by a speech recognizer that is part of a spoken dialogue system. We have developed a grammar generation strategy that automatically adapts the LMs using the semantic information that the user provides (represented as dialogue concepts), together with the information regarding the intentions of the speaker (inferred by the dialogue manager, and represented as dialogue goals). We carry out the adaptation as a linear interpolation between a background LM, and one or more of the LMs associated to the dialogue elements (concepts or goals) addressed by the user. The interpolation weights between those models are automatically estimated on each dialogue turn, using measures such as the posterior probabilities of concepts and goals, estimated as part of the inference procedure to determine the actions to be carried out. We propose two approaches to handle the LMs related to concepts and goals. Whereas in the first one we estimate a LM for each one of them, in the second one we apply several clustering strategies to group together those elements that share some common properties, and estimate a LM for each cluster. Our evaluation shows how the system can estimate a dynamic model adapted to each dialogue turn, which helps to improve the performance of the speech recognition (up to a 14.82% of relative improvement), which leads to an improvement in both the language understanding and the dialogue management tasks

Technology assessment of advanced automation for space missions

Six general classes of technology requirements derived during the mission definition phase of the study were identified as having maximum importance and urgency, including autonomous world model based information systems, learning and hypothesis formation, natural language and other man-machine communication, space manufacturing, teleoperators and robot systems, and computer science and technology

CHORUS Deliverable 2.2: Second report - identification of multi-disciplinary key issues for gap analysis toward EU multimedia search engines roadmap

After addressing the state-of-the-art during the first year of Chorus and establishing the existing landscape in multimedia search engines, we have identified and analyzed gaps within European research effort during our second year. In this period we focused on three directions, notably technological issues, user-centred issues and use-cases and socio- economic and legal aspects. These were assessed by two central studies: firstly, a concerted vision of functional breakdown of generic multimedia search engine, and secondly, a representative use-cases descriptions with the related discussion on requirement for technological challenges. Both studies have been carried out in cooperation and consultation with the community at large through EC concertation meetings (multimedia search engines cluster), several meetings with our Think-Tank, presentations in international conferences, and surveys addressed to EU projects coordinators as well as National initiatives coordinators. Based on the obtained feedback we identified two types of gaps, namely core technological gaps that involve research challenges, and “enablers”, which are not necessarily technical research challenges, but have impact on innovation progress. New socio-economic trends are presented as well as emerging legal challenges

Communicating with Machines: Conversational Agents with Personality and the Role of Extraversion

Communication with conversational agents (CA) has become increasingly important. It therefore is crucial to understand how individuals perceive interaction with CAs and how the personality of both the CA and the human can affect the interaction experience. As personality differences are manifested in language cues, we investigate whether different language style manifestations of extraversion lead to a more anthropomorphized perception (specifically perceived humanness and social presence) of the personality bots. We examine, whether individuals rate communication satisfaction of a CA similar to their own personality as higher (law of attraction). The results of our experiment indicate that highly extraverted CAs are generally better received in terms of social presence and communication satisfaction. Further, incorporating personality into CAs increases perceived humanness. Although no significant effects could be found in regard to the law of attraction, interesting findings about ambiverts could be made. The outcomes of the experiment contribute towards designing personality-adaptive CAs

Personal storytelling: Using Natural Language Generation for children with complex communication needs, in the wild...

This paper describes a Natural Language Generation system (NLG), How was School Today? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibility study of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice recording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for NLG in augmented and alternative communication are also discussed