Optimizing Dialog Strategies for Conversational Agents Interacting in AmI Environments

Proceedings of: 3rd International Symposium on Ambient Intelligence (ISAmI 2012). Salamanca (Spain), 28-30 March 2012In this paper, we describe a conversational agent which provides academic information. The dialog model of this agent has been developed by means of a statistical methodology that automatically explores the dialog space and allows learning new enhanced dialog strategies from a dialog corpus. A dialog simulation technique has been applied to acquire data required to train the dialog model and then explore the new dialog strategies. A set of measures has also been defined to evaluate the dialog strategy. The results of the evaluation show how the dialogmodel deviates from the initially predefined strategy, allowing the conversational agent to tackle new situations and generate new coherent answers for the situations already present in the initial corpus. The proposed technique can be used not only to develop new dialog managers but also to explore new enhanced dialog strategies focused on user adaptation required to interact in AmI environments.Research funded by projects CICYT TIN2011-28620-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485), and DPS2008-07029-C02-02.Publicad

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

Multimodal systems have attained increased attention in recent years, which has made possible important improvements in the technologies for recognition, processing, and generation of multimodal information. However, there are still many issues related to multimodality which are not clear, for example, the principles that make it possible to resemble human-human multimodal communication. This chapter focuses on some of the most important challenges that researchers have recently envisioned for future multimodal interfaces. It also describes current efforts to develop intelligent, adaptive, proactive, portable and affective multimodal interfaces

A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective

Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

Intelligent techniques for context-aware systems

Nowadays, with advances in communication technologies, researches are focused in the fields of designing new devices with increasing capabilities, implanting software frameworks or middleware to make these devices interoperable. Building better human interfaces is a challenging task and the adoption of Artificial Intelligence (AI) techniques to the process help associating semantic meaning to devices which makes possible the gesture recognition and voice recognition. This thesis is mainly concerned with the open problem in context-aware systems: the evaluation of these systems in Ambient Intelligence (AmI) environments. With regard to this issue, we argue that due to highly dynamic properties of the AmI environments, it should exist a methodology for evaluating these systems taking into account the type of scenarios. However in order to support with a solid ground for that discussion, some elements are to be discussed as well. In particular, we: • use a commercial platform that allows us to design and manage the contextual information of context- aware systems by means of a context manager included in the architecture; • analyze the formal representation of this contextual information by means of a knowledge based system (KBS); • discuss the possible methodologies to be used for modelling knowledge in KBS and our approach; • give reasons why intelligent agents is a valid technique to be applied to systems in AmI environments; • propose a generic multi-agent system (MAS) architecture that can be applied to a large class of envisaged AmI applications; • propose a multimodal user interface and its integration with our MAS; • propose an evaluation methodology for context-aware systems in AmI scenarios. The formulation of the above mentioned elements became necessary as this thesis was developed. The lack of an evaluation methodology for context-aware systems in AmI environments, where so many issues to be covered, took us to the main objective of this thesis. In this regard: • we provide an updated and exhaustive state-of-the-art of this matter; • examine the properties and characteristics of AmI scenarios; • put forward an evaluation methodology and experimentally test our methodology in AmI scenarios. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------La Inteligencia Ambiental y los entornos inteligentes hacen hincapié en una mayor facilidad de uso, soporte de servicios más eficientes, el apoderamiento de los usuarios, y el apoyo a las interacciones humanas. En esta visión, las personas estarán rodeadas de interfaces inteligentes e intuitivas incrustados en objetos cotidianos que nos rodean y los sistemas desarrollados para este ambiente deberán reconocer y responder a la presencia de individuos de una manera invisible y transparente a ellos. Esta tesis se centra principalmente en el problema abierto en los sistemas sensibles al contexto: la evaluación de estos sistemas en los entornos de Inteligencia Ambiental. Con respecto a este tema, se argumenta que debido a las propiedades altamente dinámica de los entornos de inteligencia ambiental, debería existir una metodología para la evaluación de estos sistemas, teniendo en cuenta el tipo de escenarios. Sin embargo, con el fin de apoyar con una base sólida para la discusión, algunos elementos deben ser discutidos también. En particular, nosotros: • Usamos una plataforma comercial que nos permite diseñar y gestionar la información contextual de los sistemas sensibles al contexto a través de un gestor de contexto incluido en la arquitectura; • Analizamos la representación formal de esta información contextual a través de un sistema basado en el conocimiento (SBC); • Discutimos las posibles metodologías que se utilizarán para el modelado del conocimiento en SBC y nuestra aproximación y propuesta; • Discutimos las razones del por qué los agentes inteligentes son una técnica válida para ser aplicada a los sistemas en entornos inteligencia ambiental; • Proponemos un sistema multi-agente (SMA), con una arquitectura genérica que se puede aplicar a una gran clase de aplicaciones de inteligencia ambiental; • Proponemos una interfaz de usuario multimodales y su integración con nuestro SMA; • Proponemos una metodología de evaluación de los sistemas sensibles al contexto en los escenarios de inteligencia ambiental. La formulación de los elementos antes mencionados se hizo necesaria en la medida que esta tesis se ha desarrollado. La falta de una metodología de evaluación de los sistemas sensibles al contexto en entornos de inteligencia ambiental, donde existen tantos temas a tratar, nos llevó al objetivo principal de esta tesis. En este sentido, en esta tesis: • Proporcionamos un estado del arte actualizado y exhaustivo de este asunto; • Examinamos las propiedades y características de los escenarios de inteligencia ambiental; • Proponemos una metodología de evaluación para este tipo de sistemas y experimentalmente probamos nuestra metodología en diversos escenarios de inteligencia ambiental

Four Mode Based Dialogue Management with Modified POMDP Model

This thesis proposes a method to manage the interaction between the user and the system dynamically, through speech or text input which updates the user goals, select system actions and calculate rewards for each system response at each time-stamp. The main focus is made on the dialog manager, which decides how to continue the dialogue. We have used POMDP technique, as it maintains a belief distribution on the dialogue states based on the observations over the dialogue even in a noisy environment. Four contextual control modes are introduced in dialogue management for decision-making mechanism, and to keep track of machine behaviour for each dialogue state. The result obtained proves that our proposed framework has overcome the limitations of prior POMDP methods, and exactly understands the actual intention of the users within the available time, providing very interactive conversation between the user and the computer

Cognitive architecture of multimodal multidimensional dialogue management

Numerous studies show that participants of real-life dialogues happen to get involved in rather dynamic non-sequential interactions. This challenges the dialogue system designs based on a reactive interlocutor paradigm and calls for dialog systems that can be characterised as a proactive learner, accomplished multitasking planner and adaptive decision maker. Addressing this call, the thesis brings innovative integration of cognitive models into the human-computer dialogue systems. This work utilises recent advances in Instance-Based Learning of Theory of Mind skills and the established Cognitive Task Analysis and ACT-R models. Cognitive Task Agents, producing detailed simulation of human learning, prediction, adaption and decision making, are integrated in the multi-agent Dialogue Man-ager. The manager operates on the multidimensional information state enriched with representations based on domain- and modality-specific semantics and performs context-driven dialogue acts interpretation and generation. The flexible technical framework for modular distributed dialogue system integration is designed and tested. The implemented multitasking Interactive Cognitive Tutor is evaluated as showing human-like proactive and adaptive behaviour in setting goals, choosing appropriate strategies and monitoring processes across contexts, and encouraging the user exhibit similar metacognitive competences