An Improved Approach of Intention Discovery with Machine Learning for POMDP-based Dialogue Management

An Embodied Conversational Agent (ECA) is an intelligent agent that works as the front end of software applications to interact with users through verbal/nonverbal expressions and to provide online assistance without the limits of time, location, and language. To help to improve the experience of human-computer interaction, there is an increasing need to empower ECA with not only the realistic look of its human counterparts but also a higher level of intelligence. This thesis first highlights the main topics related to the construction of ECA, including different approaches of dialogue management, and then discusses existing techniques of trend analysis for its application in user classification. As a further refinement and enhancement to our prior work on ECA, this thesis research proposes a cohesive framework to integrate emotion-based facial animation with improved intention discovery. In addition, a machine learning technique modelled from Q-learning (Quality-Learning) technique is introduced to support sentiment analysis for the adjustment of policy design in POMDP-based dialogue management. It is anticipated that the proposed research work is going to improve the accuracy of intention discovery while reducing the length of dialogues. Un agent de conversation incorporé (ECA) est un agent intelligent fonctionnant en amont des applications logicielles pour interagir avec les utilisateurs par le biais d\u27expressions verbales / non verbales et pour fournir une assistance en ligne sans limite de temps, de lieu et de langage. Pour aider à améliorer l\u27expérience de l\u27interaction homme-machine, il est de plus en plus nécessaire de doter la CEA de droits non seulement vis-à-vis de ses homologues humains, mais également d\u27un niveau de renseignement supérieur. Cette thèse aborde d’abord les principaux sujets liés à la construction de la CEA, y compris différentes approches de la gestion du dialogue, puis aborde les techniques existantes d’analyse des tendances pour son application à la classification des utilisateurs. Pour affiner et améliorer nos travaux antérieurs sur ECA, cette thèse de recherche propose un cadre cohérent pour intégrer une animation faciale basée sur les émotions avec une découverte de l’intention améliorée. En outre, une technique d\u27apprentissage automatique modélisée à partir de la technique Q-learning (Quality-Learning) est introduite pour prendre en charge l\u27analyse des sentiments afin d\u27ajuster la conception des stratégies dans la gestion du dialogue basée sur POMDP. On s’attend à ce que les travaux de recherche proposés améliorent la précision de la découverte de l’intention tout en réduisant la durée des dialogues

Probabilistic Dialogue Models with Prior Domain Knowledge

Probabilistic models such as Bayesian Networks are now in widespread use in spoken dialogue systems, but their scalability to complex interaction domains remains a challenge. One central limitation is that the state space of such models grows exponentially with the problem size, which makes parameter estimation increasingly difficult, especially for domains where only limited training data is available. In this paper, we show how to capture the underlying structure of a dialogue domain in terms of probabilistic rules operating on the dialogue state. The probabilistic rules are associated with a small, compact set of parameters that can be directly estimated from data. We argue that the introduction of this abstraction mechanism yields probabilistic models that are easier to learn and generalise better than their unstructured counterparts. We empirically demonstrate the benefits of such an approach learning a dialogue policy for a human-robot interaction domain based on a Wizard-of-Oz data set. Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 179–188, Seoul, South Korea, 5-6 July 2012

A statistical simulation technique to develop and evaluate conversational agents

In this paper, we present a technique for developing user simulators which are able to interact and evaluate conversational agents. Our technique is based on a statistical model that is automatically learned from a dialog corpus. This model is used by the user simulator to provide the next answer taking into account the complete history of the interaction. The main objective of our proposal is not only to evaluate the conversational agent, but also to improve this agent by employing the simulated dialogs to learn a better dialog model. We have applied this technique to design and evaluate a conversational agent which provides academic information in a multi-agent system. The results of the evaluation show that the proposed user simulation methodology can be used not only to evaluate conversational agents but also to explore new enhanced dialog strategies, thereby allowing the conversational agent to reduce the time needed to complete the dialogs and automatically detect new valid paths to achieve each of the required objectives defined for the task.This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC 2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485).Publicad

Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook

In recent years, reinforcement learning and bandits have transformed a wide range of real-world applications including healthcare, finance, recommendation systems, robotics, and last but not least, the speech and natural language processing. While most speech and language applications of reinforcement learning algorithms are centered around improving the training of deep neural networks with its flexible optimization properties, there are still many grounds to explore to utilize the benefits of reinforcement learning, such as its reward-driven adaptability, state representations, temporal structures and generalizability. In this survey, we present an overview of recent advancements of reinforcement learning and bandits, and discuss how they can be effectively employed to solve speech and natural language processing problems with models that are adaptive, interactive and scalable.Comment: To appear in Expert Systems with Applications. Accompanying INTERSPEECH 2022 Tutorial on the same topic. Including latest advancements in large language models (LLMs

A proposal for the development of adaptive spoken interfaces to access the Web

Spoken dialog systems have been proposed as a solution to facilitate a more natural human–machine interaction. In this paper, we propose a framework to model the user׳s intention during the dialog and adapt the dialog model dynamically to the user needs and preferences, thus developing more efficient, adapted, and usable spoken dialog systems. Our framework employs statistical models based on neural networks that take into account the history of the dialog up to the current dialog state in order to predict the user׳s intention and the next system response. We describe our proposal and detail its application in the Let׳s Go spoken dialog system.Work partially supported by Projects MINECO TEC2012-37832- C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/ TIC-1485