User-Aware Dialogue Management Policies over Attributed Bi-Automata

Designing dialogue policies that take user behavior into account is complicated due to user vari- ability and behavioral uncertainty. Attributed Prob- abilistic Finite State Bi-Automata (A-PFSBA) have proven to be a promising framework to develop dia- logue managers that capture the users’ actions in its structure and adapt to them online, yet developing poli- cies robust to high user uncertainty is still challenging. In this paper, the theoretical A-PFSBA dialogue man- agement framework is augmented by formally defining the notation of exploitation policies over its structure. Under such definition, multiple path based policies are implemented, those that take into account external in- formation and those which do not. These policies are evaluated on the Let’s Go corpus, before and after an online learning process whose goal is to update the ini- tial model through the interaction with end-users. In these experiments the impact of user uncertainty and the model structural learning is thoroughly analyzedSpanish Minister of Science under grants TIN2014-54288-C4- 4-R and TIN2017-85854-C4-3-R European Commission H2020 SC1-PM15 EMPATHIC project, RIA grant 69872

Audio Embedding-Aware Dialogue Policy Learning

Following the success of Natural Language Processing (NLP) transformers pretrained via self-supervised learning, similar models have been proposed recently for speech processing such as Wav2Vec2, HuBERT and UniSpeech-SAT. An interesting yet unexplored area of application of these models is Spoken Dialogue Systems, where the users’ audio signals are typically just mapped to word-level features derived from an Automatic Speech Recogniser (ASR), and then processed using NLP techniques to generate system responses. This paper reports a comprehensive comparison of dialogue policies trained using ASR-based transcriptions and extended with the aforementioned audio processing transformers in the DSTC2 task. Whilst our dialogue policies are trained with supervised and policy-based deep reinforcement learning, they are assessed using both automatic task completion metrics and a human evaluation. Our results reveal that using audio embeddings is more beneficial than detrimental in most of our trained dialogue policies, and that the benefits are stronger for supervised learning than reinforcement learning

Towards structured neural spoken dialogue modelling.

195 p.In this thesis, we try to alleviate some of the weaknesses of the current approaches to dialogue modelling,one of the most challenging areas of Artificial Intelligence. We target three different types of dialogues(open-domain, task-oriented and coaching sessions), and use mainly machine learning algorithms to traindialogue models. One challenge of open-domain chatbots is their lack of response variety, which can betackled using Generative Adversarial Networks (GANs). We present two methodological contributions inthis regard. On the one hand, we develop a method to circumvent the non-differentiability of textprocessingGANs. On the other hand, we extend the conventional task of discriminators, which oftenoperate at a single response level, to the batch level. Meanwhile, two crucial aspects of task-orientedsystems are their understanding capabilities because they need to correctly interpret what the user islooking for and their constraints), and the dialogue strategy. We propose a simple yet powerful way toimprove spoken understanding and adapt the dialogue strategy by explicitly processing the user's speechsignal through audio-processing transformer neural networks. Finally, coaching dialogues shareproperties of open-domain and task-oriented dialogues. They are somehow task-oriented but, there is norush to complete the task, and it is more important to calmly converse to make the users aware of theirown problems. In this context, we describe our collaboration in the EMPATHIC project, where a VirtualCoach capable of carrying out coaching dialogues about nutrition was built, using a modular SpokenDialogue System. Second, we model such dialogues with an end-to-end system based on TransferLearning

Dialogue Management and Language Generation for a Robust Conversational Virtual Coach: Validation and User Study

Designing human–machine interactive systems requires cooperation between different disciplines is required. In this work, we present a Dialogue Manager and a Language Generator that are the core modules of a Voice-based Spoken Dialogue System (SDS) capable of carrying out challenging, long and complex coaching conversations. We also develop an efficient integration procedure of the whole system that will act as an intelligent and robust Virtual Coach. The coaching task significantly differs from the classical applications of SDSs, resulting in a much higher degree of complexity and difficulty. The Virtual Coach has been successfully tested and validated in a user study with independent elderly, in three different countries with three different languages and cultures: Spain, France and Norway.The research presented in this paper has been conducted as part of the project EMPATHIC that has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant No. 769872. Additionally, this work has been partially funded by projects BEWORD and AMIC-PC of the Minister of Science of Technology, under Grant Nos. PID2021-126061OB-C42 and PDC2021-120846-C43, respectively. Vázquez and López Zorrilla received a PhD scholarship from the Basque Government, with Grant Nos. PRE 2020 1 0274 and PRE 2017 1 0357, respectively

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

A multilingual neural coaching model with enhanced long-term dialogue structure

In this work we develop a fully data-driven conversational agent capable of carrying out motivational coach- ing sessions in Spanish, French, Norwegian, and English. Unlike the majority of coaching, and in general well-being related conversational agents that can be found in the literature, ours is not designed by hand- crafted rules. Instead, we directly model the coaching strategy of professionals with end users. To this end, we gather a set of virtual coaching sessions through a Wizard of Oz platform, and apply state of the art Natural Language Processing techniques. We employ a transfer learning approach, pretraining GPT2 neural language models and fine-tuning them on our corpus. However, since these only take as input a local dialogue history, a simple fine-tuning procedure is not capable of modeling the long-term dialogue strategies that appear in coaching sessions. To alleviate this issue, we first propose to learn dialogue phase and scenario embeddings in the fine-tuning stage. These indicate to the model at which part of the dialogue it is and which kind of coaching session it is carrying out. Second, we develop a global deep learning system which controls the long-term structure of the dialogue. We also show that this global module can be used to visualize and interpret the decisions taken by the the conversational agent, and that the learnt representations are comparable to dialogue acts. Automatic and human evaluation show that our proposals serve to improve the baseline models. Finally, interaction experiments with coaching experts indicate that the system is usable and gives rise to positive emotions in Spanish, French and English, while the results in Norwegian point out that there is still work to be done in fully data driven approaches with very low resource languages.This work has been partially funded by the Basque Government under grant PRE_2017_1_0357 and by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 769872

Advancing PSS with complex urban systems sciences and scalable spatio-temporal models

Planning Support System (PSS) with a core of dynamic spatio-temporal model has been developed as analytical and information tools to aid and inform urban planning processes. However, scholarly communities identify that PSS has yet been popularized in planning practices, and not fully capable of meeting the challenge of understanding complex urban environments. I am dedicated to investigate and break through the bottlenecks of PSS with my experiences with University of Illinois Landuse Evolution and Impact Assessment Model (LEAM) PSS, which exemplify a PSS that that aid the process of collaboratively building spatio-temporal scenario models and transferring the knowledge to planning practitioners. I explore the future applications of PSS including Smart Cities, sentience, resilience, and environmental planning processes and their role in improving PSS usefulness in the practice of planning. PSS improvements will be presented in terms of multi-directional spatio-temporal processes and scenario planning. Moreover, I will address the process of transferring knowledge to users on model validity and ‘goodness-of-fit’ in real world planning applications. Beyond the traditional theoretical framework of PSS, the emerging Complex Urban System Sciences (CUS) challenge the core assumptions of spatial models of PSS, and pose opportunities for updating current PSS approaches into scalable spatio-temporal model that adheres to CUS principles. I will analyze this potential infusion by examining next generation PSSs within a framework of current CUS theories and advancement in statistical and computational methods. Case studies involved in my dissertation include LEAM PSS’ applications in McHenry County (IL), Peoria (IL), Chicago (IL), and St. Louis (MO). The final part of this dissertation highlights my contributions to the existing CUS theories. I will demonstrates how evidence from empirical applications can contribute to CUS theory itself. I will show how CUS can challenge the core assumptions of “distance to CBD” models that economists use to characterize urban structure and land-use

Interim research assessment 2003-2005 - Computer Science

This report primarily serves as a source of information for the 2007 Interim Research Assessment Committee for Computer Science at the three technical universities in the Netherlands. The report also provides information for others interested in our research activities