Personalizing Task-oriented Dialog Systems via Zero-shot Generalizable Reward Function

Task-oriented dialog systems enable users to accomplish tasks using natural language. State-of-the-art systems respond to users in the same way regardless of their personalities, although personalizing dialogues can lead to higher levels of adoption and better user experiences. Building personalized dialog systems is an important, yet challenging endeavor and only a handful of works took on the challenge. Most existing works rely on supervised learning approaches and require laborious and expensive labeled training data for each user profile. Additionally, collecting and labeling data for each user profile is virtually impossible. In this work, we propose a novel framework, P-ToD, to personalize task-oriented dialog systems capable of adapting to a wide range of user profiles in an unsupervised fashion using a zero-shot generalizable reward function. P-ToD uses a pre-trained GPT-2 as a backbone model and works in three phases. Phase one performs task-specific training. Phase two kicks off unsupervised personalization by leveraging the proximal policy optimization algorithm that performs policy gradients guided by the zero-shot generalizable reward function. Our novel reward function can quantify the quality of the generated responses even for unseen profiles. The optional final phase fine-tunes the personalized model using a few labeled training examples. We conduct extensive experimental analysis using the personalized bAbI dialogue benchmark for five tasks and up to 180 diverse user profiles. The experimental results demonstrate that P-ToD, even when it had access to zero labeled examples, outperforms state-of-the-art supervised personalization models and achieves competitive performance on BLEU and ROUGE metrics when compared to a strong fully-supervised GPT-2 baselineComment: 11 pages, 4 tables, 31st ACM International Conference on Information and Knowledge Management (CIKM'22

Context-awareness for adaptive information retrieval systems

Philosophiae Doctor - PhDThis research study investigates optimization of IRS to individual information needs in order of relevance. The research addressed development of algorithms that optimize the ranking of documents retrieved from IRS. In this thesis, we present two aspects of context-awareness in IR. Firstly, the design of context of information. The context of a query determines retrieved information relevance. Thus, executing the same query in diverse contexts often leads to diverse result rankings. Secondly, the relevant context aspects should be incorporated in a way that supports the knowledge domain representing users’ interests. In this thesis, the use of evolutionary algorithms is incorporated to improve the effectiveness of IRS. A context-based information retrieval system is developed whose retrieval effectiveness is evaluated using precision and recall metrics. The results demonstrate how to use attributes from user interaction behaviour to improve the IR effectivenes

Supporting the discoverability of open educational resources

Open Educational Resources (OERs), now available in large numbers, have a considerable potential to improve many aspects of society, yet one of the factors limiting this positive impact is the difficulty to discover them. This study investigates and proposes strategies to better support educators in discovering OERs, mainly focusing on secondary education. The literature suggests that the effectiveness of existing search systems could be improved by supporting high-level and domain-oriented tasks. Hence a preliminary taxonomy of discovery-related tasks was developed, based on the analysis of the literature, interpreted through Information Foraging Theory. This taxonomy was empirically evaluated with a few experienced educators, to preliminary identify an interesting class of Query By Examples (QBE) expansion by similarity tasks, which avoids the need to decompose natural high-level tasks in a complex sequence of sub-tasks. Following the Design Science Research methodology, three prototypes to support as well as to refine those tasks were iteratively designed, implemented, and evaluated involving an increasing number of educators in usability oriented studies. The resulting high-level and domain-oriented blended search/recommendation strategy, transparently replicates Google searches in specialized networks, and identifies similar resources with a QBE strategy. It makes use of a domain-oriented similarity metric based on shared schema.org/LRMI alignments to educational frameworks, and clusters results in expandable classes of comparable degree of similarity. The summative evaluation shows that educators appreciate this exploratory-oriented strategy because – balancing similarity and diversity – it supports their high-level tasks, such as lesson planning and personalization of education

Context-aware personalization environment for mobile computing

Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaCurrently, we live in a world where the amount of on-line information vastly outstrips any individual’s capability to survey it. Filtering that information in order to obtain only useful and interesting information is a solution to this problem. The mobile computing area proposes to integrate computation in users’ daily activities in an unobtrusive way, in order to guarantee an improvement in their experience and quality of life. Furthermore, it is crucial to develop smaller and more intelligent devices to achieve this area’s goals, such as mobility and energy savings. This computing area reinforces the necessity to filter information towards personalization due to its humancentred paradigm. In order to attend to this personalization necessity, it is desired to have a solution that is able to learn the users preferences and needs, resulting in the generation of profiles that represent each style of interaction between a user and an application’s resources(e.g. buttons and menus). Those profiles can be obtained by using machine learning algorithms that use data derived from the user interaction with the application, combined with context data and explicit user preferences. This work proposes an environment with a generic context-aware personalization model and a machine learning module. It is provided the possibility to personalize an application, based on user profiles obtained from data, collected from implicit and explicit user interaction. Using a provided personalization API (Application Programming Interface) and other configuration modules, the environment was tested on LEY (Less energy Empowers You), a persuasive mobile-based serious game to help people understand domestic energy usage

A Context-Adaptive Ranking Model for Effective Information Retrieval System

Abstract When using Information Retrieval (IR) systems, users often present search queries made of ad-hoc keywords. It is then up to information retrieval systems (IRS) to obtain a precise representation of user’s information need, and the context of the information. Context-aware ranking techniques have been constantly used over the past years to improve user interaction in their search activities for improved relevance of retrieved documents. Though, there have been major advances in context-adaptive systems, there is still a lack of technique that models and implements context-adaptive application. The paper addresses this problem using DROPT technique. The DROPT technique ranks individual user information needs according to relevance weights. Our proposed predictive document ranking model is computed as measures of individual user search in their domain of knowledge. The context of a query determines retrieved information relevance. Thus, relevant context aspects should be incorporated in a way that supports the knowledge domain representing users’ interests. We demonstrate the ranking task using metric measures and ANOVA, and argue that it can help an IRS adapted to a user's interaction behaviour, using context to improve the IR effectiveness

On Two Web IR Boosting Tools: Clustering and Ranking

This thesis investigates several research problems which arise in modern Web Information Retrieval (WebIR). The Holy Grail of modern WebIR is to find a way to organize and to rank results so that the most ``relevant' come first. The first break-through technique was the exploitation of the link structure of the Web graph in order to rank the result pages, using the well-known Hits and Pagerank algorithms. This link-analysis approaches have been improved and extended, but yet they seem to be insufficient in providing a satisfying search experience. In a number of situations a flat list of search results is not enough, and the users might desire to have search results grouped on-the-fly in folders of similar topics. In addition, the folders should be annotated with meaningful labels for rapid identification of the desired group of results. In other situations, users may have different search goals even when they express them with the same query. In this case the search results should be personalized according to the users' on-line activities. In order to address this need, we will discuss the algorithmic ideas behind SnakeT, a hierarchical clustering meta-search engine which personalizes searches according to the clusters selected by users on-the-fly. There are also situations where users might desire to access fresh information. In these cases, traditional link analysis could not be suitable. In fact, it is possible that there is not enough time to have many links pointing to a recently produced piece of information. In order to address this need, we will discuss the algorithmic and numerical ideas behind a new ranking algorithm suitable for ranking fresh type of information, such as news articles or blogs. When link analysis suffices to produce good quality search results, the huge amount of Web information asks for fast ranking methodologies. We will discuss numerical methodologies for accelerating the eingenvector-like computation, commonly used by link analysis. An important result of this thesis is that we show how to address the above predominant issues of Web Information Retrieval by using clustering and ranking methodologies. We will demonstrate that both clustering and ranking have a mutual reinforcement propriety which has not yet been studied intensively. This propriety can be exploited to boost the precision of both the two methodologies

MARLUI: Multi-Agent Reinforcement Learning for Adaptive UIs

Adaptive user interfaces (UIs) automatically change an interface to better support users' tasks. Recently, machine learning techniques have enabled the transition to more powerful and complex adaptive UIs. However, a core challenge for adaptive user interfaces is the reliance on high-quality user data that has to be collected offline for each task. We formulate UI adaptation as a multi-agent reinforcement learning problem to overcome this challenge. In our formulation, a user agent mimics a real user and learns to interact with a UI. Simultaneously, an interface agent learns UI adaptations to maximize the user agent's performance. The interface agent learns the task structure from the user agent's behavior and, based on that, can support the user agent in completing its task. Our method produces adaptation policies that are learned in simulation only and, therefore, does not need real user data. Our experiments show that learned policies generalize to real users and achieve on par performance with data-driven supervised learning baselines