Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

A Multi-Agent System Simulation Model for Trusted Local Energy Markets

The energy market and electric grid play a major role in everyday life. Most areas in modern society, such as: communication, health, transportation, the financial system and many others; require electrical energy to operate properly. Traditionally energy grids operate in a centralized manner. Consumers are connected to centralized utilities in the grid and energy flows from producers to Consumers. However, the rising in popularity in Renewable Energy Sources (RES) such as photovoltaic panels installed in households, small commerce and small industry wide spread the use of distributed energy generation, which the main energy grid was not designed to support. One of the possible solutions for this problem is the creation of a Local Energy Market (LeM). A LeM is a market that operates in a small physical area such as a neighborhood. Traditional consumers can become active market participants under a LeM. That is possible because the LeM is structured in such a way as to enable small-scale negotiations and energy exchanges between participants, who traditionally would only be final consumers. The LeM is capable of dealing with distributed energy generation from RES because negotiations and distribution happen at a local level, thus reducing problems with the main grid. Furthermore, the participation in the local market can reduce energy costs or even create profits for consumers, while contributing to easy the management of the grid and associated technical losses. This work explores the concept of LeM and is focused on two main objectives: designing and developing a system that allows the simulation of LeM, and designing and developing a mechanism that allows trusted negotiations in this market. To accomplish these objectives a Multi-Agent System (MAS) architecture is proposed to model and allow the simulation of LeM. Furthermore to support the market it is also proposed a trust model used to evaluate the behavior of participants and detecting faulty or malicious activities. The developed MAS models a LeM based on a Smart Grid, that is an energy grid with a cyber-physical system with smart meters and communications mechanisms. The MAS was developed with agents to model sensors, market participants and a Market Interaction Manager (MIM) agent that is responsible for managing the negotiations and for applying trust mechanisms. The trust mechanism was designed to attribute a dynamic trust value to each participant, which is reviewed during the all negotiation period. This evaluation of the participant’s trust is based on the analysis of historical data, contextual data, such as weather conditions, and by using forecasting methods to predict the participant expected behavior, allowing to penalize the ones that are exhibiting a questionable behavior in the market. A case study simulation was made with the objective of understanding how the proposed trust mechanism performed, and how the use of different forecasting methods can interfere with it. The results obtained allowed us to conclude that the trust methodology is able to update the trust of each participant, during the negotiation period, and when paired with a well performing forecasting mechanism it is able to achieve a trusted evaluation of the participants behavior. Taking into consideration these results we believe that the proposed trust methodology is capable of providing a valuable trust assessment when used by the MIM agent. This Master Thesis is developed within the scope of a project called Secure interactions and trusted Participation in local Electricity Trading (SPET), a FCT-SAICT2017 funded Research & Development project. SPET project envisions the development of a MAS that is designed to model and simulate the operations of a LeM, taking a focus on security and market trust necessary in this negotiation environment.O mercado de energia e a rede elétrica desempenham um papel importante na vida quotidiana da população. Grande parte das áreas da sociedade moderna, como é o caso da comunicação, transportes, saúde, sistema financeiro, entre outras; requer energia elétrica para funcionar corretamente. Tradicionalmente, as redes de energia operam de forma centralizada. Os consumidores estão conectados a fornecedores centralizados na rede e a energia é transferida dos produtores para os consumidores. No entanto, o aumento da popularidade das Fontes de Energia Renováveis (FER), como painéis fotovoltaicos instalados nas residências, pequeno comércio e pequena indústria, difundiu o uso da geração distribuída de energia, que a rede principal de energia não foi projetada para suportar. Uma das possíveis soluções para esse problema é a criação de um Mercado Local de Energia (MLe). Um MLe é um mercado que opera numa pequena área física, como uma vizinhança. Num MLe, os consumidores tradicionais têm a possibilidade de ser participantes ativos no mercado. Isto é possível porque o MLe está estruturado de forma a permitir negociações em pequena escala e trocas de energia entre os participantes, que tradicionalmente seriam apenas consumidores finais. O MLe é capaz de lidar com a geração de energia distribuída proveniente das FER, porque as negociações e a distribuição ocorrem a um nível local, reduzindo assim os problemas com a rede principal. Para além disso, a participação no mercado local pode reduzir os custos de energia ou até gerar lucros para os consumidores, contribuindo ainda para facilitar a gestão da rede e reduzir as perdas técnicas a ela associadas. Este trabalho explora o conceito de MLe e está focado em dois objetivos principais: projetar e desenvolver um sistema que permita a simulação de MLe, bem como um mecanismo que permita negociações confiáveis neste mercado. Para atingir estes objetivos, é proposta uma arquitetura de Sistema Multi-Agente (SMA) para modelar e permitir a simulação do MLe. Para além disso, para apoiar o mercado, também é proposto um modelo de confiança utilizado para avaliar o comportamento dos participantes e detetar falhas ou atividades maliciosas. O SMA desenvolvido modela um MLe com base numa Smart Grid, que é uma rede de energia com um sistema ciber-físico, com sensores inteligentes e mecanismos de comunicação. O SMA foi desenvolvido com agentes para modelar sensores, participantes do mercado e um agente Market Interaction Manager (MIM), responsável pela gestão das negociações e pela aplicação de mecanismos de confiança. O mecanismo de confiança foi projetado para atribuir um valor de confiança dinâmico a cada participante, que é adaptado durante todo o período de negociação. Essa avaliação da confiança do participante é baseada na análise de dados históricos, contextuais, como condições climatéricas, e no uso de métodos de previsão para antever o comportamento esperado do participante, permitindo penalizar aqueles que exibem um comportamento questionável no mercado. Foi realizada uma simulação de caso de estudo, com o objetivo de avaliar o desempenho do mecanismo de confiança proposto e de que forma é que o uso de diferentes métodos de previsão interfere neste desempenho. Os resultados obtidos permitiram concluir que a metodologia de confiança é capaz de atualizar a confiança de cada participante, durante o período de negociação e, quando combinada com um mecanismo de previsão com bom desempenho, é capaz de obter uma avaliação confiável do comportamento dos participantes. Tendo em consideração estes resultados, acreditamos que a metodologia de confiança proposta é capaz de fornecer uma avaliação de confiança valiosa quando usada pelo agente MIM. Esta tese de mestrado é desenvolvida no âmbito de um projeto chamado Secure interactions and trusted Participation in local Electricity Trading (SPET), um projeto de Investigação e Desenvolvimento (I&D) financiado pela FCT-SAICT2017. O projeto SPET tem como objetivo o desenvolvimento de um MAS para a modelação e simulação de MLe, tendo como foco a segurança e confiança necessárias neste ambiente de negociação

Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey

Dialogue-based human-AI collaboration can revolutionize collaborative problem-solving, creative exploration, and social support. To realize this goal, the development of automated agents proficient in skills such as negotiating, following instructions, establishing common ground, and progressing shared tasks is essential. This survey begins by reviewing the evolution of dialogue management paradigms in collaborative dialogue systems, from traditional handcrafted and information-state based methods to AI planning-inspired approaches. It then shifts focus to contemporary data-driven dialogue management techniques, which seek to transfer deep learning successes from form-filling and open-domain settings to collaborative contexts. The paper proceeds to analyze a selected set of recent works that apply neural approaches to collaborative dialogue management, spotlighting prevailing trends in the field. This survey hopes to provide foundational background for future advancements in collaborative dialogue management, particularly as the dialogue systems community continues to embrace the potential of large language models

Reinforcement learning for trading dialogue agents in non-cooperative negotiations

Recent advances in automating Dialogue Management have been mainly made in cooperative environments -where the dialogue system tries to help a human to meet their goals. In non-cooperative environments though, such as competitive trading, there is still much work to be done. The complexity of such an environment rises as there is usually imperfect information about the interlocutors’ goals and states. The thesis shows that non-cooperative dialogue agents are capable of learning how to successfully negotiate in a variety of trading-game settings, using Reinforcement Learning, and results are presented from testing the trained dialogue policies with humans. The agents learned when and how to manipulate using dialogue, how to judge the decisions of their rivals, how much information they should expose, as well as how to effectively map the adversarial needs in order to predict and exploit their actions. Initially the environment was a two-player trading game (“Taikun”). The agent learned how to use explicit linguistic manipulation, even with risks of exposure (detection) where severe penalties apply. A more complex opponent model for adversaries was also implemented, where we modelled all trading dialogue moves as implicitly manipulating the adversary’s opponent model, and we worked in a more complex game (“Catan”). In that multi-agent environment we show that agents can learn to be legitimately persuasive or deceitful. Agents which learned how to manipulate opponents using dialogue are more successful than ones which do not manipulate. We also demonstrate that trading dialogues are more successful when the learning agent builds an estimate of the adversarial hidden goals and preferences. Furthermore the thesis shows that policies trained in bilateral negotiations can be very effective in multilateral ones (i.e. the 4-player version of Catan). The findings suggest that it is possible to train non-cooperative dialogue agents which successfully trade using linguistic manipulation. Such non-cooperative agents may have important future applications, such as on automated debating, police investigation, games, and education

Innovation dialogue - Being strategic in the face of complexity - Conference report

The Innovation Dialogue on Being Strategic in the Face of Complexity was held in Wageningen on 31 November and 1 December 2009. The event is part of a growing dialogue in the international development sector about the complexities of social, economic and political change. It builds on two previous events hosted the Innovation Dialogue on Navigating Complexity (May 2009) and the Seminar on Institutions, Theories of Change and Capacity Development (December 2008). Over 120 people attended the event coming from a range of Dutch and international development organizations. The event was aimed at bridging practitioner, policy and academic interests. It brought together people working on sustainable business strategies, social entrepreneurship and international development. Leading thinkers and practitioners offered their insights on what it means to "be strategic in complex times". The Dialogue was organized and hosted by the Wageningen UR Centre for Development Innovation working with the Chair Groups of Communication & Innovation Studies, Disaster Studies, Education & Competence Studies and Public Administration & Policy as co; organisers. The theme of the Dialogue aligns closely with Wageningen UR’s interest in linking technological and institutional innovation in ways that enable ‘science for impact’

The dragonomic diplomacy (De)code: a study on the causal relationship between Chinese economic diplomacy preference formation and the influence of multilateral economic regimes

Since the reformation of the Chinese economy, two notable trends have developed. First, the growing prominence of multilateral economic regimes (MERs) on the political agenda of Beijing has propelled deepened engagements between Chinese policy actors and institutions, and the agencies of MERs. This development is accompanied by a second trend, which is a growing dynamism in China’s economic diplomacy within the multilateral arenas. This dynamism is reflected in the evolving national preferences and approaches for multilateral economic negotiations, from outright resistance to gradual flexibility, and in some cases, acceptance. The simultaneous and parallel developments of these two trends stem a curiosity on whether a causal relationship exist between the deepened China-MER engagements and the dynamism of China’s economic diplomacy. Has Beijing’s open-door policy to global economic integration opened new windows of opportunity for the MER agencies to influence China’s economic diplomacy and its preference formation? In what way(s) and/or in which capacities can the agencies of MERs assert influence on China’s economic diplomacy preference formation? Under what conditions is this form of external influence successful? What are the long-run implications of the deepened China-MER engagements on Beijing’s economic diplomacy preference formation structure? What does the China-MER relationship tell us about China’s economic diplomacy preference formation in the 21st century? Although China’s partake in the international political economy has received much scholarly attention, few studies have attempted to decode China’s economic diplomacy preference formation, and even fewer have investigated the important nexus between the China-MER relationship and the behaviours of Chinese economic diplomacy. This thesis is a response to the knowledge deficit in these regards. By examining China’s participation in the multilateral climate change, and trade 4 negotiations, the thesis addresses the primary research question, how do multilateral economic regimes and their agencies influence China’s economic diplomacy preference formation? The study finds that the MER agencies do affect Chinese economic diplomacy preference formation. However, their influence peaks at an absorption level whereby Chinese preferences adapt to external preferences but not to the extent of reforming traditional principles and beliefs. The comparatively more effective ways of asserting influence for the MER agencies is through a costs-and-benefits calculus, information dissemination, shuttle diplomacy proximity talks, and informal negotiation practices. In general, Chinese policy actors do not refute the influence of the MER agencies; rather they absorb and adapt to it. In addition, the MER agencies assert influence at different stages of the preference formation, and over time, implicitly establish themselves as integrated policy actors in Beijing. On the whole, this thesis contributes to a deeper understanding about how, why, and when international linkages matter in China’s economic diplomacy, and to the extent of driving preference transformation. The study provides useful analytic lenses that flesh out the variety of functions the MER agencies have in shaping and informing China’s national preferences and negotiation approaches. At the same time, it offers a fuller description of how the Chinese policy actors and institutions respond to (implicit) external interventions in its policy processes. Consequently, this thesis is a significant contribution that adds value to the scholarly debates and knowledge-building about one of the most important political and economic phenomenon of our time

Are Mediators Norm Entrepreneurs? Exploring the Role of Mediators in Norm Diffusion

Mediators are expected to include or uphold a growing number of norms in their interventions. For instance, inclusivity, gender equality, transitional justice, democracy promotion and the implementing instruments that accompany them are increasingly incorporated into the strategies of international and regional organizations, states and non-state actors that mandate mediation missions in conflicts around the world. This working paper takes one step back and asks whether mediators actually can, or have the agency to, promote these norms. It presents the analytical framework of a three-year multi-case research project on the role of mediators in norm diffusion. It examines what norms form part of the framework for mediation processes, if mediators promote these norms and how and what norms are internalized in the peace process. Through process-tracing, the research project will apply this analytical framework to mediation processes in Syria, South Sudan and Myanmar