Improving Search through A3C Reinforcement Learning based Conversational Agent

We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent. Our approach caters to subjective search where the user is seeking digital assets such as images which is fundamentally different from the tasks which have objective and limited search modalities. Labeled conversational data is generally not available in such search tasks and training the agent through human interactions can be time consuming. We propose a stochastic virtual user which impersonates a real user and can be used to sample user behavior efficiently to train the agent which accelerates the bootstrapping of the agent. We develop A3C algorithm based context preserving architecture which enables the agent to provide contextual assistance to the user. We compare the A3C agent with Q-learning and evaluate its performance on average rewards and state values it obtains with the virtual user in validation episodes. Our experiments show that the agent learns to achieve higher rewards and better states.Comment: 17 pages, 7 figure

A virtual diary companion

Chatbots and embodied conversational agents show turn based conversation behaviour. In current research we almost always assume that each utterance of a human conversational partner should be followed by an intelligent and/or empathetic reaction of chatbot or embodied agent. They are assumed to be alert, trying to please the user. There are other applications which have not yet received much attention and which require a more patient or relaxed attitude, waiting for the right moment to provide feedback to the human partner. Being able and willing to listen is one of the conditions for being successful. In this paper we have some observations on listening behaviour research and introduce one of our applications, the virtual diary companion

Implementing Social Norms using Policies

Abstract-Multi-agent systems are difficult to develop. One reason for this is that agents are embedded in a society where all agents must agree to obey certain social norms in order for the society to function. Thus, different programmers, writing different agents, must carefully obey certain agreed-upon protocols. This problem is difficult enough due to the complexity of the interactions, but it is exacerbated by the asynchronous and eventbased nature of agent-based systems: agents must asynchronously respond to incoming conversational messages, and may carry on several simultaneous conversations. Several large projects address these issues. Examples are Jade (Telecom Italia) and Cougaar (DARPA). Jade is strictly compliant with the well-known FIPA standard, which makes it useful for commercial agent development and research not directed at certain fundamental aspects of multi-agent systems. Cougaar was developed as a defense agent infrastructure, and while it is not tied to FIPA standards, it is quite prescriptive in both its interagent architecture, and its intra-agent architecture. The contribution of CASA (Collaborative Agent System Architecture) is an agent infrastructure that seeks to support agent development, but as much as possible, avoids restricting the interor intra-agent architecture or the agent interaction paradigm. This paper describes aspects of the CASA tool that mitigate the aforementioned problems for the research-oriented developer who wants to investigate deviations from standards or alternative architectures. CASA provides a policy descriptor language that abstracts the complexities of conversational interactions away from the programming level, and allows sharing of policies among different agents, even at run time. Thus, an agent programmer is free to concentrate on the properties of the agent, and not on the intricate mechanics of conversational protocols. In addition, policies may be easily modified and distributed as the need arises. Thus, a protocol researcher can concentrate on protocols without having to re-write agent behaviour each time the protocol changes. The policy approach is very flexible, and we have developed policies to support the social commitment paradigm, the BDI paradigm, as well as simpler ad-hoc protocols

Software agents in music and sound art research/creative work: Current state and a possible direction

Composers, musicians and computer scientists have begun to use software-based agents to create music and sound art in both linear and non-linear (non-predetermined form and/or content) idioms, with some robust approaches now drawing on various disciplines. This paper surveys recent work: agent technology is first introduced, a theoretical framework for its use in creating music/sound art works put forward, and an overview of common approaches then given. Identifying areas of neglect in recent research, a possible direction for further work is then briefly explored. Finally, a vision for a new hybrid model that integrates non-linear, generative, conversational and affective perspectives on interactivity is proposed

Dialogue based interfaces for universal access.

Conversation provides an excellent means of communication for almost all people. Consequently, a conversational interface is an excellent mechanism for allowing people to interact with systems. Conversational systems are an active research area, but a wide range of systems can be developed with current technology. More sophisticated interfaces can take considerable effort, but simple interfaces can be developed quite rapidly. This paper gives an introduction to the current state of the art of conversational systems and interfaces. It describes a methodology for developing conversational interfaces and gives an example of an interface for a state benefits web site. The paper discusses how this interface could improve access for a wide range of people, and how further development of this interface would allow a larger range of people to use the system and give them more functionality