A communicative robot to learn about us and the world

We describe a model for a robot that learns about the world and her com-panions through natural language communication. The model supports open-domain learning, where the robot has a drive to learn about new con-cepts, new friends, and new properties of friends and concept instances. The robot tries to fill gaps, resolve uncertainties and resolve conflicts. The absorbed knowledge consists of everything people tell her, the situations and objects she perceives and whatever she finds on the web. The results of her interactions and perceptions are kept in an RDF triple store to enable reasoning over her knowledge and experiences. The robot uses a theory of mind to keep track of who said what, when and where. Accumulating knowledge results in complex states to which the robot needs to respond. In this paper, we look into two specific aspects of such complex knowl-edge states: 1) reflecting on the status of the knowledge acquired through a new notion of thoughts and 2) defining the context during which knowl-edge is acquired. Thoughts form the basis for drives on which the robot communicates. We capture episodic contexts to keep instances of objects apart across different locations, which results in differentiating the acquired knowledge over specific encounters. Both aspects make the communica-tion more dynamic and result in more initiatives by the robo

Driving conversations using structured data

We represent dialogue data episodic Knowledge Graphs (eKG), where each incoming interaction is transformed into RDF triples, and the accumulation of conversations over time is stored in a triple store. Each series of interactions produces different eKGs; ergo, we model the dialogue flow as gradual changes to a graph. As each interaction gets incorporated into the eKG, distinct graph patterns arise (referred to as thoughts) used to generate an appropriate response to the incoming information. Choosing the type of thoughts that better guide a conversation and result in a better eKG is a problem that can be modelled as a learning task. We experiment with reinforcement learning, specifically Upper Confidence Bound (UCB), to gradually learn how to improve the state of the eKG. As a reward, we compare different graph semantic and structural measures

Evaluating Agent Interactions Through Episodic Knowledge Graphs

We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent’s behavior

Evaluating agent interactions through episodic knowledge graphs

We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent’s behavior

Evaluating agent interactions through episodic knowledge graphs: (BEST PAPER AWARD)

We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent’s behavior

Evaluating Agent Interactions Through Episodic Knowledge Graphs:(BEST PAPER AWARD)

We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent’s behavior

A communicative robot to learn about us and the world

We describe a model for a robot that learns about the world and her com-panions through natural language communication. The model supports open-domain learning, where the robot has a drive to learn about new con-cepts, new friends, and new properties of friends and concept instances. The robot tries to fill gaps, resolve uncertainties and resolve conflicts. The absorbed knowledge consists of everything people tell her, the situations and objects she perceives and whatever she finds on the web. The results of her interactions and perceptions are kept in an RDF triple store to enable reasoning over her knowledge and experiences. The robot uses a theory of mind to keep track of who said what, when and where. Accumulating knowledge results in complex states to which the robot needs to respond. In this paper, we look into two specific aspects of such complex knowl-edge states: 1) reflecting on the status of the knowledge acquired through a new notion of thoughts and 2) defining the context during which knowl-edge is acquired. Thoughts form the basis for drives on which the robot communicates. We capture episodic contexts to keep instances of objects apart across different locations, which results in differentiating the acquired knowledge over specific encounters. Both aspects make the communica-tion more dynamic and result in more initiatives by the robo