Motor Resonance as Indicator for Quality of Interaction - Does it Scale to Natural Movements?

This paper is the output of work that is financed by a grant by the Air Force Office for Scientific Research (AFOSR). © 2018 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.Detecting in an automatic manner whether a particular interaction between man and machine “works”, is an unsolved problem in human-machine interaction. No computational technique exists by which the artificial agent could perceive whether the interaction works from the viewpoint of the human or whether interactional breakdown is likely to occur. In human-robot interaction motor resonance has been proposed as a potential candidate for assessing what might be termed “quality of interaction”. Other authors have asserted that “the measure of resonance indicates the extent to which an artificial agent is considered as a social inter-actor” and call it “a plausible foundation for higher-order social cognition”. Motor interference is often used as a metric for resonance. While the above suggests that motor resonance might be suitable as general measure for the potential of an artificial agent to be conceived of as a social entity, the question remains whether it can be used as a measure for the quality of an ongoing interaction

Industrial Symbiotic Networks as Coordinated Games

We present an approach for implementing a specific form of collaborative industrial practices-called Industrial Symbiotic Networks (ISNs)-as MC-Net cooperative games and address the so called ISN implementation problem. This is, the characteristics of ISNs may lead to inapplicability of fair and stable benefit allocation methods even if the collaboration is a collectively desired one. Inspired by realistic ISN scenarios and the literature on normative multi-agent systems, we consider regulations and normative socioeconomic policies as two elements that in combination with ISN games resolve the situation and result in the concept of coordinated ISNs.Comment: 3 pages, Proc. of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018

A Study of AI Population Dynamics with Million-agent Reinforcement Learning

We conduct an empirical study on discovering the ordered collective dynamics obtained by a population of intelligence agents, driven by million-agent reinforcement learning. Our intention is to put intelligent agents into a simulated natural context and verify if the principles developed in the real world could also be used in understanding an artificially-created intelligent population. To achieve this, we simulate a large-scale predator-prey world, where the laws of the world are designed by only the findings or logical equivalence that have been discovered in nature. We endow the agents with the intelligence based on deep reinforcement learning (DRL). In order to scale the population size up to millions agents, a large-scale DRL training platform with redesigned experience buffer is proposed. Our results show that the population dynamics of AI agents, driven only by each agent's individual self-interest, reveals an ordered pattern that is similar to the Lotka-Volterra model studied in population biology. We further discover the emergent behaviors of collective adaptations in studying how the agents' grouping behaviors will change with the environmental resources. Both of the two findings could be explained by the self-organization theory in nature.Comment: Full version of the paper presented at AAMAS 2018 (International Conference on Autonomous Agents and Multiagent Systems

Addressing concept drift in reputation assessment

In this paper, we address the limitations of existing methods to select representative data for trust assessment when agent behaviours can change at varying speeds and times across a system. We propose a method that uses concept drift detection to identify and exclude unrepresentative past experiences, and show that our approach is more robust to dynamic agent behaviours

Distributed strategy adaptation with a prediction function in multi-agent task allocation

Coordinating multiple agents to complete a set of tasks under time constraints is a complex problem. Distributed consensus-based task allocation algorithms address this problem without the need for human supervision. With such algorithms, agents add tasks to their own schedule according to specified allocation strategies. Various factors, such as the available resources and number of tasks, may affect the efficiency of a particular allocation strategy. The novel idea we suggest is that each individual agent can predict locally the best task inclusion strategy, based on the limited task assignment information communicated among networked agents. Using supervised classification learning, a function is trained to predict the most appropriate strategy between two well known insertion heuristics. Using the proposed method, agents are shown to correctly predict and select the optimal insertion heuristic to achieve the overall highest number of task allocations. The adaptive agents consistently match the performances of the best non-adaptive agents across a variety of scenarios. This study aims to demonstrate the possibility and potential performance benefits of giving agents greater decision making capabilities to independently adapt the task allocation process in line with the problem of interest

DOP: Deep Optimistic Planning with Approximate Value Function Evaluation

Research on reinforcement learning has demonstrated promising results in manifold applications and domains. Still, efficiently learning effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. multi-agent systems or hyper-redundant robots). To alleviate this problem, we present DOP, a deep model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) plan effective policies. Specifically, we exploit deep neural networks to learn Q-functions that are used to attack the curse of dimensionality during a Monte-Carlo tree search. Our algorithm, in fact, constructs upper confidence bounds on the learned value function to select actions optimistically. We implement and evaluate DOP on different scenarios: (1) a cooperative navigation problem, (2) a fetching task for a 7-DOF KUKA robot, and (3) a human-robot handover with a humanoid robot (both in simulation and real). The obtained results show the effectiveness of DOP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance