Towards socially adaptive robots : A novel method for real time recognition of human-robot interaction styles

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.” DOI: 10.1109/ICHR.2008.4756004Automatically detecting different styles of play in human-robot interaction is a key challenge towards adaptive robots, i.e. robots that are able to regulate the interactions and adapt to different interaction styles of the robot users. In this paper we present a novel algorithm for pattern recognition in human-robot interaction, the Cascaded Information Bottleneck Method. We apply it to real-time autonomous recognition of human-robot interaction styles. This method uses an information theoretic approach and enables to progressively extract relevant information from time series. It relies on a cascade of bottlenecks, the bottlenecks being trained one after the other according to the existing Agglomerative Information Bottleneck Algorithm. We show that a structure for the bottleneck states along the cascade emerges and we introduce a measure to extrapolate unseen data. We apply this method to real-time recognition of Human-Robot Interaction Styles by a robot in a detailed case study. The algorithm has been implemented for real interactions between humans and a real robot. We demonstrate that the algorithm, which is designed to operate real time, is capable of classifying interaction styles, with a good accuracy and a very acceptable delay. Our future work will evaluate this method in scenarios on robot-assisted therapy for children with autism.Peer reviewe

Empowerment and State-dependent Noise : An Intrinsic Motivation for Avoiding Unpredictable Agents

Empowerment is a recently introduced intrinsic motivation algorithm based on the embodiment of an agent and the dynamics of the world the agent is situated in. Computed as the channel capacity from an agent’s actuators to an agent’s sensors, it offers a quantitative measure of how much an agent is in control of the world it can perceive. In this paper, we expand the approximation of empowerment as a Gaussian linear channel to compute empowerment based on the covariance matrix between actuators and sensors, incorporating state dependent noise. This allows for the first time the study of continuous systems with several agents. We found that if the behaviour of another agent cannot be predicted accurately, then interacting with that agent will decrease the empowerment of the original agent. This leads to behaviour realizing collision avoidance with other agents, purely from maximising an agent’s empowermentFinal Accepted Versio

An Informational Study of the Evolution of Codes in Different Population Structures

Best Student Paper Award. Attribution-NonCommercial-NoDerivs 3.0 United StatesWe consider the problem of the evolution of a code within a structured population of agents. The agents try to maximise their information about their environment by acquiring information from the outputs of other agents in the population. A naive use of information-theoretic methods would assume that every agent knows how to “interpret” the information offered by other agents. However, this assumes that one “knows” which other agents one observes, and thus which code they use. In our model, however, we wish to preclude that: it is not clear which other agents an agent is observing, and the resulting usable information is therefore influenced by the universality of the code used and by which agents an agent is “listening” to

An informational perspective on how the embodiment can relieve cognitive burden

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”Living organisms are under permanent pressure to take decisions with an impact on their success. Such decisions require information, which can be formulated in the precise sense of Shannon information. Since information processing is costly for organisms, this creates an adaptive pressure for cognition to be as informationally parsimonious as possible. Combining information theory with the theory of reinforcement learning for modeling tasks, we present a number of quantitative analyses how the cognitive burden of an agent deriving from a task can be relieved by the environment and, more specifically, its embodiment, i.e. how the agent "controller" is linked to the environment, via perception (in principle, but not further considered here) and action (this paper's main focus). The methodology presented offers a path towards a formal and quantitative treatment of Paul's and Pfeifer's concept of morphological computation in particular and their envisaged larger picture of offloading of computation onto the environment dynamics in general. In particular, it offers additional evidence for the central importance of the embodiment for the success of cognition

Kernelizing LSPE λ

We propose the use of kernel-based methods as underlying function approximator in the least-squares based policy evaluation framework of LSPE(λ) and LSTD(λ). In particular we present the ‘kernelization’ of model-free LSPE(λ). The ‘kernelization’ is computationally made possible by using the subset of regressors approximation, which approximates the kernel using a vastly reduced number of basis functions. The core of our proposed solution is an efficient recursive implementation with automatic supervised selection of the relevant basis functions. The LSPE method is well-suited for optimistic policy iteration and can thus be used in the context of online reinforcement learning. We use the high-dimensional Octopus benchmark to demonstrate this

Effects of Anticipation in Individually Motivated Behaviour on Control and Survival in a Multi-Agent Scenario with Resource Constraints

This is an open access article distributed under the Creative Commons Attribution License CC BY 3.0 which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Self-organization and survival are inextricably bound to an agent’s ability to control and anticipate its environment. Here we assess both skills when multiple agents compete for a scarce resource. Drawing on insights from psychology, microsociology and control theory, we examine how different assumptions about the behaviour of an agent’s peers in the anticipation process affect subjective control and survival strategies. To quantify control and drive behaviour, we use the recently developed information-theoretic quantity of empowerment with the principle of empowerment maximization. In two experiments involving extensive simulations, we show that agents develop risk-seeking, risk-averse and mixed strategies, which correspond to greedy, parsimonious and mixed behaviour. Although the principle of empowerment maximization is highly generic, the emerging strategies are consistent with what one would expect from rational individuals with dedicated utility models. Our results support empowerment maximization as a universal drive for guided self-organization in collective agent systemsPeer reviewedFinal Published versio

Empowerment as a metric for Optimization in HCI

We propose a novel metric for optimizing human-computer interfaces, based on the information-theoretic capacity of empowerment, a task-independent universal utility measure. Empowerment measures, for agent-environment systems with stochastic transitions, how much influence, which can be sensed by the agent sensors, an agent has on its environment. It captures the uncertainty in human-machine systems arising from different sources (i.e. noise, delays, errors, etc.) as a single quantity. We suggest the potential empowerment has as an objective optimality criterion in user interface design optimization, contributing to the more solid theoretical foundations of HCI.Peer reviewedFinal Accepted Versio

Sensor Adaptation and Development in Robots by Entropy Maximization of Sensory Data

A method is presented for adapting the sensors of a robot to the statistical structure of its current environment. This enables the robot to compress incoming sensory information and to find informational relationships between sensors. The method is applied to creating sensoritopic maps of the informational relationships of the sensors of a developing robot, where the informational distance between sensors is computed using information theory and adaptive binning. The adaptive binning method constantly estimates the probability distribution of the latest inputs to maximize the entropy in each individual sensor, while conserving the correlations between different sensors. Results from simulations and robotic experiments with visual sensors show how adaptive binning of the sensory data helps the system to discover structure not found by ordinary binning. This enables the developing perceptual system of the robot to be more adapted to the particular embodiment of the robot and the environment