Abstraction in decision-makers with limited information processing capabilities

A distinctive property of human and animal intelligence is the ability to form abstractions by neglecting irrelevant information which allows to separate structure from noise. From an information theoretic point of view abstractions are desirable because they allow for very efficient information processing. In artificial systems abstractions are often implemented through computationally costly formations of groups or clusters. In this work we establish the relation between the free-energy framework for decision making and rate-distortion theory and demonstrate how the application of rate-distortion for decision-making leads to the emergence of abstractions. We argue that abstractions are induced due to a limit in information processing capacity.Comment: Presented at the NIPS 2013 Workshop on Planning with Information Constraint

An information-theoretic on-line update principle for perception-action coupling

Inspired by findings of sensorimotor coupling in humans and animals, there has recently been a growing interest in the interaction between action and perception in robotic systems [Bogh et al., 2016]. Here we consider perception and action as two serial information channels with limited information-processing capacity. We follow [Genewein et al., 2015] and formulate a constrained optimization problem that maximizes utility under limited information-processing capacity in the two channels. As a solution we obtain an optimal perceptual channel and an optimal action channel that are coupled such that perceptual information is optimized with respect to downstream processing in the action module. The main novelty of this study is that we propose an online optimization procedure to find bounded-optimal perception and action channels in parameterized serial perception-action systems. In particular, we implement the perceptual channel as a multi-layer neural network and the action channel as a multinomial distribution. We illustrate our method in a NAO robot simulator with a simplified cup lifting task.Comment: 8 pages, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS

A Nonparametric Conjugate Prior Distribution for the Maximizing Argument of a Noisy Function

We propose a novel Bayesian approach to solve stochastic optimization problems that involve finding extrema of noisy, nonlinear functions. Previous work has focused on representing possible functions explicitly, which leads to a two-step procedure of first, doing inference over the function space and second, finding the extrema of these functions. Here we skip the representation step and directly model the distribution over extrema. To this end, we devise a non-parametric conjugate prior based on a kernel regressor. The resulting posterior distribution directly captures the uncertainty over the maximum of the unknown function. We illustrate the effectiveness of our model by optimizing a noisy, high-dimensional, non-convex objective function.Comment: 9 pages, 5 figure

Meta-learning of Sequential Strategies

In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains. To do so, we present basic algorithmic templates for building near-optimal predictors and reinforcement learners which behave as if they had a probabilistic model that allowed them to efficiently exploit task structure. Furthermore, we recast memory-based meta-learning within a Bayesian framework, showing that the meta-learned strategies are near-optimal because they amortize Bayes-filtered data, where the adaptation is implemented in the memory dynamics as a state-machine of sufficient statistics. Essentially, memory-based meta-learning translates the hard problem of probabilistic sequential inference into a regression problem.Comment: DeepMind Technical Report (15 pages, 6 figures

Data from: Occam's Razor in sensorimotor learning

A large number of recent studies suggest that the sensorimotor system employs probabilistic models to predict its environment and makes inferences about unobserved variables in line with Bayesian statistics. One of the important features of Bayesian statistics is Occam's Razor - an inbuilt preference for simpler models when comparing competing models that explain some observed data equally well. Here we test directly for Occam's Razor in sensorimotor control. We designed a sensorimotor task, where participants had to draw lines through clouds of noisy samples of an unobserved curve generated by one of two possible probabilistic models - a simple model with a large length scale, leading to smooth curves and a complex model with a short length scale, leading to more wiggly curves. In training trials participants were informed about the model that generated the stimulus so that they could learn the statistics of each model. In probe trials participants were then exposed to ambiguous stimuli. In probe trials where the ambiguous stimulus could be fit equally well by both models, we found that participants showed a clear preference for the simpler model. Moreover, we found that participants' choice behavior was quantitatively consistent with Bayesian Occam's Razor. We could also show that participants' drawn trajectories were similar to samples from the Bayesian predictive distribution over trajectories and significantly different from two non-probabilistic heuristics. In two control experiments, we show that the preference of the simpler model cannot be simply explained by a difference in physical effort or by a preference for curve smoothness. Our results suggest that Occam's Razor is a general behavioral principle already present during sensorimotor processing

Occam's Razor in sensorimotor learning - human trajectory data

The data-file contains recordings of human motion data that was acquired in a virtual reality setup. Participants of the experiment were controlling a virtual cursor using a 3D manipulandum in order to draw regression trajectories through clouds of noisy observations that were presented to them through a head mounted display. See corresponding publication and included Readme-file for details