Active Learning based on Data Uncertainty and Model Sensitivity

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisions with the environment. Active learning can quantify the uncertainty of performing the task and, in general, locate regions of missing information. We introduce a novel algorithm for active learning and demonstrate its utility for generating smooth trajectories. Our approach is based on deep generative models and metric learning in latent spaces. It relies on the Jacobian of the likelihood to detect non-smooth transitions in the latent space, i.e., transitions that lead to abrupt changes in the movement of the robot. When non-smooth transitions are detected, our algorithm asks for an additional demonstration from that specific region. The newly acquired knowledge modifies the data manifold and allows for learning a latent representation for generating smooth movements. We demonstrate the efficacy of our approach on generalising elementary skills, transitioning across different skills, and implicitly avoiding collisions with the environment. For our experiments, we use a simulated pendulum where we observe its motion from images and a 7-DoF anthropomorphic arm.Comment: Published on 2018 IEEE/RSJ International Conference on Intelligent Robots and Syste

Operationally meaningful representations of physical systems in neural networks

To make progress in science, we often build abstract representations of physical systems that meaningfully encode information about the systems. The representations learnt by most current machine learning techniques reflect statistical structure present in the training data; however, these methods do not allow us to specify explicit and operationally meaningful requirements on the representation. Here, we present a neural network architecture based on the notion that agents dealing with different aspects of a physical system should be able to communicate relevant information as efficiently as possible to one another. This produces representations that separate different parameters which are useful for making statements about the physical system in different experimental settings. We present examples involving both classical and quantum physics. For instance, our architecture finds a compact representation of an arbitrary two-qubit system that separates local parameters from parameters describing quantum correlations. We further show that this method can be combined with reinforcement learning to enable representation learning within interactive scenarios where agents need to explore experimental settings to identify relevant variables.Comment: 24 pages, 13 figure

A stochastic process model for free agency under indeterminism

The aim of this paper is to establish that free agency, which is a capacity of many animals including human beings, is compatible with indeterminism: An indeterministic world allows for the existence of free agency. The question of the compatibility of free agency and indeterminism is less discussed than its mirror image, the question of the compatibility of free agency and determinism. It is, however, of great importance for our self-conception as free agents in our (arguably) indeterministic world. We begin by explicating the notions of indeterminism and free agency and by clarifying the interrelation of free agency and the human-specific notion of free will. We then situate our claim of the compatibility of free agency and indeterminism precisely in the landscape of the current debate on freedom and determinism, exposing an unhappy asymmetry in that debate. Then we proceed to make our case by describing the mathematically precise, physically motivated model of projective simulation, which employs indeterminism as a central resource for agency modeling. Projective simulation was recently developed as an AI framework for flexible learning agents (Briegel and De las Cuevas, Scientific Reports 2:400, 2012). We argue that an indeterministic process of deliberation modeled by the dynamics of projective simulation can exemplify free agency under indeterminism, thereby establishing our compatibility claim: Free agency can develop and thrive in an indeterministic world