Information driven exploration in robotics

Imagine an intelligent robot entering an unknown room. It starts interacting with its new surroundings to understand what properties the new objects have and how they interact with each other. Finally, he gathered enough information to skillfully perform various tasks in the new environment. This is the vision behind our research towards intelligent robots. An important role in the described behavior is the ability to chose actions in order to learn new things. This ability we call exploration. It enables the robot to quickly learn about the properties of the objects. Surprisingly autonomous exploration has been mostly neglected by robotics research so far, because many fundamental problems like motor control and perception were still not satisfactory solved. The developments of recent years have, however, overcome this hurdle. State of the art methods enable us now to conduct research on exploration in robotics. On the other hand the machine learning and statistics community has developed methods and the theoretical background to lead learning algorithms to the most promising data. Under the terms active learning and experimental design many methods have been developed to improve the learning rate with fewer training data. In this thesis we combine results from both fields to develop a framework of exploration in robotics. We base our framework on the notion of information and information gain, developed in the field of information theory. And although we show that optimal exploration is a computational hard problem, we develop efficient exploration strategies using information gain as measure and Bayesian experimental design as foundation. To test the explorative behavior generated by our strategies we introduce the Physical Exploration Challenge. It formalizes the desired behavior as exploration of external degrees of freedom. External degrees of freedom are those the robot can not articulate directly but only by interacting with the environment. We present how we can model different exploration tasks of external degree of freedom: Exploring the meaning of geometric symbols by moving objects, exploring the existence of joints and their properties, and exploring how different joints in the environment are interdependent. Different robots show these exploration tasks in both simulated and real world experiments.Wie würde sich ein intelligenter Roboter verhalten, der einen ihm unbekannten Raum betritt? Vermutlich würde er anfangen all die Dinge um sich herum zu untersuchen, um sich ein Bild darüber zu verschaffen, welche Eigenschaften die Objekte ausmachen und wie sie miteinander zusammenhängen. Dieses Wissen würde es ihm dann ermöglichen verschiedenste Aufgaben in der neuen Umgebung zu erledigen. Eine zentrale Rolle bei diesem Verhalten spielt die Fähigkeit eigenständig zu entschieden, was es zu untersuchen gilt. Diese Fähigkeit nennt man Exploration. Erstaunlicherweise wurde autonome Exploration bisher in der Robotik vernachlässigt. Der Grund liegt darin, dass grundlegendere Fähigkeiten, wie zum Beispiel die Erzeugung von Bewegung oder die Wahrnehmung, die Wissenschaft bisher vor große Probleme stellten. Die Entwickelungen der letzten Jahre in diesen Bereichen ermöglichen uns aber nun Exploration der Umwelt mit Robotern zu untersuchen. Auf der anderen Seite wurden im Bereich des Maschinellen Lernens und der Statistik Methoden und theoretische Grundlagen entwickelt, die Lernalgorithmen in die Lage versetzen, selber ihre Trainingdaten zu sammeln. Dadurch kann die Lernrate mit möglichst wenig Trainingsdaten verbessert werden. Diese Methoden werden unter den Begriffen Aktives Lernen und Experimentelles Design zusammengefasst. In dieser Arbeit kombinieren wir die Resultate aus den beiden vorgenannten Feldern. Wir entwickeln damit die Grundlagen für autonome Exploration in der Robotik. Wir leiten diese Grundlagen von der Informationstheorie ab, die eine formale Definition von den Größen Information und Informationsgewinn entwickelt hat. Und obwohl wir zeigen, dass optimale Exploration nicht effzient berechenbar ist, können wir basierend auf dem Informationsgewinn Heuristiken entwicklen, die zu effizienten Explorationsstrategien führen. Um das Explorationsverhalten, dass sich aus diesen Strategien entwickelt, zu testen, führen wir die Physical Exploration Challenge ein, das Problem der physikalischen Exploration. Es formalisiert unsere Vision eines intelligenten, explorierenden Roboters als Problem der Exploration von externen Freiheitsgraden. Externe Freiheitsgrade sind solche, die der Roboter nicht direkt beeinflussen kann, sondern nur durch Interaktion mit der Umwelt. Schlussendlich modellieren wir verschiedene Explorationsaufgaben von externen Freiheitsgraden und zeigen mit verschiedenen Robotern, simulierten wie auch echten, wie diese Aufgaben gelöst werden können. Die Aufgaben umfassen dabei das Erkunden der Bedeutung von Symbolen, die geometrische Zusammenhänge widerspiegeln, die Exploration von Existenz und Eigenschaften von Gelenken in der Umwelt und wie die Stellung von Gelenken entscheidend für die Beweglichkeit andere Gelenke sein kann

Johannes Kulick, Marco Block und Raul Rojas

News and information about Himmelfarb Health Sciences Library of interest to users. Includes articles on Overleaf collaboration tool, video resources at Himmelfarb, and profiles on Brian McDonald and Leighton Ku

Entropy-based strategies for physical exploration of the environment's degrees of freedom

Abstract — Physical exploration refers to the challenge of autonomously discovering and learning how to manipulate the environment’s degrees of freedom (DOF)—by identifying promising points of interaction and pushing or pulling object parts to reveal DOF and their properties. Recent existing work focused on sub-problems like estimating DOF parameters from given data. Here, we address the integrated problem, focusing on the higher-level strategy to iteratively decide on the next exploration point before applying motion generation methods to execute the explorative action and data analysis methods to interpret the feedback. We propose to decide on exploration points based on the expected information gain, or change in entropy in the robot’s current belief (uncertain knowledge) about the DOF. To this end, we first define how we represent such a belief. This requires dealing with the fact that the robot initially does not know which random variables (which DOF, and depending on their type, which DOF properties) actually exist. We then propose methods to estimate the expected information gain for an exploratory action. We analyze these strategies in simple environments and evaluate them in combination with full motion planning and data analysis in a physical simulation environment. I

The Advantage of Cross Entropy over Entropy in Iterative Information Gathering

Gathering the most information by picking the least amount of data is a common task in experimental design or when exploring an unknown environment in reinforcement learning and robotics. A widely used measure for quantifying the information contained in some distribution of interest is its entropy. Greedily minimizing the expected entropy is therefore a standard method for choosing samples in order to gain strong beliefs about the underlying random variables. We show that this approach is prone to temporally getting stuck in local optima corresponding to wrongly biased beliefs. We suggest instead maximizing the expected cross entropy between old and new belief, which aims at challenging refutable beliefs and thereby avoids these local optima. We show that both criteria are closely related and that their difference can be traced back to the asymmetry of the Kullback-Leibler divergence. In illustrative examples as well as simulated and real-world experiments we demonstrate the advantage of cross entropy over simple entropy for practical applications.Comment: 24 page

Active Learning of Hyperparameters: An Expected Cross Entropy Criterion for Active Model Selection

In standard active learning, the learner’s goal is to reduce the predictive uncertainty with as little data as possible. We consider a slightly different problem: the learner’s goal is to uncover latent properties of the model—e.g., which features are relevant (“active feature selection”), or the choice of hyper parameters—with as little data as possible. While the two goals are clearly related, we give examples where following the predictive uncertainty objective is suboptimal for uncovering latent parameters. We propose novel measures of information gain about the latent parameter, based on the divergence between the prior and expected posterior distribution over the latent parameter in question. Notably, this is different from applying Bayesian experimental design to latent variables: we give explicit examples showing that the latter objective is prone to get stuck in local minima, unlike its application the standard predictive uncertainty. Extensive evaluations show that active learning using our measures significantly accelerates the uncovering of latent model parameters, as compared to standard version space approaches (Query-by-committee) as well as predictive uncertainty measures.