Online Learning of a Memory for Learning Rates

The promise of learning to learn for robotics rests on the hope that by extracting some information about the learning process itself we can speed up subsequent similar learning tasks. Here, we introduce a computationally efficient online meta-learning algorithm that builds and optimizes a memory model of the optimal learning rate landscape from previously observed gradient behaviors. While performing task specific optimization, this memory of learning rates predicts how to scale currently observed gradients. After applying the gradient scaling our meta-learner updates its internal memory based on the observed effect its prediction had. Our meta-learner can be combined with any gradient-based optimizer, learns on the fly and can be transferred to new optimization tasks. In our evaluations we show that our meta-learning algorithm speeds up learning of MNIST classification and a variety of learning control tasks, either in batch or online learning settings.Comment: accepted to ICRA 2018, code available: https://github.com/fmeier/online-meta-learning ; video pitch available: https://youtu.be/9PzQ25FPPO

Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks

In order to robustly execute a task under environmental uncertainty, a robot needs to be able to reactively adapt to changes arising in its environment. The environment changes are usually reflected in deviation from expected sensory traces. These deviations in sensory traces can be used to drive the motion adaptation, and for this purpose, a feedback model is required. The feedback model maps the deviations in sensory traces to the motion plan adaptation. In this paper, we develop a general data-driven framework for learning a feedback model from demonstrations. We utilize a variant of a radial basis function network structure --with movement phases as kernel centers-- which can generally be applied to represent any feedback models for movement primitives. To demonstrate the effectiveness of our framework, we test it on the task of scraping on a tilt board. In this task, we are learning a reactive policy in the form of orientation adaptation, based on deviations of tactile sensor traces. As a proof of concept of our method, we provide evaluations on an anthropomorphic robot. A video demonstrating our approach and its results can be seen in https://youtu.be/7Dx5imy1KcwComment: 8 pages, accepted to be published at the International Conference on Robotics and Automation (ICRA) 201

A New Data Source for Inverse Dynamics Learning

Modern robotics is gravitating toward increasingly collaborative human robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, adaptive, and compliant robots. These tools require us to model the system dynamics accurately -- a difficult task. The fundamental problem remains that simulation and reality diverge--we do not know how to accurately change a robot's state. Thus, recent research on improving inverse dynamics models has been focused on making use of machine learning techniques. Traditional learning techniques train on the actual realized accelerations, instead of the policy's desired accelerations, which is an indirect data source. Here we show how an additional training signal -- measured at the desired accelerations -- can be derived from a feedback control signal. This effectively creates a second data source for learning inverse dynamics models. Furthermore, we show how both the traditional and this new data source, can be used to train task-specific models of the inverse dynamics, when used independently or combined. We analyze the use of both data sources in simulation and demonstrate its effectiveness on a real-world robotic platform. We show that our system incrementally improves the learned inverse dynamics model, and when using both data sources combined converges more consistently and faster.Comment: IROS 201

Learning Feedback Terms for Reactive Planning and Control

With the advancement of robotics, machine learning, and machine perception, increasingly more robots will enter human environments to assist with daily tasks. However, dynamically-changing human environments requires reactive motion plans. Reactivity can be accomplished through replanning, e.g. model-predictive control, or through a reactive feedback policy that modifies on-going behavior in response to sensory events. In this paper, we investigate how to use machine learning to add reactivity to a previously learned nominal skilled behavior. We approach this by learning a reactive modification term for movement plans represented by nonlinear differential equations. In particular, we use dynamic movement primitives (DMPs) to represent a skill and a neural network to learn a reactive policy from human demonstrations. We use the well explored domain of obstacle avoidance for robot manipulation as a test bed. Our approach demonstrates how a neural network can be combined with physical insights to ensure robust behavior across different obstacle settings and movement durations. Evaluations on an anthropomorphic robotic system demonstrate the effectiveness of our work.Comment: 8 pages, accepted to be published at ICRA 2017 conferenc

Myristoyl CoA:Protein N-Myristoyl Transferase: A Target for a Novel Antimalarial Drug

Malaria, an illness caused by protozoan parasites of the genus Plasmodium, continues to be a key global health issue; around 40% of the world’s population are at risk and more than one million people are killed each year according to the World Health Organisation (WHO). It is transmitted via bites of infected female mosquitoes (Anopheles) and its severest form, falciparum malaria, can lead to death if left untreated. Effective malarial treatment is complex due to drug resistance and socioeconomic issues in many of the most affected areas. An enzyme from the parasite, myristoyl CoA:protein N-myristoyl transferase (NMT), has been identified as a potential target for antimalarial drugs. N-Myristoyl transferase, which catalyses the co-translational transfer of myristic acid to an N-terminal glycine of certain substrate proteins, has been shown to be essential for various pathogens. This thesis demonstrates the design, synthesis and analysis of potential inhibitors of Plasmodium falciparum NMT. Approximately 50 inhibitors with systematic variations based on a benzothiazole scaffold have been synthesised. It is known that these benzothiazoles compete with binding of peptide substrate within the NMT enzyme binding cleft. Differences between the peptide binding pockets of P. falciparum and human NMTs were exploited to design effective and selective new antimalarial treatments. The level of inhibition was measured using SPA that monitors the transfer of 3H-labelled myristoyl CoA to the N-terminus of a polypeptide substrate. A plot of enzyme activity as a function of inhibitor concentration gave inhibition curves from which IC50-values were derived. In vitro tests resulted in four hits with improved activity in the low micromolar region against P. falciparum NMT compared to the lead compound. Nevertheless, the inhibitors were not exceptionally selective over Homo sapiens NMT with an IC50 in the low micromolar region also. Selections of the most promising inhibitors have been tested in vivo and considerable reductions in parasitemia were noted