Many fields are now snowed under with an avalanche of data, which raises
considerable challenges for computer scientists. Meanwhile, robotics (among
other fields) can often only use a few dozen data points because acquiring them
involves a process that is expensive or time-consuming. How can an algorithm
learn with only a few data points

Mouret, Jean-Baptiste

English

arXiv

International audienceMany fields are now snowed under with an avalanche of data, which raises considerable challenges for computer scientists. Meanwhile, robotics (among other fields) can often only use a few dozen data points because acquiring them involves a process that is expensive or time-consuming. How can an algorithm learn with only a few data points

HAL portal of the University of Lorraine

Micro-Data Learning: The Other End of the Spectrum

INRIA a CCSD electronic archive server

HAL Id: hal-01374786https://hal.inria.fr/hal-01374786Submitted on 1 Oct 2016HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Distributed under a Creative Commons Attribution| 4.0 International LicenseMicro-Data Learning: The Other End of the SpectrumJean-Baptiste MouretTo cite this version:Jean-Baptiste Mouret. Micro-Data Learning: The Other End of the Spectrum. ERCIM News,ERCIM, 2016, pp.2. ￿hal-01374786￿ERCIM NEWS 107   October 201618Special Theme: Machine LearningWatching a child learn reveals how wellhumans can learn: a child may only needa few examples of a concept to “learn it”.By contrast, the impressive resultsachieved with modern machine learning(in particular, by deep learning) are madepossible largely by the use of hugedatasets. For instance, the ImageNet data-base used in image recognition containsabout 1.2 million labelled examples;DeepMinds's AlphaGo used more than 38million positions to train their algorithmto play Go; and the same company usedmore than 38 days of play to train a neuralnetwork to play Atari 2600 games, suchas Space Invaders or Breakout.Like children, robots have to face thereal world, in which trying somethingmight take seconds, hours, or days. Andseeing the consequence of this trialmight take much more. When robotsshare our world, they are expected tolearn like humans or animals, that is, infar fewer than a million trials. Robotsare not alone to be cursed by the price ofdata: Any learning process that involvesphysical tests or precise simulations(e.g., computational fluid dynamics)comes up against the same issue. Inshort, while data might be abundant inthe virtual world, it is often a scarceresource in the physical world. I refer tothis challenge as “micro-data” learning(see Figure 1).The first precept of micro-data learningis to choose as wisely as possible what totest next (active learning). Since compu-tation tends to become cheaper everyyear, it is often effective to trade dataresources for computational resources,that is, to employ computationally inten-sive algorithms to select the next datapoint to acquire. Bayesian optimisation[1] is such a data-efficient algorithm thathas recently attracted a lot of interest inthe machine learning community. Usingthe data acquired so far, this algorithmcreates a probabilistic model of the func-tion that needs to be optimised (e.g., thewalking speed of a robot or the lift gen-erated by an airfoil); it then exploits thismodel to identify the most promisingpoints of the search space. It can, forexample, find good values for the gaitof a quadruped robot (Sony Aibo / 15parameters to learn) in just two hours oflearning.The second precept of micro-datalearning is to exploit every bit of infor-mation from each test. For instance,when a robotic arm tries to reach a pointin space, the learning algorithm can per-form the movement, then, at the end ofthe trial, measure the distance to thetarget. In this case, each test corre-sponds to a single data point. However,the algorithm can also record the posi-tion of the "hand" every 10ms, thus get-ting thousands of data points from asingle test. This is a very effectiveapproach for learning control strategiesin robotics; for example, the Pilco algo-rithm can learn to balance a non-actu-ated pole on an actuated moving cart in15-20 seconds (about 3-5 trials) [2].The third precept of micro-data learningis to use the "right" prior knowledge.Most problems are indeed simply toohard to be learned from scratch in a fewtrials, even with the best algorithms:The quick learning ability of humansand animals is due largely to their priorknowledge about what could and couldnot work. When using priors, it is crit-ical to make them as explicit as pos-sible, and to make sure that the learningalgorithm can question or even ignorethem. In academic examples, it can alsobe challenging to distinguish betweenprior knowledge that is useful and priorknowledge that actually gives the solu-Micro-Data Learning: The Other End of the Spectrumby Jean-Baptiste Mouret (Inria)Many fields are now snowed under with an avalanche of data, which raises considerablechallenges for computer scientists. Meanwhile, robotics (among other fields) can oftenonly use a few dozen data points because acquiring them involves a process that isexpensive or time-consuming. How can an algorithm learn with only a few data points?Figure 1: Modern machine learning (e.g., deep learning) is designed to work with a largeamount of data. For example, the Go player AlphaGo by DeepMind used a dataset of 38 millionpositions, and the deep reinforcement learning experiments from the same team used theequivalent of 38 days to learn to play Atari 2600 video games. Robotics is at the opposite end ofthe spectrum: most of the time, it is difficult to perform more than a few dozen trials. Learningwith such a small amount of data is what we term "Micro-data learning".ERCIM NEWS 107   October 2016 19tion to the algorithm, which leavesnothing to learn.We focused on prior knowledge in ourrecent article about damage recovery inrobotics [3, L1]. In this scenario, a six-legged walking robot needs to discovera new way to walk by trial-and-errorbecause it is damaged. Before the mis-sion, a novel algorithm explores a largesearch space with a simulation of theintact robot to identify the most prom-ising solution of each "family".Metaphorically, this algorithm takes theneedles out of a haystack to make astack of needles. If the robot is dam-aged, the learning algorithm, which is aderivative of Bayesian optimisation [1],exploits this prior knowledge to choosethe best trials. In our experiments, therobot discovers compensatory gaits inless than two minutes and a dozen trials,for the five damage conditions that wetested [3].In this learning approach, a data-effi-cient learning algorithm that works withthe physical, damaged robot is guidedby prior knowledge based on a simula-tion of the intact robot. This micro-datalearning algorithm makes it possible tolearn a complex task in only a few trials.The subsequent challenge is to exploitmore knowledge from the trials [2] andselect the next trials while taking thecontext into account (e.g., potentialobstacles).Link:[L1] http://www.resibots.euReferences: [1] B. Shahriari, et al.: “Taking thehuman out of the loop: A review ofbayesian optimization”, Proc. of theIEEE, 2016.[2] M. P. Deisenroth, D. Fox, C. E.Rasmussen: “Gaussian processes fordata-efficient learning in robotics andcontrol”, IEEE Trans. on PatternAnalysis and Machine Intelligence,2016.[3] A. Cully, et al.: “Robots that canadapt like animals”, Nature, 2015.Please contact: Jean-Baptiste MouretInria, Francejean-baptiste.mouret@inria.frAfter a history spanning over fivedecades, artificial general intelligencestill remains out of reach. Machinelearning has common roots with AIresearch, but focuses on more attain-able goals and has achieved tremen-dous success in many applicationfields. Similarly, a universal quantumcomputer is still far ahead in the distantfuture: the criterion for this machine isto be able to simulate an arbitraryclosed quantum system. Nevertheless,uses of quantum information pro-cessing are proliferating: two notableexamples are quantum key distributionsystems and quantum random numbergenerators.Recently, there has been a surge ofinterest in the intersection of machinelearning and quantum information pro-cessing. Combining ideas from thesetwo fields leads to tremendous benefitsfor both. We are collaborating on sev-eral subjects in this domain betweenICFO-The Institute of PhotonicSciences, the Autonomous Universityof Barcelona, the University of theBasque Country, all in Spain, as well asthe University of Calgary, Canada. At the highest level, abstracting of theactual algorithms and focusing on thefoundations of statistical learningtheory, we can ask what it means tolearn with quantum data and channels,what induction and transduction meanin this setting, how we can define fig-ures of merit to quantify performance,and eventually establish bounds on gen-eralisation performance using sampleand model complexity. We studiedsupervised learning, and proved that inthe asymptotic limit and under anassumption of exchangeability,quantum entanglement does not breakour traditional notion of induction [L1].This is an important stepping stonetowards understanding generalisationproperties of quantum learning proto-cols.The next natural question to ask is thatgiven a universal quantum computer,what kind of protocols can we use forMaking Learning Physical: Machine Intelligenceand Quantum Resourcesby Peter Wittek (ICFO-The Institute of Photonic Sciences and University of Borås)It is not only machine learning that is advancing rapidly: quantum information processing haswitnessed several breakthroughs in recent years. In theory, quantum protocols can offer anexponential speedup for certain learning algorithms, but even contemporary implementations showremarkable results – this new field is called quantum machine learning. The benefits work both ways:classical machine learning finds more and more applicability in problems in quantum computing.Figure 1: Overview of the interplay between quantum information processing and machinelearning.

Archive Ouverte en Sciences de l'Information et de la Communication

HAL: Hyper Article en Ligne

HAL-CentraleSupelec

https://hal.inria.fr/hal-01374786/file/ercim_mouret.pdf

Micro-Data Learning: The Other End of the Spectrum

Abstract

Similar works

Full text

Available Versions

HAL portal of the University of Lorraine

INRIA a CCSD electronic archive server

Archive Ouverte en Sciences de l'Information et de la Communication

HAL: Hyper Article en Ligne

HAL-CentraleSupelec