Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

Intrinsically motivated spontaneous exploration is a key enabler of autonomous lifelong learning in human children. It enables the discovery and acquisition of large repertoires of skills through self-generation, self-selection, self-ordering and self-experimentation of learning goals. We present an algorithmic approach called Intrinsically Motivated Goal Exploration Processes (IMGEP) to enable similar properties of autonomous or self-supervised learning in machines. The IMGEP algorithmic architecture relies on several principles: 1) self-generation of goals, generalized as fitness functions; 2) selection of goals based on intrinsic rewards; 3) exploration with incremental goal-parameterized policy search and exploitation of the gathered data with a batch learning algorithm; 4) systematic reuse of information acquired when targeting a goal for improving towards other goals. We present a particularly efficient form of IMGEP, called Modular Population-Based IMGEP, that uses a population-based policy and an object-centered modularity in goals and mutations. We provide several implementations of this architecture and demonstrate their ability to automatically generate a learning curriculum within several experimental setups including a real humanoid robot that can explore multiple spaces of goals with several hundred continuous dimensions. While no particular target goal is provided to the system, this curriculum allows the discovery of skills that act as stepping stone for learning more complex skills, e.g. nested tool use. We show that learning diverse spaces of goals with intrinsic motivations is more efficient for learning complex skills than only trying to directly learn these complex skills

Observation(s), entretien(s) : ce que le terrain fait au chercheur dans la problématisation de la transmission mémorielle infra-familiale

International audienceÀ travers une résidence artistique et scientifique menée au sein du projet Mémoire de clandestinités organisé par l’association Les Rias située à Saint-Apollinaire de Rias en Nord-Ardèche nous avons entrepris d’interroger les formes de conservation et de transmission de la mémoire et ses mécanismes communicationnels au sein de la famille. Opérant à travers la collecte de traces, la transmission mémorielle intra-familiale se nourrit des échanges humains. Permettant d’élaborer le récit, elle contribue à la construction de l’identité. Penser l’idée de « Résistance » en lien avec la construction de l’identité collective nous a amené à analyser les formes de patrimonialisation empruntées par des enfants de résistants, croisant reconstitutions mémorielles et analyse du témoignage de « résistants » de la Seconde Guerre mondiale et travail plastique. L’enquête de terrain ethnographique et la recherche photographique ont permis d’identifier les représentations et les pratiques mémorielles tout en interrogeant la méthodologie et la conception de la production artistique. Lors de nos immersions, nous avons pris soin d’adopter une position bivalente, à la fois objectivante et participante dans le milieu familier des interviewés, attitude indispensable au développement d’une relation de confiance avec les témoins. Outre la collecte des données nécessaires à la démarche archivistique et la production artistique, s’est jouée une reconfiguration de notre façon de problématiser les actions de patrimonialisation que nous avons pu analyser et croiser avec les nombreux échanges avec Pablo Garcia, jeune plasticien auteur de nombreux travaux photographiques et vidéos portant sur la mémoire historique, la trace et l’utopie sociale ayant rejoint le projet mais aussi avec notre propre production artistique. Ce terrain, riche et passionnant nous amené à relativiser les concepts qui semblaient pourtant majeur en début de recherche. La pratique ethnographique nous a ainsi engagée dans une reconfiguration esthétique, communicationnelle et méthodologique de notre projet de thèse

Transfer learning for time series classification

Transfer learning for deep neural networks is the process of first training a base network on a source dataset, and then transferring the learned features (the network's weights) to a second network to be trained on a target dataset. This idea has been shown to improve deep neural network's generalization capabilities in many computer vision tasks such as image recognition and object localization. Apart from these applications, deep Convolutional Neural Networks (CNNs) have also recently gained popularity in the Time Series Classification (TSC) community. However, unlike for image recognition problems, transfer learning techniques have not yet been investigated thoroughly for the TSC task. This is surprising as the accuracy of deep learning models for TSC could potentially be improved if the model is fine-tuned from a pre-trained neural network instead of training it from scratch. In this paper, we fill this gap by investigating how to transfer deep CNNs for the TSC task. To evaluate the potential of transfer learning, we performed extensive experiments using the UCR archive which is the largest publicly available TSC benchmark containing 85 datasets. For each dataset in the archive, we pre-trained a model and then fine-tuned it on the other datasets resulting in 7140 different deep neural networks. These experiments revealed that transfer learning can improve or degrade the model's predictions depending on the dataset used for transfer. Therefore, in an effort to predict the best source dataset for a given target dataset, we propose a new method relying on Dynamic Time Warping to measure inter-datasets similarities. We describe how our method can guide the transfer to choose the best source dataset leading to an improvement in accuracy on 71 out of 85 datasets.Comment: Accepted at IEEE International Conference on Big Data 201

Adversarial Attacks on Deep Neural Networks for Time Series Classification

Time Series Classification (TSC) problems are encountered in many real life data mining tasks ranging from medicine and security to human activity recognition and food safety. With the recent success of deep neural networks in various domains such as computer vision and natural language processing, researchers started adopting these techniques for solving time series data mining problems. However, to the best of our knowledge, no previous work has considered the vulnerability of deep learning models to adversarial time series examples, which could potentially make them unreliable in situations where the decision taken by the classifier is crucial such as in medicine and security. For computer vision problems, such attacks have been shown to be very easy to perform by altering the image and adding an imperceptible amount of noise to trick the network into wrongly classifying the input image. Following this line of work, we propose to leverage existing adversarial attack mechanisms to add a special noise to the input time series in order to decrease the network's confidence when classifying instances at test time. Our results reveal that current state-of-the-art deep learning time series classifiers are vulnerable to adversarial attacks which can have major consequences in multiple domains such as food safety and quality assurance.Comment: Accepted at IJCNN 201

Deep learning for time series classification: a review

Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8,730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.Comment: Accepted at Data Mining and Knowledge Discover

L'élevage, le conseil, et l'écopathologie. Résultats d'une approche didactique de situations de diagnostic d'élevage en production porcine

International audienc

Deep constrained clustering applied to satellite image time series

International audienceThe advent of satellite imagery is generating an unprecedented amount of remote sensing images. Current satellites now achieve frequent revisits and high mission availability and provide series of images of the Earth captured at different dates that can be seen as time series. Analyzing satellite image time series allows to perform continuous wide range Earth observation with applications in agricultural mapping , environmental disaster monitoring, etc. However, the lack of large quantity of labeled data generally prevents from easily applying supervised methods. On the contrary, unsupervised methods do not require expert knowledge but sometimes provide poor results. In this context, constrained clustering, which is a class of semi-supervised learning algorithms , is an alternative and offers a good trade-off of supervision. In this paper, we explore the use of constraints with deep clustering approaches to process satellite image time series. Our experimental study relies on deep embedded clustering and the deep constrained framework using pairwise constraints (must-link and cannot-link). Experiments on a real dataset composed of 11 satellite images show promising results and open many perspectives for applying deep constrained clustering to satellite image time series

Etude expérimentale des champs magnétiques en surface d'une cible irradiée par laser et leurs implications sur le faisceau d'électrons

This thesis concerns magnetic fields, generated by the interaction between strong laser pulse (intensity up to1018 W/cm2) and solid target, and their effects on the fast electron beam. Indeed, the various magnetic fields created during this interaction can inuence the divergence of the fast electron beam. The magnetic field createdduring this interaction have a fundamental role on the fast electron beam characteristics : its source and its transportin the material. Diagnotics of polarimetry and crossed interferometry were developed during this thesis to observethe on-surface magnetic field of the target, and in particular, their spatial and temporal evolutions. Two types oftemporal evolution of the magnetic field were observed according to the contrast in intensity of the laser pulse : afast rise of magnetic field followed by a slower decrease created by the travel of the fast electrons in the material,and a slower growth of logarithmic form created by the pre-pulse of the laser by thermoelectric effect. The interpretation of our results obtained by these diagnotics allowed us to estimate the resistivity of the plasma.This resistivity named "anomalously high resistivity" in the literature can be explained by taking into account theinuence of the magnetic field on the electrons transport (creation of an anisotropy) and thus on the resitivity.The last diagnotic allowing the estimation of the magnetic field detailed in this thesis is the proton deectometry. itallows to observe the deviation of a proton beam during its propagation under the inuence of electric and magneticfields. Other experiments were focused on the fast electron beam divergence. Two main diagnotics were used : the K α imaging and the coherent transition radiation (C.T.R) imaging at the rear side of solid targets. These diagnoticsallowed to estimate the fast electron beam divergence for two distinct energetic electron populations. The differenceof divergence coming from characteristics of both diagnotics (electrons in charge of the emissions in different energies). The diagnotics of on-surface magnetic fields of target irradiated by intense laser, such as the technics of polarimetry and crossed interferometry developed in this thesis, are dedicated to be combined with diagnotics determining the evolution of the radial size of the fast electron beam generated by the laser-matter interaction. Their simultaneous use, and the correlation between their respective data, should allow to establish experimentally, in the short term, the inuence of the on-surface magnetic fields on the fast electron beam initial characteristics, in particular the angular and energy distributions. Our results of polarimetry on the spatio-temporal evolution of the magnetic fields of surface establish the state of the art for this type of measures. There are possible improvements, in particular as regards their use in conditions of irradiation by lasers of intensities > 1018 W/cm2. These perspectives are also the object of discussions in this manuscript.Cette thèse porte sur la caractérisation des champs magnétiques générés par l'interaction entre un laser d'intensité de 1017 W/cm2 à 1018 W/cm2 et de cibles solides, et leurs effets sur le faisceau d'électrons chauds. En effet, les différents champs magnétiques créés lors de cette interaction ont un rôle fondamental sur les caractéristiques du faisceau d'électrons chauds : sa source et son transport dans la matière. Des diagnostics de polarimétrie et d'interférométrie croisée ont été développés lors de cette thèse pour observer le champ magnétique en surface de la cible irradiée par laser et en particulier leurs évolutions spatiale et temporelle. Deux différents régimes ont été observés selon le contraste en intensité de l'impulsion laser : un possédant une montée rapide de champ magnétique suivie d'une décroissance plus lente créées par le déplacement des électrons chauds dans la matière, et un possédant une croissance plus lente de forme logarithmique créée par la pré-impulsion du laser par effet thermoélectrique. L'interprétation de nos résultats obtenues par ces diagnostics ont permis d'évaluer la résistivité du plasma. Cette résistivité nommée anormale dans la littérature se comprend en estimant l'influence du champ magnétique sur l'anisotropie du transport des électrons et donc sur la résistivité. Le dernier diagnostic permettant l'estimation du champ magnétique détaillé dans cette thèse est la déflectométrie protonique. Elle permet d'observer la déviation d'un faisceau de protons lors de sa propagation sous l'effet de champs électrique et magnétique. D'autres expériences se sont focalisées sur la divergence de ce faisceau d'électrons. Deux diagnostics principaux ont été utilisés : l'imagerie K α et l'imagerie du rayonnement de transition cohérente (C.T.R.) en face arrière de cibles

Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration

Intrinsically motivated goal exploration algorithms enable machines to discover repertoires of policies that produce a diversity of effects in complex environments. These exploration algorithms have been shown to allow real world robots to acquire skills such as tool use in high-dimensional continuous state and action spaces. However, they have so far assumed that self-generated goals are sampled in a specifically engineered feature space, limiting their autonomy. In this work, we propose to use deep representation learning algorithms to learn an adequate goal space. This is a developmental 2-stage approach: first, in a perceptual learning stage, deep learning algorithms use passive raw sensor observations of world changes to learn a corresponding latent space; then goal exploration happens in a second stage by sampling goals in this latent space. We present experiments where a simulated robot arm interacts with an object, and we show that exploration algorithms using such learned representations can match the performance obtained using engineered representations

Modular Active Curiosity-Driven Discovery of Tool Use

International audienceThis article studies algorithms used by a learner to explore high-dimensional structured sensorimotor spaces such as in tool use discovery. In particular, we consider goal babbling architectures that were designed to explore and learn solutions to fields of sensorimotor problems, i.e. to acquire inverse models mapping a space of parameterized sensorimotor problems/effects to a corresponding space of parameterized motor primitives. However, so far these architectures have not been used in high-dimensional spaces of effects. Here, we show the limits of existing goal babbling architectures for efficient exploration in such spaces, and introduce a novel exploration architecture called Model Babbling (MB). MB exploits efficiently a modular representation of the space of parameterized problems/effects. We also study an active version of Model Babbling (the MACOB architecture). These architectures are compared in a simulated experimental setup with an arm that can discover and learn how to move objects using two tools with different properties, embedding structured high-dimensional continuous motor and sensory spaces