Integration of social and economic information drives cooperation in a collective decision making task.

Social decision-making presents arguably the most complex problem an animal can face. Collective, economic decision-making requires the integration of predictions based on the outcomes of prior interactions alongside predictions generated from ongoing social information. Many economic decisions are made as individuals interact with each other, however how the manner in which animals perceive and display social information affects economic decisions remains largely overlooked. Hence we developed a social dilemma task, traditionally focused on how experienced outcomes affect choices, but allow each rat player access to proximate social information.(...

New Paradigms for Active Learning

In traditional active learning, learning algorithms (or learners) mainly focus on the performance of the final model built and the total number of queries needed for learning a good model. However, in many real-world applications, active learners have to focus on the learning process for achieving finer goals, such as minimizing the number of mistakes in predicting unlabeled examples. These learning goals are common and important in real-world applications. For example, in direct marketing, a sales agent (learner) has to focus on the process of selecting customers to approach, and tries to make correct predictions (i.e., fewer mistakes) on the customers who will buy the product. However, traditional active learning algorithms cannot achieve the finer learning goals due to the different focuses. In this thesis, we study how to control the learning process in active learning such that those goals can be accomplished. According to various learning tasks and goals, we address four new active paradigms as follows. The first paradigm is learning actively and conservatively. Under this paradigm, the learner actively selects and predicts the most certain example (thus, conservatively) iteratively during the learning process. The goal of this paradigm is to minimize the number of mistakes in predicting unlabeled examples during active learning. Intuitively the conservative strategy is less likely to make mistakes, i.e., more likely to achieve the learning goal. We apply this new learning strategy in an educational software, as well as direct marketing. The second paradigm is learning actively and aggressively. Under this paradigm, unlabeled examples and multiple oracles are available. The learner actively selects the best multiple oracles to label the most uncertain example (thus, aggressively) iteratively during the learning process. The learning goal is to learn a good model with guaranteed label quality. The third paradigm is learning actively with conservative-aggressive tradeoff. Under this learning paradigm, firstly, unlabeled examples are available and learners are allowed to select examples actively to learn. Secondly, to obtain the labels, two actions can be considered: querying oracles and making predictions. Lastly, cost has to be paid for querying oracles or for making wrong predictions. The tradeoff between the two actions is necessary for achieving the learning goal: minimizing the total cost for obtaining the labels. The last paradigm is learning actively with minimal/maximal effort. Under this paradigm, the labels of the examples are all provided and learners are allowed to select examples actively to learn. The learning goal is to control the learning process by selecting examples actively such that the learning can be accomplished with minimal effort or a good model can be built fast with maximal effort. For each of the four learning paradigms, we propose effective learning algorithms accordingly and demonstrate empirically that related learning problems in real applications can be solved well and the learning goals can be accomplished. In summary, this thesis focuses on controlling the learning process to achieve fine goals in active learning. According to various real application tasks, we propose four novel learning paradigms, and for each paradigm we propose efficient learning algorithms to solve the learning problems. The experimental results show that our learning algorithms outperform other state-of-the-art learning algorithms

Efficient instance and hypothesis space revision in Meta-Interpretive Learning

Inductive Logic Programming (ILP) is a form of Machine Learning. The goal of ILP is to induce hypotheses, as logic programs, that generalise training examples. ILP is characterised by a high expressivity, generalisation ability and interpretability. Meta-Interpretive Learning (MIL) is a state-of-the-art sub-field of ILP. However, current MIL approaches have limited efficiency: the sample and learning complexity respectively are polynomial and exponential in the number of clauses. My thesis is that improvements over the sample and learning complexity can be achieved in MIL through instance and hypothesis space revision. Specifically, we investigate 1) methods that revise the instance space, 2) methods that revise the hypothesis space and 3) methods that revise both the instance and the hypothesis spaces for achieving more efficient MIL. First, we introduce a method for building training sets with active learning in Bayesian MIL. Instances are selected maximising the entropy. We demonstrate this method can reduce the sample complexity and supports efficient learning of agent strategies. Second, we introduce a new method for revising the MIL hypothesis space with predicate invention. Our method generates predicates bottom-up from the background knowledge related to the training examples. We demonstrate this method is complete and can reduce the learning and sample complexity. Finally, we introduce a new MIL system called MIGO for learning optimal two-player game strategies. MIGO learns from playing: its training sets are built from the sequence of actions it chooses. Moreover, MIGO revises its hypothesis space with Dependent Learning: it first solves simpler tasks and can reuse any learned solution for solving more complex tasks. We demonstrate MIGO significantly outperforms both classical and deep reinforcement learning. The methods presented in this thesis open exciting perspectives for efficiently learning theories with MIL in a wide range of applications including robotics, modelling of agent strategies and game playing.Open Acces

SiMAMT: A Framework for Strategy-Based Multi-Agent Multi-Team Systems

Multi-agent multi-team systems are commonly seen in environments where hierarchical layers of goals are at play. For example, theater-wide combat scenarios where multiple levels of command and control are required for proper execution of goals from the general to the foot soldier. Similar structures can be seen in game environments, where agents work together as teams to compete with other teams. The different agents within the same team must, while maintaining their own ‘personality’, work together and coordinate with each other to achieve a common team goal. This research develops strategy-based multi-agent multi-team systems, where strategy is framed as an instrument at the team level to coordinate the multiple agents of a team in a cohesive way. A formal specification of strategy and strategy-based multi-agent multi-team systems is provided. A framework is developed called SiMAMT (strategy- based multi-agent multi-team systems). The different components of the framework, including strategy simulation, strategy inference, strategy evaluation, and strategy selection are described. A graph-matching approximation algorithm is also developed to support effective and efficient strategy inference. Examples and experimental results are given throughout to illustrate the proposed framework, including each of its composite elements, and its overall efficacy. This research make several contributions to the field of multi-agent multi-team systems: a specification for strategy and strategy-based systems, and a framework for implementing them in real-world, interactive-time scenarios; a robust simulation space for such complex and intricate interaction; an approximation algorithm that allows for strategy inference within these systems in interactive-time; experimental results that verify the various sub-elements along with a full-scale integration experiment showing the efficacy of the proposed framework

Deep Learning Techniques for Music Generation -- A Survey

This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content. We propose a methodology based on five dimensions for our analysis: Objective - What musical content is to be generated? Examples are: melody, polyphony, accompaniment or counterpoint. - For what destination and for what use? To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file). Representation - What are the concepts to be manipulated? Examples are: waveform, spectrogram, note, chord, meter and beat. - What format is to be used? Examples are: MIDI, piano roll or text. - How will the representation be encoded? Examples are: scalar, one-hot or many-hot. Architecture - What type(s) of deep neural network is (are) to be used? Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks. Challenge - What are the limitations and open challenges? Examples are: variability, interactivity and creativity. Strategy - How do we model and control the process of generation? Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation. For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described and are used to exemplify the various choices of objective, representation, architecture, challenge and strategy. The last section includes some discussion and some prospects.Comment: 209 pages. This paper is a simplified version of the book: J.-P. Briot, G. Hadjeres and F.-D. Pachet, Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Systems, Springer, 201

Apprentissage permanent par feedback endogène, application à un système robotique

Les applications robotiques sont liées à l'environnement sociotechnique dynamique dans lequel elles sont intégrées. Dans ce contexte, l'auto-adaptation est une préoccupation centrale et la conception d'applications intelligentes dans de tels environnements nécessite de les considérer comme des systèmes complexes. Le domaine de la robotique est très vaste. L'accent est mis sur les systèmes qui s'adaptent aux contraintes de leur environnement et non sur la mécanique ou le traitement du signal. À la lumière de ce contexte, l'objectif de cette thèse est la conception d'un mécanisme d'apprentissage capable d'apprendre de manière continue en utilisant des feedbacks endogènes (i.e. des interactions internes) dans des environnements sociotechniques dynamiques. Ce mécanisme d'apprentissage doit aussi vérifier plusieurs propriétés qui sont essentielles dans ce contexte comme : l'agnosticité, l'apprentissage tout au long de la vie, l'apprentissage en ligne, l'auto-observation, la généralisation des connaissances, le passage à l'échelle, la tolérance au volume de données et l'explicabilité. Les principales contributions consistent en la construction de l'apprentissage endogène par contextes et la conception du mécanisme d'apprentissage ELLSA pour Endogenous Lifelong Learner by Self-Adaptation. Le mécanisme d'apprentissage proposé est basé sur les systèmes multi-agents adaptatifs combinés à l'apprentissage endogène par contextes. La création de l'apprentissage endogène par contextes est motivée par la caractérisation d'imprécisions d'apprentissage qui sont détectées par des négociations locales entre agents. L'apprentissage endogène par contextes comprends aussi un mécanisme de génération de données artificielles pour améliorer les modèles d'apprentissage tout en réduisant la quantité nécessaire de données d'apprentissage. Dans un contexte d'apprentissage tout au long de la vie, ELLSA permet une mise à jour dynamique des modèles d'apprentissage. Il introduit des stratégies d'apprentissage actif et d'auto-apprentissage pour résoudre les imprécisions d'apprentissage. L'utilisation de ces stratégies dépend de la disponibilité des données d'apprentissage. Afin d'évaluer ses contributions, ce mécanisme est appliqué à l'apprentissage de fonctions mathématiques et à un problème réel dans le domaine de la robotique : le problème de la cinématique inverse. Le scénario d'application est l'apprentissage du contrôle de bras robotiques multi-articulés. Les expériences menées montrent que l'apprentissage endogène par contextes permet d'améliorer les performances d'apprentissage grâce à des mécanismes internes. Elles mettent aussi en évidence des propriétés du système selon les objectifs de la thèse : feedback endogènes, agnosticité, apprentissage tout au long de la vie, apprentissage en ligne, auto-observation, généralisation, passage à l'échelle, tolérance au volume de données et explicabilité.Robotic applications are linked to the dynamic sociotechnical environment in which they are embedded. In this scope, self-adaptation is a central concern and the design of intelligent applications in such environments requires to consider them as complex systems. The field of robotics is very broad. The focus is made on systems that adapt to the constraints of their environment and not on mechanics or signal processing. In light of this context, the objective of this thesis is the design of a learning mechanism capable of continuous learning using endogenous feedback (i.e. internal interactions) in dynamic sociotechnical environments. This learning mechanism must also verify several properties that are essential in this context such as: agnosticity, lifelong learning, online learning, self-observation, knowledge generalization, scalability, data volume tolerance and explainability. The main contributions consist of the construction of Endogenous Context Learning and the design of the learning mechanism ELLSA for Endogenous Lifelong Learner by Self-Adaptation. The proposed learning mechanism is based on Adaptive Multi-Agent Systems combined with Context Learning. The creation of Endogenous Context Learning is motivated by the characterization of learning inaccuracies that are detected by local negotiations between agents. Endogenous Context Learning also includes an artificial data generation mechanism to improve learning models while reducing the amount of the required learning data. In a Lifelong Learning setting, ELLSA enables dynamic updating of learning models. It introduces Active Learning and Self-Learning strategies to resolve learning inaccuracies. The use of these strategies depends on the availability of learning data. In order to evaluate its contributions, this mechanism is applied to the learning of mathematical functions and to a real problem in the field of robotics: the Inverse Kinematics problem. The application scenario is the learning of the control of multi-jointed robotic arms. The conducted experiments show that Endogenous Context Learning enables to improve the learning performances thanks to internal mechanisms. They also highlight the properties of the system according to the objectives of the thesis: endogenous feedback, agnosticity, lifelong learning, online learning, self-observation, knowledge generalization, scalability, data volume tolerance and explainability

Towards Real-World Data Streams for Deep Continual Learning

Continual Learning deals with Artificial Intelligent agents striving to learn from an ever-ending stream of data. Recently, Deep Continual Learning focused on the design of new strategies to endow Artificial Neural Networks with the ability to learn continuously without forgetting previous knowledge. In fact, the learning process of any Artificial Neural Network model is well-known to lack the sufficient stability to preserve existing knowledge when learning new information. This phenomenon, called catastrophic forgetting or simply forgetting, is considered one of the main obstacles for the design of effective Continual Learning agents. However, existing strategies designed to mitigate forgetting have been evaluated on a restricted set of Continual Learning scenarios. The most used one is, by far, the Class-Incremental scenario applied on object detection tasks. Even though it drove interest in Continual Learning, Class-Incremental scenarios strongly constraint the properties of the data stream, thus limiting its ability to model real-world environments. The core of this thesis concerns the introduction of three Continual Learning data streams, whose design is centered around specific real-world environments properties. First, we propose the Class- Incremental with Repetition scenario, which builds a data stream including both the introduction of new concepts and the repetition of previous ones. Repetition is naturally present in many environments and it constitutes an important source of information. Second, we formalize the Continual Pre-Training scenario, which leverages a data stream of unstructured knowledge to keep a pre-trained model updated over time. One important objective of this scenario is to study how to continuously build general, robust representations that does not strongly depend on the specific task to be solved. This is a fundamental property of real-world agents, which build cross-task knowledge and then adapts it to specific needs. Third, we study Continual Learning scenarios where data streams are composed by temporally-correlated data. Temporal correlation is ubiquitous and lies at the foundation of most environments we, as humans, experience during our life. We leverage Recurrent Neural Networks as our main model, due to their intrinsic ability to model temporal correlations. We discovered that, when applied to recurrent models, Continual Learning strategies behave in an unexpected manner. This highlights the limits of the current experimental validation, mostly focused on Computer Vision tasks. Ultimately, the introduction of new data streams contributed to deepen our understanding of how Artificial Neural Networks learn continuously. We discover that forgetting strongly depends on the properties of the data stream and we observed large changes from one data stream to another. Moreover, when forgetting is mild, we were able to effectively mitigate it with simple strategies, or even without any specific ones. Loosening the focus on forgetting allows us to turn our attention to other interesting problems, outlined in this thesis, like (i) separation between continual representation learning and quick adaptation to novel tasks, (ii) robustness to unbalanced data streams and (iii) ability to continuously learn temporal correlations. These objectives currently defy existing strategies and will likely represent the next challenge for Continual Learning research