A comparative study between motivated learning and reinforcement learning

This paper analyzes advanced reinforcement learning techniques and compares some of them to motivated learning. Motivated learning is briefly discussed indicating its relation to reinforcement learning. A black box scenario for comparative analysis of learning efficiency in autonomous agents is developed and described. This is used to analyze selected algorithms. Reported results demonstrate that in the selected category of problems, motivated learning outperformed all reinforcement learning algorithms we compared with

Learning by Asking Questions

We introduce an interactive learning framework for the development and testing of intelligent visual systems, called learning-by-asking (LBA). We explore LBA in context of the Visual Question Answering (VQA) task. LBA differs from standard VQA training in that most questions are not observed during training time, and the learner must ask questions it wants answers to. Thus, LBA more closely mimics natural learning and has the potential to be more data-efficient than the traditional VQA setting. We present a model that performs LBA on the CLEVR dataset, and show that it automatically discovers an easy-to-hard curriculum when learning interactively from an oracle. Our LBA generated data consistently matches or outperforms the CLEVR train data and is more sample efficient. We also show that our model asks questions that generalize to state-of-the-art VQA models and to novel test time distributions

Supportive and Antagonistic Behaviour in Distributed Computational Creativity via Coupled Empowerment Maximisation

Abstract There has been a strong tendency in distributed computational creativity systems to embrace embodied and situated agents for their flexible and adaptive behaviour. Intrinsically motivated agents are particularly successful in this respect, because they do not rely on externally specified goals, and can thus react flexibly to changes in open-ended environments. While supportive and antagonistic behaviour is omnipresent when people interact in creative tasks, existing implementations cannot establish such behaviour without constraining their agents' flexibility by means of explicitly specified interaction rules. More open approaches in contrast cannot guarantee that support or antagonistic behaviour ever comes about. We define the information-theoretic principle of coupled empowerment maximisation as an intrinsically motivated frame for supportive and antagonistic behaviour within which agents can interact with maximum flexibility. We provide an intuition and a formalisation for an arbitrary number of agents. We then draw on several case-studies of co-creative and social creativity systems to make detailed predictions of the potential effect the underlying empowerment maximisation principle might have on the behaviour of creative agents

Intrinsic Rewards for Maintenance, Approach, Avoidance and Achievement Goal Types

In reinforcement learning, reward is used to guide the learning process. The reward is often designed to be task-dependent, and it may require significant domain knowledge to design a good reward function. This paper proposes general reward functions for maintenance, approach, avoidance, and achievement goal types. These reward functions exploit the inherent property of each type of goal and are thus task-independent. We also propose metrics to measure an agent's performance for learning each type of goal. We evaluate the intrinsic reward functions in a framework that can autonomously generate goals and learn solutions to those goals using a standard reinforcement learning algorithm. We show empirically how the proposed reward functions lead to learning in a mobile robot application. Finally, using the proposed reward functions as building blocks, we demonstrate how compound reward functions, reward functions to generate sequences of tasks, can be created that allow the mobile robot to learn more complex behaviors

Evolving Robust Policy Coverage Sets in Multi-Objective Markov Decision Processes Through Intrinsically Motivated Self-Play

Many real-world decision-making problems involve multiple conflicting objectives that can not be optimized simultaneously without a compromise. Such problems are known as multi-objective Markov decision processes and they constitute a significant challenge for conventional single-objective reinforcement learning methods, especially when an optimal compromise cannot be determined beforehand. Multi-objective reinforcement learning methods address this challenge by finding an optimal coverage set of non-dominated policies that can satisfy any user's preference in solving the problem. However, this is achieved with costs of computational complexity, time consumption, and lack of adaptability to non-stationary environment dynamics. In order to address these limitations, there is a need for adaptive methods that can solve the problem in an online and robust manner. In this paper, we propose a novel developmental method that utilizes the adversarial self-play between an intrinsically motivated preference exploration component, and a policy coverage set optimization component that robustly evolves a convex coverage set of policies to solve the problem using preferences proposed by the former component. We show experimentally the effectiveness of the proposed method in comparison to state-of-the-art multi-objective reinforcement learning methods in stationary and non-stationary environments

Intrinsic motivation, curiosity and learning: theory and applications in educational technologies

International audienceThis article studies the bi-directional causal interactions between curiosity and learning, and discusses how understanding these interactions can be leveraged in educational technology applications. First, we review recent results showing how state curiosity, and more generally the experience of novelty and surprise, can enhance learning and memory retention. Then, we discuss how psychology and neuroscience have conceptualized curiosity and intrinsic motivation, studying how the brain can be intrinsically rewarded by novelty, complexity or other measures of information. We explain how the framework of computational reinforcement learning can be used to model such mechanisms of curiosity. Then, we discuss the learning progress (LP) hypothesis, which posits a positive feedback loop between curiosity and learning. We outline experiments with robots that show how LP-driven attention and exploration can self-organize a developmental learning curriculum scaffolding efficient acquisition of multiple skills/tasks.. Finally, we discuss recent work exploiting these conceptual and computational models in educational technologies, showing in particular how Intelligent Tutoring Systems can be designed to foster curiosity and learning

Примена виртуелних светова у истраживању теорије агената и инжењерском образовању

The focus of this doctoral dissertation is on exploring the potentials of virtual worlds, for applications in research and education. Regarding this, there are two central aspects that are explored in the dissertation. The first one considers the concept of autonomous agents, and agent theory in general, in the context of virtual worlds. The second aspect is related to the educational applications of virtual worlds, while especially focusing on the concept of virtual laboratories. An introduction to basic terminology related to the subject is given at the start of the dissertation. After that, a thorough analysis of the role of agents in virtual worlds is presented. This, among others, includes the analysis of the techniques that shape the agent’s behavior. The development of the virtual gamified educational system, specially dedicated to agents is then presented in the dissertation, along with a thorough description. While, in the end, analysis of the concept of virtual laboratories in STE (Science, Technology, and Engineering) disciplines is performed, and existing solutions are evaluated according to the criteria defined in the dissertation.Фокус ове докторске дисертације је на истраживању потенцијала виртуелних светова за примене у истраживањима и образовању. У вези са тим, постоје два главна аспекта која су обрађена у дисертацији. Први аспект се тиче концепта аутономних агената, као и теорије агената у целини, а у контексту виртуелних светова. Други аспект је везан за примену виртуелних светова у образовању, при чему је посебан акценат стављен на виртуелне лабораторије. На почетку дисертације је дат кратак увод који се тиче терминологије и појединих појмова везаних за област којом се ова дисертција бави. Након тога је представљена систематична и темељна анализа улоге агената у виртуелним световима. Између осталог, ово укључује и анализу техника потребних за обликовање понашања агената. Потом је у дисертацији детаљно представљен развој оригиналног виртуелног образовног система посвећеног агентима. На крају, анализиран је концепт виртуелних лабораторија у НТИ (наука, технологија, инжењерство) дисциплинама и извршена је евалуација постојећих решења у складу са критеријумима који су дефинисани у дисертацији

Goal Reasoning: Papers from the ACS Workshop

This technical report contains the 14 accepted papers presented at the Workshop on Goal Reasoning, which was held as part of the 2015 Conference on Advances in Cognitive Systems (ACS-15) in Atlanta, Georgia on 28 May 2015. This is the fourth in a series of workshops related to this topic, the first of which was the AAAI-10 Workshop on Goal-Directed Autonomy; the second was the Self-Motivated Agents (SeMoA) Workshop, held at Lehigh University in November 2012; and the third was the Goal Reasoning Workshop at ACS-13 in Baltimore, Maryland in December 2013