Learning the Structure of Continuous Markov Decision Processes

There is growing interest in artificial, intelligent agents which can operate autonomously for an extended period of time in complex environments and fulfill a variety of different tasks. Such agents will face different problems during their lifetime which may not be foreseeable at the time of their deployment. Thus, the capacity for lifelong learning of new behaviors is an essential prerequisite for this kind of agents as it enables them to deal with unforeseen situations. However, learning every complex behavior anew from scratch would be cumbersome for the agent. It is more plausible to consider behavior to be modular and let the agent acquire a set of reusable building blocks for behavior, the so-called skills. These skills might, once acquired, facilitate fast learning and adaptation of behavior to new situations. This work focuses on computational approaches for skill acquisition, namely which kind of skills shall be acquired and how to acquire them. The former is commonly denoted as skill discovery and the latter as skill learning . The main contribution of this thesis is a novel incremental skill acquisition approach which is suited for lifelong learning. In this approach, the agent learns incrementally a graph-based representation of a domain and exploits certain properties of this graph such as its bottlenecks for skill discovery. This thesis proposes a novel approach for learning a graph-based representation of continuous domains based on formalizing the problem as a probabilistic generative model. Furthermore, a new incremental agglomerative clustering approach for identifying bottlenecks of such graphs is presented. Thereupon, the thesis proposes a novel intrinsic motivation system which enables an agent to intelligently allocate time between skill discovery and skill learning in developmental settings, where the agent is not constrained by external tasks. The results of this thesis show that the resulting skill acquisition approach is suited for continuous domains and can deal with domain stochasticity and different explorative behavior of the agent. The acquired skills are reusable and versatile and can be used in multi-task and lifelong learning settings in high-dimensional problems

Learning plan selection for BDI agent systems

Belief-Desire-Intention (BDI) is a popular agent-oriented programming approach for developing robust computer programs that operate in dynamic environments. These programs contain pre-programmed abstract procedures that capture domain know-how, and work by dynamically applying these procedures, or plans, to different situations that they encounter. Agent programs built using the BDI paradigm, however, do not traditionally do learning, which becomes important if a deployed agent is to be able to adapt to changing situations over time. Our vision is to allow programming of agent systems that are capable of adjusting to ongoing changes in the environment’s dynamics in a robust and effective manner. To this end, in this thesis we develop a framework that can be used by programmers to build adaptable BDI agents that can improve plan selection over time by learning from their experiences. These learning agents can dynamically adjust their choice of which plan to select in which situation, based on a growing understanding of what works and a sense of how reliable this understanding is. This reliability is given by a perceived measure of confidence, that tries to capture how well-informed the agent’s most recent decisions were and how well it knows the most recent situations that it encountered. An important focus of this work is to make this approach practical. Our framework allows learning to be integrated into BDI programs of reasonable complexity, including those that use recursion and failure recovery mechanisms. We show the usability of the framework in two complete programs: an implementation of the Towers of Hanoi game where recursive solutions must be learnt, and a modular battery system controller where the environment dynamics changes in ways that may require many learning and relearning phases

Catalysis in Quantum Information Theory

Catalysts open up new reaction pathways which can speed up chemical reactions while not consuming the catalyst. A similar phenomenon has been discovered in quantum information science, where physical transformations become possible by utilizing a (quantum) degree of freedom that remains unchanged throughout the process. In this review, we present a comprehensive overview of the concept of catalysis in quantum information science and discuss its applications in various physical contexts.Comment: Review paper; Comments and suggestions welcome

Proceedings of the 1st International Conference on Live Coding

Open Access peer reviewed papers on live coding published at the 1st International Conference on Live Coding (ICLC) in Leeds

Connected mathematics : builiding concrete relationships with mathematical knowledge

Thesis (Ph. D.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1993.Includes bibliographical references (leaves 201-209).by Uriel Jospeh Wilensky.Ph.D

Predicting companies stock price direction by using sentiment analysis of news articles

This paper summarizes our experience teaching several courses at Metropolitan College of Boston University Computer Science department over five years. A number of innovative teaching techniques are presented in this paper. We specifically address the role of a project archive, when designing a course. This research paper explores survey results from every running of courses, from 2014 to 2019. During each class, students participated in two distinct surveys: first, dealing with key learning outcomes, and, second, with teaching techniques used. This paper makes several practical recommendations based on the analysis of collected data. The research validates the value of a sound repository of technical term projects and the role such repository plays in effective teaching and learning of computer science courses.Published versio

Proof and Proving in Mathematics Education

undefine