Reinforcement Learning: A Survey

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.Comment: See http://www.jair.org/ for any accompanying file

Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning

Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems. This paper introduces a novel framework, Logic-LM, which integrates LLMs with symbolic solvers to improve logical problem-solving. Our method first utilizes LLMs to translate a natural language problem into a symbolic formulation. Afterward, a deterministic symbolic solver performs inference on the formulated problem. We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations. We demonstrate Logic-LM's effectiveness on five logical reasoning datasets: ProofWriter, PrOntoQA, FOLIO, LogicalDeduction, and AR-LSAT. On average, Logic-LM achieves a significant performance boost of 39.2% over using LLM alone with standard prompting and 18.4% over LLM with chain-of-thought prompting. Our findings suggest that Logic-LM, by combining LLMs with symbolic logic, offers a promising avenue for faithful logical reasoning. Code and data are publicly available at https://github.com/teacherpeterpan/Logic-LLM.Comment: EMNLP 2023 (Findings, long paper

Dagstuhl News January - December 2002

"Dagstuhl News" is a publication edited especially for the members of the Foundation "Informatikzentrum Schloss Dagstuhl" to thank them for their support. The News give a summary of the scientific work being done in Dagstuhl. Each Dagstuhl Seminar is presented by a small abstract describing the contents and scientific highlights of the seminar as well as the perspectives or challenges of the research topic

A Software Design Pattern Based Approach to Auto Dynamic Difficulty in Video Games

From the point of view of skill levels, reflex speeds, hand-eye coordination, tolerance for frustration, and motivations, video game players may vary drastically. Auto dynamic difficulty (ADD) in video games refers to the technique of automatically adjusting different aspects of a video game in real time, based on the player’s ability and emergence factors in order to provide the optimal experience to users from such a large demography and increase replay value. In this thesis, we describe a collection of software design patterns for enabling auto dynamic difficulty in video games. We also discuss the benefits of a design pattern based approach in terms of software quality factors and process improvements based on our experience of applying it in three different video games. Additionally, we present a semi-automatic framework to assist in applying our design pattern based approach in video games. Finally, we conducted a preliminary user study where a Post-Degree Diploma student at the University of Western Ontario applied the design pattern based approach to create ADD in two arcade style games

Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language

A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts. We introduce a model of inductive learning that seeks to be human-like in that sense. It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood. By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.Comment: NeurIPS 2023 ora

MemCA: all-memristor design for deterministic and probabilistic cellular automata hardware realization

© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksInspired by the behavior of natural systems, Cellular Automata (CA) tackle the demanding long-distance information transfer of conventional computers by the massive parallel computation performed by a set of locally-coupled dynamical nodes. Although CA are envisioned as powerful deterministic computers, their intrinsic capabilities are expanded after the memristor’s probabilistic switching is introduced into CA cells, resulting in new hybrid deterministic and probabilistic memristor-based CA (MemCA). In the proposed MemCA hardware realization, memristor devices are incorporated in both the cell and rule modules, composing the very first all-memristor CA hardware, designed with mixed CMOS/Memristor circuits. The proposed implementation accomplishes high operating speed and reduced area requirements, exploiting also memristor as an entropy source in every CA cell. MemCA’s functioning is showcased in deterministic and probabilistic operation, which can be externally modified by the selection of programming voltage amplitude, without changing the design. Also, the proposed MemCA system includes a reconfigurable rule module implementation that allows for spatial and temporal rule inhomogeneity.Peer ReviewedPostprint (published version

SCRATCH LANGUAGE OF PROGRAMMING VS ENGLISH LANGUAGE: COMPARING MATHEMATICAL AND LINGUISTIC FEATURES

This paper focuses on Scratch language of programming and traces its math and linguistic features. From a complex consideration about Scratch language programming in linguistic paradigm, focusing on structural, semantic and syntactic features and logic of its narration, this research attempts to clarify specifics of the language and correlate it with the English language features. Global integration of ideas and sciences underline the crucial importance of programming and language conglomerate. Human-computer interfaces, software systems, and development of various programming languages depend on well-balanced structure, shape, logic, and appearance of the actual code. Dynamic characteristics of the Scratch programming environment sustain the creation of interactive and media-rich projects. Ad expansion of Scratch for mediation of animated stories, music videos, science projects, tutorials, and other contents necessitates multifaceted analysis of this programming environment and evokes the interest of researching Scratch from the math and linguistic perspective as one possible projection on various aspects of the considered programming language