17,810 research outputs found
A literature survey of low-rank tensor approximation techniques
During the last years, low-rank tensor approximation has been established as
a new tool in scientific computing to address large-scale linear and
multilinear algebra problems, which would be intractable by classical
techniques. This survey attempts to give a literature overview of current
developments in this area, with an emphasis on function-related tensors
Reinforcement Learning: A Survey
This paper surveys the field of reinforcement learning from a
computer-science perspective. It is written to be accessible to researchers
familiar with machine learning. Both the historical basis of the field and a
broad selection of current work are summarized. Reinforcement learning is the
problem faced by an agent that learns behavior through trial-and-error
interactions with a dynamic environment. The work described here has a
resemblance to work in psychology, but differs considerably in the details and
in the use of the word ``reinforcement.'' The paper discusses central issues of
reinforcement learning, including trading off exploration and exploitation,
establishing the foundations of the field via Markov decision theory, learning
from delayed reinforcement, constructing empirical models to accelerate
learning, making use of generalization and hierarchy, and coping with hidden
state. It concludes with a survey of some implemented systems and an assessment
of the practical utility of current methods for reinforcement learning.Comment: See http://www.jair.org/ for any accompanying file
Verifiable Reinforcement Learning via Policy Extraction
While deep reinforcement learning has successfully solved many challenging
control tasks, its real-world applicability has been limited by the inability
to ensure the safety of learned policies. We propose an approach to verifiable
reinforcement learning by training decision tree policies, which can represent
complex policies (since they are nonparametric), yet can be efficiently
verified using existing techniques (since they are highly structured). The
challenge is that decision tree policies are difficult to train. We propose
VIPER, an algorithm that combines ideas from model compression and imitation
learning to learn decision tree policies guided by a DNN policy (called the
oracle) and its Q-function, and show that it substantially outperforms two
baselines. We use VIPER to (i) learn a provably robust decision tree policy for
a variant of Atari Pong with a symbolic state space, (ii) learn a decision tree
policy for a toy game based on Pong that provably never loses, and (iii) learn
a provably stable decision tree policy for cart-pole. In each case, the
decision tree policy achieves performance equal to that of the original DNN
policy
Recommended from our members
Computation and Learning in High Dimensions (hybrid meeting)
The most challenging problems in science often involve the learning and
accurate computation of high dimensional functions.
High-dimensionality is a typical feature for a multitude of problems
in various areas of science.
The so-called curse of dimensionality typically negates the use of
traditional numerical techniques for the solution of
high-dimensional problems. Instead, novel theoretical and
computational approaches need to be developed to make them tractable
and to capture fine resolutions and relevant features. Paradoxically,
increasing computational power may even serve to heighten this demand,
since the wealth of new computational data itself becomes a major
obstruction. Extracting essential information from complex
problem-inherent structures and developing rigorous models to quantify
the quality of information in a high-dimensional setting pose
challenging tasks from both theoretical and numerical perspective.
This has led to the emergence of several new computational methodologies,
accounting for the fact that by now well understood methods drawing on
spatial localization and mesh-refinement are in their original form no longer viable.
Common to these approaches is the nonlinearity of the solution method.
For certain problem classes, these methods have
drastically advanced the frontiers of computability.
The most visible of these new methods is deep learning. Although the use of deep neural
networks has been extremely successful in certain
application areas, their mathematical understanding is far from complete.
This workshop proposed to deepen the understanding of
the underlying mathematical concepts that drive this new evolution of
computational methods and to promote the exchange of ideas emerging in various
disciplines about how to treat multiscale and high-dimensional problems
- …