805 research outputs found
Reinforcement Learning via AIXI Approximation
This paper introduces a principled approach for the design of a scalable
general reinforcement learning agent. This approach is based on a direct
approximation of AIXI, a Bayesian optimality notion for general reinforcement
learning agents. Previously, it has been unclear whether the theory of AIXI
could motivate the design of practical algorithms. We answer this hitherto open
question in the affirmative, by providing the first computationally feasible
approximation to the AIXI agent. To develop our approximation, we introduce a
Monte Carlo Tree Search algorithm along with an agent-specific extension of the
Context Tree Weighting algorithm. Empirically, we present a set of encouraging
results on a number of stochastic, unknown, and partially observable domains.Comment: 8 LaTeX pages, 1 figur
Context tree switching
This paper describes the Context Tree Switching technique, a modification of Context Tree
Weighting for the prediction of binary, stationary, n-Markov sources. By modifying Context
Tree Weighting’s recursive weighting scheme, it is possible to mix over a strictly larger class of
models without increasing the asymptotic time or space complexity of the original algorithm.
We prove that this generalization preserves the desirable theoretical properties of Context Tree
Weighting on stationary n-Markov sources, and show empirically that this new technique leads
to consistent improvements over Context Tree Weighting as measured on the Calgary Corpus
Reinforcement Learning via AIXI Approximation
This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. This approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforcement learning agents. Previously, it has been unclear whether the theory of AIXI could motivate the design of practical algorithms. We answer this hitherto open question in the affirmative, by providing the first computationally feasible approximation to the AIXI agent. To develop our approximation, we introduce a Monte Carlo Tree Search algorithm along with an agent-specific extension of the Context Tree Weighting algorithm. Empirically, we present a set of encouraging results on a number of stochastic, unknown, and partially observable domains
Structural Color 3D Printing By Shrinking Photonic Crystals
The rings, spots and stripes found on some butterflies, Pachyrhynchus
weevils, and many chameleons are notable examples of natural organisms
employing photonic crystals to produce colorful patterns. Despite advances in
nanotechnology, we still lack the ability to print arbitrary colors and shapes
in all three dimensions at this microscopic length scale. Commercial nanoscale
3D printers based on two-photon polymerization are incapable of patterning
photonic crystal structures with the requisite ~300 nm lattice constant to
achieve photonic stopbands/ bandgaps in the visible spectrum and generate
colors. Here, we introduce a means to produce 3D-printed photonic crystals with
a 5x reduction in lattice constants (periodicity as small as 280 nm), achieving
sub-100-nm features with a full range of colors. The reliability of this
process enables us to engineer the bandstructures of woodpile photonic crystals
that match experiments, showing that observed colors can be attributed to
either slow light modes or stopbands. With these lattice structures as 3D color
volumetric elements (voxels), we printed 3D microscopic scale objects,
including the first multi-color microscopic model of the Eiffel Tower measuring
only 39-microns tall with a color pixel size of 1.45 microns. The technology to
print 3D structures in color at the microscopic scale promises the direct
patterning and integration of spectrally selective devices, such as photonic
crystal-based color filters, onto free-form optical elements and curved
surfaces
Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
We focus on the challenge of finding a diverse collection of quality
solutions on complex continuous domains. While quality diver-sity (QD)
algorithms like Novelty Search with Local Competition (NSLC) and MAP-Elites are
designed to generate a diverse range of solutions, these algorithms require a
large number of evaluations for exploration of continuous spaces. Meanwhile,
variants of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are
among the best-performing derivative-free optimizers in single-objective
continuous domains. This paper proposes a new QD algorithm called Covariance
Matrix Adaptation MAP-Elites (CMA-ME). Our new algorithm combines the
self-adaptation techniques of CMA-ES with archiving and mapping techniques for
maintaining diversity in QD. Results from experiments based on standard
continuous optimization benchmarks show that CMA-ME finds better-quality
solutions than MAP-Elites; similarly, results on the strategic game Hearthstone
show that CMA-ME finds both a higher overall quality and broader diversity of
strategies than both CMA-ES and MAP-Elites. Overall, CMA-ME more than doubles
the performance of MAP-Elites using standard QD performance metrics. These
results suggest that QD algorithms augmented by operators from state-of-the-art
optimization algorithms can yield high-performing methods for simultaneously
exploring and optimizing continuous search spaces, with significant
applications to design, testing, and reinforcement learning among other
domains.Comment: Accepted to GECCO 202
The UN Sustainable Development Goals (SDGs): Contributions from the Humanities
It is widely agreed that achieving the United Nations Sustainable Development Goals requires the insights, knowledge and comparative perspectives of Humanities disciplines. The UNESCO Hangzhou Declaration of 2013 highlighted the importance of culture as ‘an enabler and driver of sustainable development’. Yet the Humanities have so far featured relatively little in work on the SDGs
An Empirical Evaluation of Deep Learning on Highway Driving
Numerous groups have applied a variety of deep learning techniques to
computer vision problems in highway perception scenarios. In this paper, we
presented a number of empirical evaluations of recent deep learning advances.
Computer vision, combined with deep learning, has the potential to bring about
a relatively inexpensive, robust solution to autonomous driving. To prepare
deep learning for industry uptake and practical applications, neural networks
will require large data sets that represent all possible driving environments
and scenarios. We collect a large data set of highway data and apply deep
learning and computer vision algorithms to problems such as car and lane
detection. We show how existing convolutional neural networks (CNNs) can be
used to perform lane and vehicle detection while running at frame rates
required for a real-time system. Our results lend credence to the hypothesis
that deep learning holds promise for autonomous driving.Comment: Added a video for lane detectio
- …