36,680 research outputs found
Learning Representations in Model-Free Hierarchical Reinforcement Learning
Common approaches to Reinforcement Learning (RL) are seriously challenged by
large-scale applications involving huge state spaces and sparse delayed reward
feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address
this scalability issue by learning action selection policies at multiple levels
of temporal abstraction. Abstraction can be had by identifying a relatively
small set of states that are likely to be useful as subgoals, in concert with
the learning of corresponding skill policies to achieve those subgoals. Many
approaches to subgoal discovery in HRL depend on the analysis of a model of the
environment, but the need to learn such a model introduces its own problems of
scale. Once subgoals are identified, skills may be learned through intrinsic
motivation, introducing an internal reward signal marking subgoal attainment.
In this paper, we present a novel model-free method for subgoal discovery using
incremental unsupervised learning over a small memory of the most recent
experiences (trajectories) of the agent. When combined with an intrinsic
motivation learning mechanism, this method learns both subgoals and skills,
based on experiences in the environment. Thus, we offer an original approach to
HRL that does not require the acquisition of a model of the environment,
suitable for large-scale applications. We demonstrate the efficiency of our
method on two RL problems with sparse delayed feedback: a variant of the rooms
environment and the first screen of the ATARI 2600 Montezuma's Revenge game
Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior
Previous research using evolutionary computation in Multi-Agent Systems
indicates that assigning fitness based on team vs.\ individual behavior has a
strong impact on the ability of evolved teams of artificial agents to exhibit
teamwork in challenging tasks. However, such research only made use of
single-objective evolution. In contrast, when a multiobjective evolutionary
algorithm is used, populations can be subject to individual-level objectives,
team-level objectives, or combinations of the two. This paper explores the
performance of cooperatively coevolved teams of agents controlled by artificial
neural networks subject to these types of objectives. Specifically, predator
agents are evolved to capture scripted prey agents in a torus-shaped grid
world. Because of the tension between individual and team behaviors, multiple
modes of behavior can be useful, and thus the effect of modular neural networks
is also explored. Results demonstrate that fitness rewarding individual
behavior is superior to fitness rewarding team behavior, despite being applied
to a cooperative task. However, the use of networks with multiple modules
allows predators to discover intelligent behavior, regardless of which type of
objectives are used
Computing data for Levin-Wen with defects
We demonstrate how to do many computations for non-chiral topological phases
with defects. These defects may be 1-dimensional domain walls or 0-dimensional
point defects.
Using as a guiding example, we demonstrate how
domain wall fusion and associators can be computed using generalized tube
algebra techniques. These domain walls can be both between distinct or
identical phases. Additionally, we show how to compute all possible point
defects, and the fusion and associator data of these. Worked examples,
tabulated data and Mathematica code are provided.Comment: 17+25 pages, many tables and attached cod
Anomalies and entanglement renormalization
We study 't Hooft anomalies of discrete groups in the framework of
(1+1)-dimensional multiscale entanglement renormalization ansatz states on the
lattice. Using matrix product operators, general topological restrictions on
conformal data are derived. An ansatz class allowing for optimization of MERA
with an anomalous symmetry is introduced. We utilize this class to numerically
study a family of Hamiltonians with a symmetric critical line. Conformal data
is obtained for all irreducible projective representations of each anomalous
symmetry twist, corresponding to definite topological sectors. It is
numerically demonstrated that this line is a protected gapless phase. Finally,
we implement a duality transformation between a pair of critical lines using
our subclass of MERA.Comment: 12+18 pages, 6+5 figures, 0+2 tables, v2 published versio
Fusing Binary Interface Defects in Topological Phases: The case
A binary interface defect is any interface between two (not necessarily
invertible) domain walls. We compute all possible binary interface defects in
Kitaev's model and all possible fusions between them.
Our methods can be applied to any Levin-Wen model. We also give physical
interpretations for each of the defects in the model.
These physical interpretations provide a new graphical calculus which can be
used to compute defect fusion.Comment: 27+10 pages, 2+5 tables, comments welcom
Tensor Networks with a Twist: Anyon-permuting domain walls and defects in PEPS
We study the realization of anyon-permuting symmetries of topological phases
on the lattice using tensor networks. Working on the virtual level of a
projected entangled pair state, we find matrix product operators (MPOs) that
realize all unitary topological symmetries for the toric and color codes. These
operators act as domain walls that enact the symmetry transformation on anyons
as they cross. By considering open boundary conditions for these domain wall
MPOs, we show how to introduce symmetry twists and defect lines into the state.Comment: 11 pages, 6 figures, 2 appendices, v2 published versio
- …