7,325 research outputs found
Projective simulation with generalization
The ability to generalize is an important feature of any intelligent agent.
Not only because it may allow the agent to cope with large amounts of data, but
also because in some environments, an agent with no generalization capabilities
cannot learn. In this work we outline several criteria for generalization, and
present a dynamic and autonomous machinery that enables projective simulation
agents to meaningfully generalize. Projective simulation, a novel, physical
approach to artificial intelligence, was recently shown to perform well in
standard reinforcement learning problems, with applications in advanced
robotics as well as quantum experiments. Both the basic projective simulation
model and the presented generalization machinery are based on very simple
principles. This allows us to provide a full analytical analysis of the agent's
performance and to illustrate the benefit the agent gains by generalizing.
Specifically, we show that already in basic (but extreme) environments,
learning without generalization may be impossible, and demonstrate how the
presented generalization machinery enables the projective simulation agent to
learn.Comment: 14 pages, 9 figure
Benchmarking projective simulation in navigation problems
Projective simulation (PS) is a model for intelligent agents with a
deliberation capacity that is based on episodic memory. The model has been
shown to provide a flexible framework for constructing reinforcement-learning
agents, and it allows for quantum mechanical generalization, which leads to a
speed-up in deliberation time. PS agents have been applied successfully in the
context of complex skill learning in robotics, and in the design of
state-of-the-art quantum experiments. In this paper, we study the performance
of projective simulation in two benchmarking problems in navigation, namely the
grid world and the mountain car problem. The performance of PS is compared to
standard tabular reinforcement learning approaches, Q-learning and SARSA. Our
comparison demonstrates that the performance of PS and standard learning
approaches are qualitatively and quantitatively similar, while it is much
easier to choose optimal model parameters in case of projective simulation,
with a reduced computational effort of one to two orders of magnitude. Our
results show that the projective simulation model stands out for its simplicity
in terms of the number of model parameters, which makes it simple to set up the
learning agent in unknown task environments.Comment: 8 pages, 10 figure
Projective simulation for artificial intelligence
We propose a model of a learning agent whose interaction with the environment
is governed by a simulation-based projection, which allows the agent to project
itself into future situations before it takes real action. Projective
simulation is based on a random walk through a network of clips, which are
elementary patches of episodic memory. The network of clips changes
dynamically, both due to new perceptual input and due to certain compositional
principles of the simulation process. During simulation, the clips are screened
for specific features which trigger factual action of the agent. The scheme is
different from other, computational, notions of simulation, and it provides a
new element in an embodied cognitive science approach to intelligent action and
learning. Our model provides a natural route for generalization to
quantum-mechanical operation and connects the fields of reinforcement learning
and quantum computation.Comment: 22 pages, 18 figures. Close to published version, with footnotes
retaine
A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems
We introduce a kind of partial observability to the projective simulation
(PS) learning method. It is done by adding a belief projection operator and an
observability parameter to the original framework of the efficiency of the PS
model. I provide theoretical formulations, network representations, and
situated scenarios derived from the invasion toy problem as a starting point
for some multi-agent PS models.Comment: 28 pages, 21 figure
Slow Forcing in the Projective Dynamics Method
We provide a proof that when there is no forcing the recently introduced
projective dynamics Monte Carlo algorithm gives the exact lifetime of the
metastable state, within statistical uncertainties. We also show numerical
evidence illustrating that for slow forcing the approach to the zero-forcing
limit is rather rapid. The model studied numerically is the 3-dimensional
3-state Potts ferromagnet.Comment: 1 figure, invited submission to CCP'98 conference, submitted to
Computer Physics Communication
Generating sequential space-filling designs using genetic algorithms and Monte Carlo methods
In this paper, the authors compare a Monte Carlo method and an optimization-based approach using genetic algorithms for sequentially generating space-filling experimental designs. It is shown that Monte Carlo methods perform better than genetic algorithms for this specific problem
Classification of Matrix Product States with a Local (Gauge) Symmetry
Matrix Product States (MPS) are a particular type of one dimensional tensor
network states, that have been applied to the study of numerous quantum many
body problems. One of their key features is the possibility to describe and
encode symmetries on the level of a single building block (tensor), and hence
they provide a natural playground for the study of symmetric systems. In
particular, recent works have proposed to use MPS (and higher dimensional
tensor networks) for the study of systems with local symmetry that appear in
the context of gauge theories. In this work we classify MPS which exhibit local
invariance under arbitrary gauge groups. We study the respective tensors and
their structure, revealing known constructions that follow known gauging
procedures, as well as different, other types of possible gauge invariant
states
- …