14 research outputs found
Benchmarking projective simulation in navigation problems
Projective simulation (PS) is a model for intelligent agents with a
deliberation capacity that is based on episodic memory. The model has been
shown to provide a flexible framework for constructing reinforcement-learning
agents, and it allows for quantum mechanical generalization, which leads to a
speed-up in deliberation time. PS agents have been applied successfully in the
context of complex skill learning in robotics, and in the design of
state-of-the-art quantum experiments. In this paper, we study the performance
of projective simulation in two benchmarking problems in navigation, namely the
grid world and the mountain car problem. The performance of PS is compared to
standard tabular reinforcement learning approaches, Q-learning and SARSA. Our
comparison demonstrates that the performance of PS and standard learning
approaches are qualitatively and quantitatively similar, while it is much
easier to choose optimal model parameters in case of projective simulation,
with a reduced computational effort of one to two orders of magnitude. Our
results show that the projective simulation model stands out for its simplicity
in terms of the number of model parameters, which makes it simple to set up the
learning agent in unknown task environments.Comment: 8 pages, 10 figure
Projective simulation with generalization
The ability to generalize is an important feature of any intelligent agent.
Not only because it may allow the agent to cope with large amounts of data, but
also because in some environments, an agent with no generalization capabilities
cannot learn. In this work we outline several criteria for generalization, and
present a dynamic and autonomous machinery that enables projective simulation
agents to meaningfully generalize. Projective simulation, a novel, physical
approach to artificial intelligence, was recently shown to perform well in
standard reinforcement learning problems, with applications in advanced
robotics as well as quantum experiments. Both the basic projective simulation
model and the presented generalization machinery are based on very simple
principles. This allows us to provide a full analytical analysis of the agent's
performance and to illustrate the benefit the agent gains by generalizing.
Specifically, we show that already in basic (but extreme) environments,
learning without generalization may be impossible, and demonstrate how the
presented generalization machinery enables the projective simulation agent to
learn.Comment: 14 pages, 9 figure
Exact exchange-correlation potential of a ionic Hubbard model with a free surface
We use Lanczos exact diagonalization to compute the exact
exchange-correlation (xc) potential of a Hubbard chain with large binding
energy ("the bulk") followed by a chain with zero binding energy ("the
vacuum"). Several results of density functional theory in the continuum
(sometimes controversial) are verified in the lattice. In particular we show
explicitly that the fundamental gap is given by the gap in the Kohn-Sham
spectrum plus a contribution due to the jump of the xc-potential when a
particle is added. The presence of a staggered potential and a nearest-neighbor
interaction V allows to simulate a ionic solid. We show that in the ionic
regime in the small hopping amplitude limit the xc-contribution to the gap
equals V, while in the Mott regime it is determined by the Hubbard U
interaction. In addition we show that correlations generates a new potential
barrier at the surface