Understanding structure of concurrent actions

Whereas most work in reinforcement learning (RL) ignores the structure or relationships between actions, in this paper we show that exploiting structure in the action space can improve sample efficiency during exploration. To show this we focus on concurrent action spaces where the RL agent selects multiple actions per timestep. Concurrent action spaces are challenging to learn in especially if the number of actions is large as this can lead to a combinatorial explosion of the action space. This paper proposes two methods: a first approach uses implicit structure to perform high-level action elimination using task-invariant actions; a second approach looks for more explicit structure in the form of action clusters. Both methods are context-free, focusing only on an analysis of the action space and show a significant improvement in policy convergence times

Can Machines Think in Radio Language?

People can think in auditory, visual and tactile forms of language, so can machines principally. But is it possible for them to think in radio language? According to a first principle presented for general intelligence, i.e. the principle of language's relativity, the answer may give an exceptional solution for robot astronauts to talk with each other in space exploration.Comment: 4 pages, 1 figur

Polaron band formation in the Holstein model

We present numerical exact results for the polaronic band structure of the Holstein molecular crystal model in one and two dimensions. The use of direct Lanczos diagonalization technique, preserving the full dynamics and quantum nature of phonons, allows us to analyze in detail the renormalization of both quasiparticle bandwidth and dispersion by the electron-phonon interaction. For the two-dimensional case some of our exact data are compared with the results obtained in the framework of a recently developed finite cluster strong-coupling perturbation theory.Comment: 10 pages (LaTeX), 6 figures (ps), submitted to Phys. Rev.

A combinatorial approach to knot recognition

This is a report on our ongoing research on a combinatorial approach to knot recognition, using coloring of knots by certain algebraic objects called quandles. The aim of the paper is to summarize the mathematical theory of knot coloring in a compact, accessible manner, and to show how to use it for computational purposes. In particular, we address how to determine colorability of a knot, and propose to use SAT solving to search for colorings. The computational complexity of the problem, both in theory and in our implementation, is discussed. In the last part, we explain how coloring can be utilized in knot recognition

Optical absorption and single-particle excitations in the 2D Holstein t-J model

To discuss the interplay of electronic and lattice degrees of freedom in systems with strong Coulomb correlations we have performed an extensive numerical study of the two-dimensional Holstein t-J model. The model describes the interaction of holes, doped in a quantum antiferromagnet, with a dispersionsless optical phonon mode. We apply finite-lattice Lanczos diagonalization, combined with a well-controlled phonon Hilbert space truncation, to the Hamiltonian. The focus is on the dynamical properties. In particular we have evaluated the single-particle spectral function and the optical conductivity for characteristic hole-phonon couplings, spin exchange interactions and phonon frequencies. The results are used to analyze the formation of hole polarons in great detail. Links with experiments on layered perovskites are made. Supplementary we compare the Chebyshev recursion and maximum entropy algorithms, used for calculating spectral functions, with standard Lanczos methods.Comment: 32 pages, 12 figures, submitted to Phys. Rev.

Self-Modification of Policy and Utility Function in Rational Agents

Any agent that is part of the environment it interacts with and has versatile actuators (such as arms and fingers), will in principle have the ability to self-modify -- for example by changing its own source code. As we continue to create more and more intelligent agents, chances increase that they will learn about this ability. The question is: will they want to use it? For example, highly intelligent systems may find ways to change their goals to something more easily achievable, thereby `escaping' the control of their designers. In an important paper, Omohundro (2008) argued that goal preservation is a fundamental drive of any intelligent system, since a goal is more likely to be achieved if future versions of the agent strive towards the same goal. In this paper, we formalise this argument in general reinforcement learning, and explore situations where it fails. Our conclusion is that the self-modification possibility is harmless if and only if the value function of the agent anticipates the consequences of self-modifications and use the current utility function when evaluating the future.Comment: Artificial General Intelligence (AGI) 201

Calculation of Densities of States and Spectral Functions by Chebyshev Recursion and Maximum Entropy

We present an efficient algorithm for calculating spectral properties of large sparse Hamiltonian matrices such as densities of states and spectral functions. The combination of Chebyshev recursion and maximum entropy achieves high energy resolution without significant roundoff error, machine precision or numerical instability limitations. If controlled statistical or systematic errors are acceptable, cpu and memory requirements scale linearly in the number of states. The inference of spectral properties from moments is much better conditioned for Chebyshev moments than for power moments. We adapt concepts from the kernel polynomial approximation, a linear Chebyshev approximation with optimized Gibbs damping, to control the accuracy of Fourier integrals of positive non-analytic functions. We compare the performance of kernel polynomial and maximum entropy algorithms for an electronic structure example.Comment: 8 pages RevTex, 3 postscript figure