Unsupervised Modeling of Partially Observable Environments

Abstract. We present an architecture based on self-organizing maps for learning a sensory layer in a learning system. The architecture, temporal network for transitions (TNT), enjoys the freedoms of unsupervised learning, works on-line, in non-episodic environments, is computationally light, and scales well. TNT generates a predictive model of its internal representation of the world, making planning methods available for both the exploitation and exploration of the environment. Experiments demonstrate that TNT learns nice representations of classical reinforcement learning mazes of varying size (up to 20 × 20) under conditions of high-noise and stochastic actions. Keywords: Self-Organizing Maps, POMDPs, Reinforcement Learning

Grammatical Evolution with Bidirectional Representation

Abstract. Grammatical evolution is an evolutionary algorithm designed to evolve programs in any language. Grammatical evolution operates on binary strings and the mapping of the genotype onto the phenotype (the tree representation of the programs) is provided through the grammar described in the form of production rules. The program trees are constructed in a pre-order fashion, which means that as the genome is traversed first the left most branch of the tree is completed then the second from the left one etc. Once two individuals are crossed over by means of simple one-point crossover the tail parts of the chromosomes (originally encoding the structures on the right side of the program tree) may map on different program structures within the new context. Here we present a bidirectional representation which helps to equalize the survival rate of both the program structures appearing on the left and right side of the program parse tree