Sparsity through evolutionary pruning prevents neuronal networks from overfitting

Modern Machine learning techniques take advantage of the exponentially rising calculation power in new generation processor units. Thus, the number of parameters which are trained to resolve complex tasks was highly increased over the last decades. However, still the networks fail - in contrast to our brain - to develop general intelligence in the sense of being able to solve several complex tasks with only one network architecture. This could be the case because the brain is not a randomly initialized neural network, which has to be trained by simply investing a lot of calculation power, but has from birth some fixed hierarchical structure. To make progress in decoding the structural basis of biological neural networks we here chose a bottom-up approach, where we evolutionarily trained small neural networks in performing a maze task. This simple maze task requires dynamical decision making with delayed rewards. We were able to show that during the evolutionary optimization random severance of connections lead to better generalization performance of the networks compared to fully connected networks. We conclude that sparsity is a central property of neural networks and should be considered for modern Machine learning approaches

Qualitative Analysis of POMDPs with Temporal Logic Specifications for Robotics Applications

We consider partially observable Markov decision processes (POMDPs), that are a standard framework for robotics applications to model uncertainties present in the real world, with temporal logic specifications. All temporal logic specifications in linear-time temporal logic (LTL) can be expressed as parity objectives. We study the qualitative analysis problem for POMDPs with parity objectives that asks whether there is a controller (policy) to ensure that the objective holds with probability 1 (almost-surely). While the qualitative analysis of POMDPs with parity objectives is undecidable, recent results show that when restricted to finite-memory policies the problem is EXPTIME-complete. While the problem is intractable in theory, we present a practical approach to solve the qualitative analysis problem. We designed several heuristics to deal with the exponential complexity, and have used our implementation on a number of well-known POMDP examples for robotics applications. Our results provide the first practical approach to solve the qualitative analysis of robot motion planning with LTL properties in the presence of uncertainty

Building a Nest by an Automaton

A robot modeled as a deterministic finite automaton has to build a structure from material available to it. The robot navigates in the infinite oriented grid Z x Z. Some cells of the grid are full (contain a brick) and others are empty. The subgraph of the grid induced by full cells, called the field, is initially connected. The (Manhattan) distance between the farthest cells of the field is called its span. The robot starts at a full cell. It can carry at most one brick at a time. At each step it can pick a brick from a full cell, move to an adjacent cell and drop a brick at an empty cell. The aim of the robot is to construct the most compact possible structure composed of all bricks, i.e., a nest. That is, the robot has to move all bricks in such a way that the span of the resulting field be the smallest. Our main result is the design of a deterministic finite automaton that accomplishes this task and subsequently stops, for every initially connected field, in time O(sz), where s is the span of the initial field and z is the number of bricks. We show that this complexity is optimal

Q-learning for autonomous mobile robot obstacle avoidance

An approach to the problem of autonomous mobile robot obstacle avoidance using Reinforcement Learning, more precisely Q-Learning, is presented in this paper. Reinforcement Learning in Robotics has been a challenging topic for the past few years. The ability to equip a robot with a powerful enough tool to allow an autonomous discovery of an optimal behavior through trial-and-error interactions with its environment has been a reason for numerous deep research projects. In this paper, two different Q-Learning approaches are presented as well as an extensive hyperparameter study. These algorithms were developed for a simplistically simulated Bot'n Roll ONE A (Fig. 1). The simulated robot communicates with the control script via ROS. The robot must surpass three levels of iterative complexity mazes similar to the ones presented on RoboParty [1] educational event challenge. For both algorithms, an extensive hyperparameter search was taken into account by testing hundreds of simulations with different parameters. Both Q-Learning solutions develop different strategies trying to solve the three labyrinths, enhancing its learning ability as well as discovering different approaches to certain situations, and finishing the task in complex environments

The cognitive mapping of virtual space

Not availabl

JapanActionGo

This report describes the creation process of Maze of Rebirth, a Virtual Reality game in which the player must escape the world of darkness, Yomi: a procedurally-generated multi-floor maze. The project was developed over the course of three months at Ritsumeikan University Biwako-Kusatsu Campus in Japan with the help of the Noma and Takada labs, as well as a couple of months in Worcester, Massachusetts

The Neural Basis of a Cognitive Map

It has been proposed that as animals explore their environment they build and maintain a cognitive map, an internal representation of their surroundings (Tolman, 1948). We tested this hypothesis using a task designed to assess the ability of rats to make a spatial inference (take a novel shortcut)(Roberts et al., 2007). Our findings suggest that rats are unable to make a spontaneous spatial inference. Furthermore, they bear similarities to experiments which have been similarly unable to replicate or support Tolman’s (1948) findings. An inability to take novel shortcuts suggests that rats do not possess a cognitive map (Bennett, 1996). However, we found evidence of alternative learning strategies, such as latent learning (Tolman & Honzik, 1930b) , which suggest that rats may still be building such a representation, although it does not appear they are able to utilise this information to make complex spatial computations. Neurons found in the hippocampus show remarkable spatial modulation of their firing rate and have been suggested as a possible neural substrate for a cognitive map (O'Keefe & Nadel, 1978). However, the firing of these place cells often appears to be modulated by features of an animal’s behaviour (Ainge, Tamosiunaite, et al., 2007; Wood, Dudchenko, Robitsek, & Eichenbaum, 2000). For instance, previous experiments have demonstrated that the firing rate of place fields in the start box of some mazes are predictive of the animal’s final destination (Ainge, Tamosiunaite, et al., 2007; Ferbinteanu & Shapiro, 2003). We sought to understand whether this prospective firing is in fact related to the goal the rat is planning to navigate to or the route the rat is planning to take. Our results provide strong evidence for the latter, suggesting that rats may not be aware of the location of specific goals and may not be aware of their environment in the form of a contiguous map. However, we also found behavioural evidence that rats are aware of specific goal locations, suggesting that place cells in the hippocampus may not be responsible for this representation and that it may reside elsewhere (Hok, Chah, Save, & Poucet, 2013). Unlike their typical activity in an open field, place cells often have multiple place fields in geometrically similar areas of a multicompartment environment (Derdikman et al., 2009; Spiers et al., 2013). For example, Spiers et al. (2013) found that in an environment composed of four parallel compartments, place cells often fired similarly in multiple compartments, despite the active movement of the rat between them. We were able to replicate this phenomenon, furthermore, we were also able to show that if the compartments are arranged in a radial configuration this repetitive firing does not occur as frequently. We suggest that this place field repetition is driven by inputs from Boundary Vector Cells (BVCs) in neighbouring brain regions which are in turn greatly modulated by inputs from the head direction system. This is supported by a novel BVC model of place cell firing which predicts our observed results accurately. If place cells form the neural basis of a cognitive map one would predict spatial learning to be difficult in an environment where repetitive firing is observed frequently (Spiers et al., 2013). We tested this hypothesis by training animals on an odour discrimination task in the maze environments described above. We found that rats trained in the parallel version of the task were significantly impaired when compared to the radial version. These results support the hypothesis that place cells form the neural basis of a cognitive map; in environments where it is difficult to discriminate compartments based on the firing of place cells, rats find it similarly difficult to discriminate these compartments as shown by their behaviour. The experiments reported here are discussed in terms of a cognitive map, the likelihood that such a construct exists and the possibility that place cells form the neural basis of such a representation. Although the results of our experiments could be interpreted as evidence that animals do not possess a cognitive map, ultimately they suggest that animals do have a cognitive map and that place cells form a more than adequate substrate for this representation

Do not ask whether they have a cognitive map, but how they find their way about.

find their way abou