A new class of neural architectures to model episodic memory : computational studies of distal reward learning

A computational cognitive neuroscience model is proposed, which models episodic memory based on the mammalian brain. A computational neural architecture instantiates the proposed model and is tested on a particular task of distal reward learning. Categorical Neural Semantic Theory informs the architecture design. To experiment upon the computational brain model, embodiment and an environment in which the embodiment exists are simulated. This simulated environment realizes the Morris Water Maze task, a well established biological experimental test of distal reward learning. The embodied neural architecture is treated as a virtual rat and the environment it acts in as a virtual water tank. Performance levels of the neural architectures are evaluated through analysis of embodied behavior in the distal reward learning task. Comparison is made to biological rat experimental data, as well as comparison to other published models. In addition, differences in performance are compared between the normal and categorically informed versions of the architecture

Spatial learning and navigation in the rat:a biomimetic model

Animals behave in different ways depending on the specific task they are required to solve. In certain cases, if a cue marks the goal location, they can rely on simple stimulusresponse associations. In contrast, other tasks require the animal to be endowed with a representation of space. Such a representation (i.e. cognitive map) allows the animal to locate itself within a known environment and perform complex target-directed behaviour. In order to efficiently perform, the animal not only should be able to exhibit these types of behaviour, but it should be able to select which behaviour is the most appropriate at any given task conditions. Neurophysiological and behavioural experiments provide important information on how such processes may take place in the rodent's brain. Specifically, place- and orientation sensitive cells in the rat Hippocampus have been interpreted as a neural substrate for spatial abilities related to the theory of the cognitive map proposed in the late 1940s by Tolman. Moreover, recent dissociation experiments using selectively located lesions, as well as pharmacological studies have shown that different brain regions may be involved in different types of behaviour. Accordingly, one memory system involving the hippocampus and the ventral striatum would be responsible for cognitive navigation, while navigation based on stimulus-response associations would be mediated by the dorsolateral striatum. Based on these studies, the aim of this work is to develop a neural network model of the spatial abilities of the rat. The model, based on functional properties and anatomical inter-connections of the brain areas involved in spatial learning should be able to establish a distributed representation of space composed of place-sensitive units. Such a representation takes into account both internal and external sensory information, and the model reproduces physiological properties of place cells such as changes in their directional dependence. Moreover, the spatial representation may be used to perform cognitive navigation. Modelled place cells drive an extra-hippocampal population of action-coding cells, allowing the establishment of place-response associations. These associations encoded in synaptic connections between place- and action-cells are modified by means of reinforcement learning. In a similar way, simple sensory input can be used to establish stimulus-response associations. These associations are encoded in a different set of action cells which corresponds to a different neural substrate encoding for non-cognitive navigation strategies (i.e. taxon or praxic). Both cognitive and non-cognitive navigation strategies compete for action control to determine the actual behaviour of the agent. Tests of the performance of the model show that it is able to establish a representation of space, and modelled place cells reproduce some physiological properties of their biological counterparts. Furthermore, the model reproduces goal-based behaviour based on both cognitive and non-cognitive strategies as well as behaviour in conflicting situations reported in experimental studies in animals

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp

Spatial navigation in geometric mazes:a computational model of rodent behavior

Navigation is defined as the capability of planning and performing a path from the current position towards a desired location. Different types, or strategies, of navigation are used by animals depending on the task they are trying to solve. Visible goals can be approached directly, while navigation to a hidden goal usually requires a memorized representation of relative positions of the goal and surrounding landmarks. Neurophysiological and behavioral experiments on rodents suggest that different brain areas are responsible for the expression of different navigation strategies. Specifically, dorsal striatum has been related to storage and recall of stimulus-response associations underlying simple goal-approaching behaviors, whereas hippocampus is thought to store the spatial representation of the environment. Such a representation is built during an unrewarded spatial exploration and appears to be employed in cases when simple stimulus-response strategies fail. Discovery of neurons with spatially correlated activity, i.e. place cells and grid cells, in the hippocampal formation complements behavioral and lesion data suggesting its role for spatial orientation. The overall objective of this work is to study the neurophysiological mechanisms underlying rodent spatial behavior, in particular those that are responsible for the implementation of different navigational strategies. Special attention is devoted to the question of how various types of sensory cues influence goal-oriented behavior. The model of a navigating rat described in this work is based on functional and anatomical properties of brain regions involved in encoding and storage of space representation and action generation. In particular, place and grid cells are modeled by two interconnected populations of artificial neurons. Together, they form a network for spatial learning, capable of combining different types of sensory inputs to produce a distributed representation of location. Goal-directed actions can be generated in the model via two different neural pathways: the first one drives stimulus-response behavior and associates visual input directly to motor responses; the second one associates motor actions with places and hence depends on the representation of location. The visual input is represented by responses of a large number of orientation-sensitive filters to visual images generated according to the position and orientation of the simulated rat in a virtual three-dimensional world. The model was tested in a large array of tasks designed by analogy to experimental studies on animal behavior. Results of several experimental studies, behavioral as wells as neurophysiological, were reproduced. Based on these results we formulated a hypothesis about the influence that the rat's perception of surrounding environment exerts on goal-oriented behavior. This hypothesis may provide an insight into several issues in animal behavior research that were not addressed by theoretical models until now

The Context-Aware Learning Model

The ultimate goal of this research is to build a novel, generalized, arbitrary-depth, neural controller that performs reward- and experience-based neuromodulatory learning, which is online, bootstrapping, interactive, incremental, and dynamic. Autonomous agents, such as robots, maybe able to adapt to uncertain environments if they use reward-based, interactive learning. Unfortunately, typical reward-based models are based on discrete state and action spaces whereas many interesting applications contain continuous spaces. This suggests the use of an artificial neural controller with continuous weights. Adapting the neuromodulatory features of biological brains into a robot controller plays an important role in building more biological robots; however, a biologically feasible learning model does not necessarily promote increased learning efficiency or optimizing the neural networks in a generalized way. For these reasons, this research introduces the Context-Aware Learning Model (CALM) and four different learning algorithms that operate within this model, all of which use logistic regression backpropagation and hyperbolic, reward-based learning. This research introduces a novel way of combining reward- and experience-based learning with an arbitrary-depth artificial neural network and shows how specific behavioral neurobiological features are applied in building a novel neuromodulatory learning mechanism. CALM is evaluated with five metrics on six synthetic data sets and shows promising performances