Evolution of Memory in Reactive Artificial Neural Networks

In the neuronal circuits of natural and artificial agents, memory is usually implemented with recurrent connections, since recurrence allows past agent state to affect the present, on-going behavior. Here, an interesting question arises in the context of evolution: how reactive agents could have evolved into cognitive ones with internalized memory? This study strives to find an answer to the question by simulating neuroevolution on artificial neural networks, with the hypothesis that internalization of external material interaction can be a plausible evolutionary path leading to a fully internalized memory system. A series of computational experiments were performed to gradually verify the above hypothesis. The first experiment demonstrated the possibility that external materials can be used as memory-aids for a memoryless reactive artificial agents in a simple 1-dimensional environment. Here, the reactive artificial agents used environmental markers as memory references to be successful in the ball-catching task that requires memory. Motivated by the result of the first experiment, an extended experiment was conducted to tackle a more complex memory problem using the same principle of external material interaction. This time, the reactive artificial agents are tasked to remember the locations of food items and the nest in a 2-dimensional environment. Such path-following behavior is a trivial foraging strategy of various lower animals such as ants and fish. The final experiment was designed to show the evolution of internal recurrence. In this experiment, I showed the evolutionary advantage of external material interaction by comparing the results from neural network topology evolution algorithms with and without the material interaction mechanism. The result confirmed that the agents with external material interaction learned to solve the memory task faster and more accurately. The results of the experiments provide insights on the possible evolutionary route to an internalized memory. The use of external material interaction can help reactive artificial agents to go beyond the functionality restricted by their simple network structure. Moreover, it allows much faster convergence with higher accuracy than the topological evolution of the artificial agents. These results suggest one plausible evolutionary path from reactive, through external material interaction, to recurrent structure

Evolutionary, developmental neural networks for robust robotic control

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 136-143).The use of artificial evolution to synthesize controllers for physical robots is still in its infancy. Most applications are on very simple robots in artificial environments, and even these examples struggle to span the "reality gap," a name given to the difference between the performance of a simulated robot and the performance of a.real robot using the same evolved controller. This dissertation describes three methods for improving the use of artificial evolution as a tool for generating controllers for physical robots. First, the evolutionary process must incorporate testing on the physical robot. Second, repeated structure on the robot should be exploited. Finally, prior knowledge about the robot and task should be meaningfully incorporated. The impact of these three methods, both in simulation and on physical robots, is demonstrated, quantified, and compared to hand-designed controllers.by Bryan Adams.Ph.D