Evolution and learning in artificial ecosystems

A generic model is presented for ecosystems inhabited by artificial animals, or animats, that develop over time. The individual animats develop continuously by means of generic mechanisms for learning, forgetting, and decisionmaking.At the same time, the animat populations develop in an evolutionary process based on fixed mechanisms for sexual and asexual reproduction, mutation, and death. The animats of the ecosystems move, eat, learn, make decisions, interact with other animats, reproduce, and die. Each animat has its individual sets of homeostatic variables, sensors, and motors.It also has its own memory graph that forms the basis of its decision-making. This memory graph has an architecture (i.e. graph topology) that changes over time via mechanisms for adding and removing nodes. Our approach combines genetic algorithms, reinforcement learning, homeostatic decision-making, and dynamic concept formation. To illustrate the generality of the model, five examples of ecosystems are given, ranging from a simpleworld inhabited by a single frog to a more complex world in which grass, sheep, and wolves interact

A Comparison of Different Cognitive Paradigms Using Simple Animats in a Virtual Laboratory, with Implications to the Notion of Cognition

In this thesis I present a virtual laboratory which implements five different models for controlling animats: a rule-based system, a behaviour-based system, a concept-based system, a neural network, and a Braitenberg architecture. Through different experiments, I compare the performance of the models and conclude that there is no best model, since different models are better for different things in different contexts. The models I chose, although quite simple, represent different approaches for studying cognition. Using the results as an empirical philosophical aid, I note that there is no best approach for studying cognition, since different approaches have all advantages and disadvantages, because they study different aspects of cognition from different contexts. This has implications for current debates on proper approaches for cognition: all approaches are a bit proper, but none will be proper enough. I draw remarks on the notion of cognition abstracting from all the approaches used to study it, and propose a simple classification for different types of cognition

Learning visual docking for non-holonomic autonomous vehicles

This paper presents a new method of learning visual docking skills for non-holonomic vehicles by direct interaction with the environment. The method is based on a reinforcement algorithm, which speeds up Q-learning by applying memorybased sweeping and enforcing the “adjoining property”, a filtering mechanism to only allow transitions between states that satisfy a fixed distance. The method overcomes some limitations of reinforcement learning techniques when they are employed in applications with continuous non-linear systems, such as car-like vehicles. In particular, a good approximation to the optimal behaviour is obtained by a small look-up table. The algorithm is tested within an image-based visual servoing framework on a docking task. The training time was less than 1 hour on the real vehicle. In experiments, we show the satisfactory performance of the algorithm

Modeling the aggregated exposure and responses of bowhead whales Balaena mysticetus to multiple sources of anthropogenic underwater sound

Potential responses of marine mammals to anthropogenic underwater sound are usually assessed by researchers and regulators on the basis of exposure to a single, relatively loud sound source. However, marine mammals typically receive sounds from multiple, dynamic sources. We developed a method to aggregate modeled sounds from multiple sources and estimate the sound levels received by individuals. To illustrate the method, we modeled the sound fields of 9 sources associated with oil development and estimated the sound received over 47 d by a population of 10 000 simulated bowhead whales Balaena mysticetus on their annual migration through the Alaskan Beaufort Sea. Empirical data were sufficient to parameterize simulations of the distribution of individual whales over time and their range of movement patterns. We ran 2 simulations to estimate the sound exposure history and distances traveled by bowhead whales: one in which they could change their movement paths (avert) in response to set levels of sound and one in which they could not avert. When animals could not avert, about 2% of the simulated population was exposed to root mean square (rms) sound pressure levels (SPL) \u3e = 180 dB re 1 mu Pa, a level that regulators in the U.S. often associate with injury. When animals could avert from sound levels that regulators often associate with behavioral disturbance (rms SPL \u3e 160 dB re 1 mu Pa), \u3c 1% of the simulated population was exposed to levels associated with injury. Nevertheless, many simulated bowhead whales received sound levels considerably above ambient throughout their migration. Our method enables estimates of the aggregated level of sound to which populations are exposed over extensive areas and time periods

General self-motivation and strategy identification : Case studies based on Sokoban and Pac-Man

(c) 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.In this paper, we use empowerment, a recently introduced biologically inspired measure, to allow an AI player to assign utility values to potential future states within a previously unencountered game without requiring explicit specification of goal states. We further introduce strategic affinity, a method of grouping action sequences together to form "strategies," by examining the overlap in the sets of potential future states following each such action sequence. We also demonstrate an information-theoretic method of predicting future utility. Combining these methods, we extend empowerment to soft-horizon empowerment which enables the player to select a repertoire of action sequences that aim to maintain anticipated utility. We show how this method provides a proto-heuristic for nonterminal states prior to specifying concrete game goals, and propose it as a principled candidate model for "intuitive" strategy selection, in line with other recent work on "self-motivated agent behavior." We demonstrate that the technique, despite being generically defined independently of scenario, performs quite well in relatively disparate scenarios, such as a Sokoban-inspired box-pushing scenario and in a Pac-Man-inspired predator game, suggesting novel and principle-based candidate routes toward more general game-playing algorithms.Peer reviewedFinal Accepted Versio

Basic language learning in artificial animals

We explore a general architecture for artificial animals, or animats, that develops over time. The architecture combines reinforcementlearning, dynamic concept formation, and homeostatic decision-making aimed at need satisfaction. We show that thisarchitecture, which contains no ad hoc features for language processing, is capable of basic language learning of three kinds: (i)learning to reproduce phonemes that are perceived in the environment via motor babbling; (ii) learning to reproduce sequences ofphonemes corresponding to spoken words perceived in the environment; and (iii) learning to ground the semantics of spoken wordsin sensory experience by associating spoken words (e.g. the word “cold”) to sensory experience (e.g. the activity of a sensor forcold temperature) and vice versa

Embodied Robot Models for Interdisciplinary Emotion Research

Due to their complex nature, emotions cannot be properly understood from the perspective of a single discipline. In this paper, I discuss how the use of robots as models is beneficial for interdisciplinary emotion research. Addressing this issue through the lens of my own research, I focus on a critical analysis of embodied robots models of different aspects of emotion, relate them to theories in psychology and neuroscience, and provide representative examples. I discuss concrete ways in which embodied robot models can be used to carry out interdisciplinary emotion research, assessing their contributions: as hypothetical models, and as operational models of specific emotional phenomena, of general emotion principles, and of specific emotion ``dimensions''. I conclude by discussing the advantages of using embodied robot models over other models.Peer reviewe