Evolution of Reinforcement Learning in Uncertain Environments: Emergence of Risk-Aversion and Matching

Reinforcement learning (RL) is a fundamental process by which organisms learn to achieve a goal from interactions with the environment. We use Artificial Life techniques to derive (near-)optimal neuronal learning rules in a simple neural network model of decision-making in simulated bumblebees foraging for nectar. The resulting networks exhibit efficient RL, allowing the bees to respond rapidly to changes in reward contingencies. Furthermore, the evolved synaptic plasticity dynamics give rise to varying exploration/exploitation levels from which emerge the well-documented foraging strategies of risk aversion and probability matching. These are shown to be a direct result of optimal RL, providing a biologically founded, parsimonious and novel explanation for these behaviors. Our results are corroborated by a rigorous mathematical analysis and by experiments in mobile robots

Rationality for Adaptive Collective Decision Making Based on Emotion-Related Valuing and Contagion

Abstract. In this paper it is explored how adaptive collective decision making can be evaluated with respect to learning speed and rationality of the decisions. A collective decision model is presented based on interacting adaptive agents that learn from their experiences by a Hebbian learning mechanism. The decision making process makes use of emotionrelated valuing of decision options on the one hand based on predictive loops through feeling states, and on the other hand based on contagion. The resulting collective decision making process is analysed from the perspective of learning speed and rationality. Simulation results and the extent of rationality of the model over time are presented and analysed. It is shown how the collectiveness amplifies both learning speed and rationality of the decisions