The influence of the noradrenergic system on optimal control of neural plasticity

Decision making under uncertainty is challenging for any autonomous agent. The challenge increases when the environment’s stochastic properties change over time, i.e., when the environment is volatile. In order to efficiently adapt to volatile environments, agents must primarily rely on recent outcomes to quickly change their decision strategies; in other words, they need to increase their knowledge plasticity. On the contrary, in stable environments, knowledge stability must be preferred to preserve useful information against noise. Here we propose that in mammalian brain, the locus coeruleus (LC) is one of the nuclei involved in volatility estimation and in the subsequent control of neural plasticity. During a reinforcement learning task, LC activation, measured by means of pupil diameter, coded both for environmental volatility and learning rate. We hypothesize that LC could be responsible, through norepinephrinic modulation, for adaptations to optimize decision making in volatile environments. We also suggest a computational model on the interaction between the anterior cingulate cortex (ACC) and LC for volatility estimation

Deliberative Evolution in Multi-Agent Systems

Item does not contain fulltextEvolution of automated systems, in particular evolution of automated agents based on agent deliberation, is the topic of this paper. Evolution is not a merely material process, it requires interaction within and between individuals, their environments and societies of agents. An architecture for an individual agent capable of (1) deliberation about the creation of new agents, and (2) (run-time) creation of a new agent on the basis of this, is presented. The agent architecture is based on an existing generic agent model, and includes explicit formal conceptual representations of both design structures of agents and (behavioural) properties of agents. The process of deliberation is based on an existing generic reasoning model of design. The architecture has been designed using the compositional development method DESIRE, and has been tested in a prototype implementation

"Exhibitionists" and "voyeurs" do it better: A shared environment for flexible coordination with tacit messages

Coordination between multiple autonomous agents is a major issue for open multi-agent systems. This paper proposes the notion of Behavioural Implicit Communication (BIC) originally devised in human and animal societies as a new and critical coordination mechanism also for artificial agents. BIC is a parasitical form of communication that exploits both some environmental properties and the agents? capacity to interpret their actions. In this paper we abstract from the agents? architecture to focus on the interaction mediated by the environment. Observability of the environment ? and in particular of agents? actions ? is crucial for implementing BIC-based form of coordination in artificial societies. Accordingly in this paper we introduce an abstract model of environment providing services to enhance observation power of agents, enabling BIC and other form of observation-based coordination. Also, we describe a typology of environments and examples of observation based coordination with and without implicit communication

Image scoring in ad-hoc networks : an investigation on realistic settings

Encouraging cooperation in distributed Multi-Agent Systems (MAS) remains an open problem. Emergent application domains such as Mobile Ad-hoc Networks (MANETs) are characterised by constraints including sparse connectivity and a lack of direct interaction history. Image scoring, a simple model of reputation proposed by Nowak and Sigmund, exhibits low space and time complexity and promotes cooperation through indirect reciprocity, in which an agent can expect cooperation in the future without repeat interactions with the same partners. The low overheads of image scoring make it a promising technique for ad-hoc networking domains. However, the original investigation of Nowak and Sigmund is limited in that it (i) used a simple idealised setting, (ii) did not consider the effects of incomplete information on the mechanism’s efficacy, and (iii) did not consider the impact of the network topology connecting agents. We address these limitations by investigating more realistic values for the number of interactions agents engage in, and show that incomplete information can cause significant errors in decision making. As the proportion of incorrect decisions rises, the efficacy of image scoring falls and selfishness becomes more dominant. We evaluate image scoring on three different connection topologies: (i) completely connected, which closely approximates Nowak and Sigmund’s original setup, (ii) random, with each pair of nodes connected with a constant probability, and (iii) scale-free, which is known to model a number of real world environments including MANETs