Novelty of Behaviour as a Basis for the Neuro-evolution of Operant Reward Learning

An agent that deviates from a usual or previous course of action can be said to display novel or varying behaviour. Novelty of behaviour can be seen as the result of real or apparent randomness in decision making, which prevents an agent from repeating exactly past choices. In this paper, novelty of behaviour is considered as an evolutionary precursor of the exploring skill in reward learning, and conservative behaviour as the precursor of exploitation. Novelty of behaviour in neural control is hypothesised to be an important factor in the neuro-evolution of operant reward learning. Agents capable of varying behaviour, as opposed to conservative, when exposed to reward stimuli appear to acquire on a faster evolutionary scale the meaning and use of such reward information. The hypothesis is validated by comparing the performance during evolution in two environments that either favour or are neutral to novelty. Following these findings, we suggest that neuro-evolution of operant reward learning is fostered by environments where behavioural novelty is intrinsically beneficial, i.e. where varying or exploring behaviour is associated with low risk

What causes aberrant salience in schizophrenia? A role for impaired short-term habituation and the GRIA1 (GluA1) AMPA receptor subunit.

The GRIA1 locus, encoding the GluA1 (also known as GluRA or GluR1) AMPA glutamate receptor subunit, shows genome-wide association to schizophrenia. As well as extending the evidence that glutamatergic abnormalities have a key role in the disorder, this finding draws attention to the behavioural phenotype of Gria1 knockout mice. These mice show deficits in short-term habituation. Importantly, under some conditions the attention being paid to a recently presented neutral stimulus can actually increase rather than decrease (sensitization). We propose that this mouse phenotype represents a cause of aberrant salience and, in turn, that aberrant salience (and the resulting positive symptoms) in schizophrenia may arise, at least in part, from a glutamatergic genetic predisposition and a deficit in short-term habituation. This proposal links an established risk gene with a psychological process central to psychosis and is supported by findings of comparable deficits in short-term habituation in mice lacking the NMDAR receptor subunit Grin2a (which also shows association to schizophrenia). As aberrant salience is primarily a dopaminergic phenomenon, the model supports the view that the dopaminergic abnormalities can be downstream of a glutamatergic aetiology. Finally, we suggest that, as illustrated here, the real value of genetically modified mice is not as ‘models of schizophrenia’ but as experimental tools that can link genomic discoveries with psychological processes and help elucidate the underlying neural mechanisms

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented

Nonhuman gamblers: lessons from rodents, primates, and robots

The search for neuronal and psychological underpinnings of pathological gambling in humans would benefit from investigating related phenomena also outside of our species. In this paper, we present a survey of studies in three widely different populations of agents, namely rodents, non-human primates, and robots. Each of these populations offers valuable and complementary insights on the topic, as the literature demonstrates. In addition, we highlight the deep and complex connections between relevant results across these different areas of research (i.e., cognitive and computational neuroscience, neuroethology, cognitive primatology, neuropsychiatry, evolutionary robotics), to make the case for a greater degree of methodological integration in future studies on pathological gambling

Neurofly 2008 abstracts : the 12th European Drosophila neurobiology conference 6-10 September 2008 Wuerzburg, Germany

This volume consists of a collection of conference abstracts

Lymnaea stagnalis as model for translational neuroscience research: from pond to bench

The purpose of this review is to illustrate how a reductionistic, but sophisticated, approach based on the use of a simple model system such as the pond snail Lymnaea stagnalis (L. stagnalis), might be useful to address fundamental questions in learning and memory. L. stagnalis, as a model, provides an interesting platform to investigate the dialog between the synapse and the nucleus and vice versa during memory and learning. More importantly, the "molecular actors" of the memory dialogue are well-conserved both across phylogenetic groups and learning paradigms, involving single- or multi-trials, aversion or reward, operant or classical conditioning. At the same time, this model could help to study how, where and when the memory dialog is impaired in stressful conditions and during aging and neurodegeneration in humans and thus offers new insights and targets in order to develop innovative therapies and technology for the treatment of a range of neurological and neurodegenerative disorders

Evolution of Swarm Robotics Systems with Novelty Search

Novelty search is a recent artificial evolution technique that challenges traditional evolutionary approaches. In novelty search, solutions are rewarded based on their novelty, rather than their quality with respect to a predefined objective. The lack of a predefined objective precludes premature convergence caused by a deceptive fitness function. In this paper, we apply novelty search combined with NEAT to the evolution of neural controllers for homogeneous swarms of robots. Our empirical study is conducted in simulation, and we use a common swarm robotics task - aggregation, and a more challenging task - sharing of an energy recharging station. Our results show that novelty search is unaffected by deception, is notably effective in bootstrapping the evolution, can find solutions with lower complexity than fitness-based evolution, and can find a broad diversity of solutions for the same task. Even in non-deceptive setups, novelty search achieves solution qualities similar to those obtained in traditional fitness-based evolution. Our study also encompasses variants of novelty search that work in concert with fitness-based evolution to combine the exploratory character of novelty search with the exploitatory character of objective-based evolution. We show that these variants can further improve the performance of novelty search. Overall, our study shows that novelty search is a promising alternative for the evolution of controllers for robotic swarms.Comment: To appear in Swarm Intelligence (2013), ANTS Special Issue. The final publication will be available at link.springer.co

Assessment of mGluR5 KO mice under conditions of low stress using a rodent touchscreen apparatus reveals impaired behavioural flexibility driven by perseverative responses

Genetic and pharmacological manipulations targeting metabotropic glutamate receptor 5 (mGluR5) affect performance in behavioural paradigms that depend on cognitive flexibility. Many of these studies involved exposing mice to highly stressful conditions including electric foot shocks or water immersion and forced swimming. Because mGluR5 is also implicated in resilience and stress responses, however, apparent impairments in inhibitory learning may have been an artifact of manipulation-induced changes in affective state. To address this, we present here a characterization of cognitive flexibility in mGluR5 knockout (KO) mice conducted with a rodent touchscreen cognitive assessment apparatus in which the animals experience significantly less stress. Our results indicate a significant reversal learning impairment relative to wild-type (WT) controls in the two-choice Visual Discrimination-Reversal (VDR) paradigm. Upon further analysis, we found that this deficit is primarily driven by a prolonged period of perseveration in the early phase of reversal. We also observed a similar perseveration phenotype in the KO mice in the Extinction (EXT) paradigm. In addition, mGluR5 KO mice show higher breakpoints in the touchscreen Progressive Ratio (PR) and altered decision making in the Effort-related Choice (ERC) tasks. Interestingly, this impairment in PR is an additional manifestation of an increased propensity to perseverate on the emission of relatively simplistic behavioural outputs. Together, these findings suggest that under conditions of low stress, mGluR5 KO mice exhibit a pronounced perseverative phenotype that blunts cognitive flexibility

Challenges and advanced concepts for the assessment of learning and memory function in mice

The mechanisms underlying the formation and retrieval of memories are still an active area of research and discussion. Manifold models have been proposed and refined over the years, with most assuming a dichotomy between memory processes involving non-conscious and conscious mechanisms. Despite our incomplete understanding of the underlying mechanisms, tests of memory and learning count among the most performed behavioral experiments. Here, we will discuss available protocols for testing learning and memory using the example of the most prevalent animal species in research, the laboratory mouse. A wide range of protocols has been developed in mice to test, e.g., object recognition, spatial learning, procedural memory, sequential problem solving, operant- and fear conditioning, and social recognition. Those assays are carried out with individual subjects in apparatuses such as arenas and mazes, which allow for a high degree of standardization across laboratories and straightforward data interpretation but are not without caveats and limitations. In animal research, there is growing concern about the translatability of study results and animal welfare, leading to novel approaches beyond established protocols. Here, we present some of the more recent developments and more advanced concepts in learning and memory testing, such as multi-step sequential lockboxes, assays involving groups of animals, as well as home cage-based assays supported by automated tracking solutions; and weight their potential and limitations against those of established paradigms. Shifting the focus of learning tests from the classical experimental chamber to settings which are more natural for rodents comes with a new set of challenges for behavioral researchers, but also offers the opportunity to understand memory formation and retrieval in a more conclusive way than has been attainable with conventional test protocols. We predict and embrace an increase in studies relying on methods involving a higher degree of automatization, more naturalistic- and home cage-based experimental setting as well as more integrated learning tasks in the future. We are confident these trends are suited to alleviate the burden on animal subjects and improve study designs in memory research

Origins of altered reinforcement effects in ADHD

Attention-deficit/hyperactivity disorder (ADHD), characterized by hyperactivity, impulsiveness and deficient sustained attention, is one of the most common and persistent behavioral disorders of childhood. ADHD is associated with catecholamine dysfunction. The catecholamines are important for response selection and memory formation, and dopamine in particular is important for reinforcement of successful behavior. The convergence of dopaminergic mesolimbic and glutamatergic corticostriatal synapses upon individual neostriatal neurons provides a favorable substrate for a three-factor synaptic modification rule underlying acquisition of associations between stimuli in a particular context, responses, and reinforcers. The change in associative strength as a function of delay between key stimuli or responses, and reinforcement, is known as the delay of reinforcement gradient. The gradient is altered by vicissitudes of attention, intrusions of irrelevant events, lapses of memory, and fluctuations in dopamine function. Theoretical and experimental analyses of these moderating factors will help to determine just how reinforcement processes are altered in ADHD. Such analyses can only help to improve treatment strategies for ADHD