Adaptive optimal training of animal behavior

Abstract Neuroscience experiments often require training animals to perform tasks designed to elicit various sensory, cognitive, and motor behaviors. Training typically involves a series of gradual adjustments of stimulus conditions and rewards in order to bring about learning. However, training protocols are usually hand-designed, relying on a combination of intuition, guesswork, and trial-and-error, and often require weeks or months to achieve a desired level of task performance. Here we combine ideas from reinforcement learning and adaptive optimal experimental design to formulate methods for adaptive optimal training of animal behavior. Our work addresses two intriguing problems at once: first, it seeks to infer the learning rules underlying an animal's behavioral changes during training; second, it seeks to exploit these rules to select stimuli that will maximize the rate of learning toward a desired objective. We develop and test these methods using data collected from rats during training on a two-interval sensory discrimination task. We show that we can accurately infer the parameters of a policy-gradient-based learning algorithm that describes how the animal's internal model of the task evolves over the course of training. We then formulate a theory for optimal training, which involves selecting sequences of stimuli that will drive the animal's internal policy toward a desired location in the parameter space. Simulations show that our method can in theory provide a substantial speedup over standard training methods. We feel these results will hold considerable theoretical and practical implications both for researchers in reinforcement learning and for experimentalists seeking to train animals

Peacemaking among inconsistent rationalities?

Kacelnik, Schuck-Paim and Pompilio (this volume, p. 377) show that rationality axioms from economics are neither necessary nor sufficient to guarantee that animal behavior is biologically adaptive. To illustrate that biological adaptiveness does not imply conformity with the consistency axioms of economics, Kacelnik et al describe animals that sensibly experiment with actions yielding sub-maximum levels of short-term energy intake to monitor their environments for change, leading to apparently intransitive patterns of choice that are nevertheless biologically adaptive. Invalidating the converse claim that economic rationality implies biological adaptiveness is Kacelnik et al’s example of female ruffs that are worse off when they conform to the constant-ratio rule, frequently interpreted as a normative consistency requirement of economic rationality. Together, the two examples demonstrate that axiomatic norms are both unnecessary and insufficient for determining whether a particular behavior is biologically adaptive. Additionally, Kacelnik et al call into question what has been reported in the animal behavior literature as preference reversals, such as risk attitudes among wild rufous hummingbirds or the food-hoarding propensities of grey jays. Kacelnik et al attribute apparent reversals to state-dependent fitness functions modulated by subtle differences in the training phase of animal experiments. For example, animals trained on menus that include a strictly dominated option will tend to have lower accumulated energy reserves and therefore exhibit systematically different patterns of choice––not because they fail to maximize, but because their training has induced systematically different nutritional states. Another possible explanation for preference reversals in animal studies with strictly dominated, or “decoy” options is that menus containing dominated items may convey valid information about future opportunities (Houston and McNamara, 1999). If menus are correlated through time, then menus with inferior options today predict scarcity in the future and imply a distinct optimal course of action, in violation of regularity assumptions that posit invariance with respect to the inclusion of strictly dominated alternatives. In environments with payoff structures that can be modeled as cooperative games, a family’s best response sometimes requires individual family members to behave suboptimally as part of a diversification strategy that reduces the risk of reproductive failure (Hutchinson, 1996). Futhermore, theoretical biologists have documented the fragility of expected fitness maximizing behaviour with respect to the assumption of stable environments. Once the model allows for shocks to the environment’s stochastic structure, simple behavior rules that are suboptimal (in terms of expected fitness) when viewed narrowly from the perspective of unchanging payoffs in a fixed environment may outperform rules based on maximazation within a static small world (Bookstaber and Langsam, 1985).Rationality, rationalities, irrationality, bounded rationality, biology, biological rationality

Robust sound event detection in bioacoustic sensor networks

Bioacoustic sensors, sometimes known as autonomous recording units (ARUs), can record sounds of wildlife over long periods of time in scalable and minimally invasive ways. Deriving per-species abundance estimates from these sensors requires detection, classification, and quantification of animal vocalizations as individual acoustic events. Yet, variability in ambient noise, both over time and across sensors, hinders the reliability of current automated systems for sound event detection (SED), such as convolutional neural networks (CNN) in the time-frequency domain. In this article, we develop, benchmark, and combine several machine listening techniques to improve the generalizability of SED models across heterogeneous acoustic environments. As a case study, we consider the problem of detecting avian flight calls from a ten-hour recording of nocturnal bird migration, recorded by a network of six ARUs in the presence of heterogeneous background noise. Starting from a CNN yielding state-of-the-art accuracy on this task, we introduce two noise adaptation techniques, respectively integrating short-term (60 milliseconds) and long-term (30 minutes) context. First, we apply per-channel energy normalization (PCEN) in the time-frequency domain, which applies short-term automatic gain control to every subband in the mel-frequency spectrogram. Secondly, we replace the last dense layer in the network by a context-adaptive neural network (CA-NN) layer. Combining them yields state-of-the-art results that are unmatched by artificial data augmentation alone. We release a pre-trained version of our best performing system under the name of BirdVoxDetect, a ready-to-use detector of avian flight calls in field recordings.Comment: 32 pages, in English. Submitted to PLOS ONE journal in February 2019; revised August 2019; published October 201

Bestial boredom: a biological perspective on animal boredom and suggestions for its scientific investigation

Boredom is likely to have adaptive value in motivating exploration and learning, and many animals may possess the basic neurological mechanisms to support it. Chronic inescapable boredom can be extremely aversive, and understimulation can harm neural, cognitive and behavioural flexibility. Wild and domesticated animals are at particular risk in captivity, which is often spatially and temporally monotonous. Yet biological research into boredom has barely begun, despite having important implications for animal welfare, the evolution of motivation and cognition, and for human dysfunction at individual and societal levels. Here I aim to facilitate hypotheses about how monotony affects behaviour and physiology, so that boredom can be objectively studied by ethologists and other scientists. I cover valence (pleasantness) and arousal (wakefulness) qualities of boredom, because both can be measured, and I suggest boredom includes suboptimal arousal and aversion to monotony. Because the suboptimal arousal during boredom is aversive, individuals will resist low arousal. Thus, behavioural indicators of boredom will, seemingly paradoxically, include signs of increasing drowsiness, alongside bouts of restlessness, avoidance and sensation-seeking behaviour. Valence and arousal are not, however, sufficient to fully describe boredom. For example, human boredom is further characterized by a perception that time ‘drags’, and this effect of monotony on time perception can too be behaviourally assayed in animals. Sleep disruption and some abnormal behaviour may also be caused by boredom. Ethological research into this emotional phenomenon will deepen understanding of its causes, development, function and evolution, and will enable evidence-based interventions to mitigate human and animal boredom

The Evolution of Reaction-diffusion Controllers for Minimally Cognitive Agents

No description supplie