Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics

A continuous time model for multiagent systems governed by reinforcement learning with scale-free memory is developed. The agents are assumed to act independently of one another in optimizing their choice of possible actions via trial-and-error search. To gain awareness about the action value the agents accumulate in their memory the rewards obtained from taking a specific action at each moment of time. The contribution of the rewards in the past to the agent current perception of action value is described by an integral operator with a power-law kernel. Finally a fractional differential equation governing the system dynamics is obtained. The agents are considered to interact with one another implicitly via the reward of one agent depending on the choice of the other agents. The pairwise interaction model is adopted to describe this effect. As a specific example of systems with non-transitive interactions, a two agent and three agent systems of the rock-paper-scissors type are analyzed in detail, including the stability analysis and numerical simulation. Scale-free memory is demonstrated to cause complex dynamics of the systems at hand. In particular, it is shown that there can be simultaneously two modes of the system instability undergoing subcritical and supercritical bifurcation, with the latter one exhibiting anomalous oscillations with the amplitude and period growing with time. Besides, the instability onset via this supercritical mode may be regarded as "altruism self-organization". For the three agent system the instability dynamics is found to be rather irregular and can be composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur

Competition and habitat quality influence age and sex distribution in wintering rusty blackbirds.

Bird habitat quality is often inferred from species abundance measures during the breeding and non-breeding season and used for conservation management decisions. However, during the non-breeding season age and sex classes often occupy different habitats which suggest a need for more habitat-specific data. Rusty Blackbird (Euphagus carolinus) is a forested wetland specialist wintering in bottomland hardwood forests in the south-eastern U. S. and belongs to the most steeply declining songbirds in the U.S. Little information is available to support priority birds such as the Rusty Blackbird wintering in this threatened habitat. We assessed age and sex distribution and body condition of Rusty Blackbirds among the three major habitats used by this species in the Lower Mississippi Alluvial Valley and also measured food availability. Overall, pecan groves had the highest biomass mainly driven by the amount of nuts. Invertebrate biomass was highest in forests but contributed only a small percentage to overall biomass. Age and sex classes were unevenly distributed among habitats with adult males primarily occupying pecan groves containing the highest nut biomass, females being found in forests which had the lowest nut biomass and young males primarily staying in forest fragments along creeks which had intermediate nut biomass. Males were in better body condition than females and were in slightly better condition in pecan groves. The results suggest that adult males occupy the highest quality habitat and may competitively exclude the other age and sex classes

Competition and Habitat Quality Influence Age and Sex Distribution in Wintering Rusty Blackbirds

Bird habitat quality is often inferred from species abundance measures during the breeding and non-breeding season and used for conservation management decisions. However, during the non-breeding season age and sex classes often occupy different habitats which suggest a need for more habitat-specific data. Rusty Blackbird (Euphagus carolinus) is a forested wetland specialist wintering in bottomland hardwood forests in the south-eastern U. S. and belongs to the most steeply declining songbirds in the U.S. Little information is available to support priority birds such as the Rusty Blackbird wintering in this threatened habitat. We assessed age and sex distribution and body condition of Rusty Blackbirds among the three major habitats used by this species in the Lower Mississippi Alluvial Valley and also measured food availability. Overall, pecan groves had the highest biomass mainly driven by the amount of nuts. Invertebrate biomass was highest in forests but contributed only a small percentage to overall biomass. Age and sex classes were unevenly distributed among habitats with adult males primarily occupying pecan groves containing the highest nut biomass, females being found in forests which had the lowest nut biomass and young males primarily staying in forest fragments along creeks which had intermediate nut biomass. Males were in better body condition than females and were in slightly better condition in pecan groves. The results suggest that adult males occupy the highest quality habitat and may competitively exclude the other age and sex classes

Competition and Habitat Quality Influence Age and Sex Distribution in Wintering Rusty Blackbirds

Bird habitat quality is often inferred from species abundance measures during the breeding and non-breeding season and used for conservation management decisions. However, during the non-breeding season age and sex classes often occupy different habitats which suggest a need for more habitat-specific data. Rusty Blackbird (Euphagus carolinus) is a forested wetland specialist wintering in bottomland hardwood forests in the south-eastern U. S. and belongs to the most steeply declining songbirds in the U.S. Little information is available to support priority birds such as the Rusty Blackbird wintering in this threatened habitat. We assessed age and sex distribution and body condition of Rusty Blackbirds among the three major habitats used by this species in the Lower Mississippi Alluvial Valley and also measured food availability. Overall, pecan groves had the highest biomass mainly driven by the amount of nuts. Invertebrate biomass was highest in forests but contributed only a small percentage to overall biomass. Age and sex classes were unevenly distributed among habitats with adult males primarily occupying pecan groves containing the highest nut biomass, females being found in forests which had the lowest nut biomass and young males primarily staying in forest fragments along creeks which had intermediate nut biomass. Males were in better body condition than females and were in slightly better condition in pecan groves. The results suggest that adult males occupy the highest quality habitat and may competitively exclude the other age and sex classes

Age and sex composition in different habitats.

<p>Frequency of age and sex classes captured in pecan groves, forest fragments along creeks and forests in three consecutive winters. Frequencies of age and sex classes were summed up for sites of the same habitat type within a given year.</p

Food availability in different habitats.

<p>Mean (± SE) dry invertebrate (A, B) and dry nut biomass (C, D) are shown for two winters (2008/ 2009 and 2009/ 2010) in pecan groves (<i>n</i> = 5 and <i>n</i> = 7), forest fragments along creeks (<i>n</i> = 5 and <i>n</i> = 8) and forests (<i>n</i> = 4 and <i>n</i> = 5). For invertebrates, overall mean biomass (mean of wet and dry sites; black bars) and mean biomass for dry and wet sites (gray bars), respectively, are shown. Nut biomass includes pecan and acorn nuts. Mind the different scales in plot C and D.</p

Evaluation of two minimally invasive techniques for electroencephalogram recording in wild or freely behaving animals.

Insight into the function of sleep may be gained by studying animals in the ecological context in which sleep evolved. Until recently, technological constraints prevented electroencephalogram (EEG) studies of animals sleeping in the wild. However, the recent development of a small recorder (Neurologger 2) that animals can carry on their head permitted the first recordings of sleep in nature. To facilitate sleep studies in the field and to improve the welfare of experimental animals, herein, we test the feasibility of using minimally invasive surface and subcutaneous electrodes to record the EEG in barn owls. The EEG and behaviour of four adult owls in captivity and of four chicks in a nest box in the field were recorded. We scored a 24-h period for each adult bird for wakefulness, slow-wave sleep (SWS), and rapid-eye movement (REM) sleep using 4 s epochs. Although the quality and stability of the EEG signals recorded via subcutaneous electrodes were higher when compared to surface electrodes, the owls' state was readily identifiable using either electrode type. On average, the four adult owls spent 13.28 h awake, 9.64 h in SWS, and 1.05 h in REM sleep. We demonstrate that minimally invasive methods can be used to measure EEG-defined wakefulness, SWS, and REM sleep in owls and probably other animals

Extra-pair mating and evolution of cooperative neighbourhoods

A striking but unexplained pattern in biology is the promiscuous mating behaviour in socially monogamous species. Although females commonly solicit extra-pair copulations, the adaptive reason has remained elusive. We use evolutionary modelling of breeding ecology to show that females benefit because extra-pair paternity incentivizes males to shift focus from a single brood towards the entire neighbourhood, as they are likely to have offspring there. Male-male cooperation towards public goods and dear enemy effects of reduced territorial aggression evolve from selfish interests, and lead to safer and more productive neighbourhoods. The mechanism provides adaptive explanations for the common empirical observations that females engage in extra-pair copulations, that neighbours dominate as extra-pair sires, and that extra-pair mating correlates with predation mortality and breeding density. The models predict cooperative behaviours at breeding sites where males cooperate more towards public goods than females. Where maternity certainty makes females care for offspring at home, paternity uncertainty and a potential for offspring in several broods make males invest in communal benefits and public goods. The models further predict that benefits of extra-pair mating affect whole nests or neighbourhoods, and that cuckolding males are often cuckolded themselves. Derived from ecological mechanisms, these new perspectives point towards the evolution of sociality in birds, with relevance also for mammals and primates including humans