When is it adaptive to be patient? A general framework for evaluating delayed rewards

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.The tendency of animals to seek instant gratification instead of waiting for greater long-term benefits has been described as impatient, impulsive or lacking in self-control. How can we explain the evolution of such seemingly irrational behaviour? Here we analyse optimal behaviour in a variety of simple choice situations involving delayed rewards. We show that preferences for more immediate rewards should depend on a variety of factors, including whether the choice is a one-off or is likely to be repeated, the information the animal has about the continuing availability of the rewards and the opportunity to gain rewards through alternative activities. In contrast to the common assertion that rational animals should devalue delayed rewards exponentially, we find that this pattern of discounting is optimal only under restricted circumstances. We predict preference reversal whenever waiting for delayed rewards entails loss of opportunities elsewhere, but the direction of this reversal depends on whether the animal will face the same choice repeatedly. Finally, we question the ecological relevance of standard laboratory tests for impulsive behaviour, arguing that animals rarely face situations analogous to the self-control paradigm in their natural environment. To understand the evolution of impulsiveness, a more promising strategy would be to identify decision rules that are adaptive in a realistic ecological setting, and examine how these rules determine patterns of behaviour in simultaneous choice tests.We thank the European Research Council for financial support (Advanced Grant 250209 to A.I.H.)

Evolution of a flexible rule for foraging that copes with environmental variation

PublishedThis is the final version of the article. It first appeared from Oxford University Press via http://www.currentzoology.org/paperdetail.asp?id=12437Models of adaptive behaviour typically assume that animals behave as though they have highly complex, detailed strategies for making decisions. In reality, selection favours the optimal balance between the costs and benefits of complexity. Here we investigate this trade-off for an animal that has to decide whether or not to forage for food - and so how much energy reserves to store - depending on the food availability in its environment. We evolve a decision rule that controls the target reserve level for different ranges of food availability, but where increasing complexity is costly in that metabolic rate increases with the sensitivity of the rule. The evolved rule tends to be much less complex than the optimal strategy but performs almost as well, while being less costly to implement. It achieves this by being highly sensitive to changing food availability at low food abun-dance - where it provides a close fit to the optimal strategy - but insensitive when food is plentiful. When food availability is high, the target reserve level that evolves is much higher than under the optimal strategy, which has implications for our under-standing of obesity. Our work highlights the important principle of generalisability of simple decision-making mechanisms, which enables animals to respond reasonably well to conditions not directly experienced by themselves or their ancestors.This work was supported by the European Research Council (Advanced Grant 250209 to A.I.H.) and a College for Life Sciences Fellowship at the Wissenschaftskolleg zu Berlin awarded to A.D.H

Clarifying the relationship between prospect theory and risk-sensitive foraging theory

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.When given a choice between options with uncertain outcomes, people tend to be loss averse and risk averse regarding potential gains and risk prone regarding potential losses. These features of human decision making are captured by prospect theory (PT)-a hugely influential descriptive model of choice, but one which lacks any unifying principle that might explain why such preferences exist. Recently there have been several attempts to connect PT with risk-sensitive foraging theory (RSFT), a normative framework developed by evolutionary biologists to explain how animals should choose optimally when faced with uncertain foraging options. Although this seems a promising direction, here we show that current approaches are overly simplistic, and, despite their claims, they leave key features of PT unaccounted for. A common problem is the failure to appreciate the central concept of reproductive value in RSFT, which depends on the decision maker's current state and the particular situation it faces. Reproductive value provides a common currency in which decisions can be compared in a logical way. In contrast, existing models provide no rational justification for the reference state in PT. Evolutionary approaches to understanding PT preferences must confront this basic problem.This work was supported by the European Research Council (ERC Advanced Grant 250209 to AIH)

Evolution of trust and trustworthiness: social awareness favours personality differences

This is the final version of the article. Available from the publisher via the DOI in this record.Interest in the evolution and maintenance of personality is burgeoning. Individuals of diverse animal species differ in their aggressiveness, fearfulness, sociability and activity. Strong trade-offs, mutation-selection balance, spatio-temporal fluctuations in selection, frequency dependence and good-genes mate choice are invoked to explain heritable personality variation, yet for continuous behavioural traits, it remains unclear which selective force is likely to maintain distinct polymorphisms. Using a model of trust and cooperation, we show how allowing individuals to monitor each other's cooperative tendencies, at a cost, can select for heritable polymorphisms in trustworthiness. This variation, in turn, favours costly 'social awareness' in some individuals. Feedback of this sort can explain the individual differences in trust and trustworthiness so often documented by economists in experimental public goods games across a range of cultures. Our work adds to growing evidence that evolutionary game theorists can no longer afford to ignore the importance of real world inter-individual variation in their models.P.A.S. was funded by NERC grant NER/A/S/2003/00616 to J.M.M. and A.I.H

Adaptive learning can result in a failure to profit from good conditions: implications for understanding depression.

Published onlineThis is the final version of the article. It first appeared from Oxford University Press via http://dx.doi.org/10.1093/emph/eov009BACKGROUND AND OBJECTIVES: Depression is a major medical problem diagnosed in an increasing proportion of people and for which commonly prescribed psychoactive drugs are frequently ineffective. Development of treatment options may be facilitated by an evolutionary perspective; several adaptive reasons for proneness to depression have been proposed. A common feature of many explanations is that depressive behaviour is a way to avoid costly effort where benefits are small and/or unlikely. However, this viewpoint fails to explain why low mood persists when the situation improves. We investigate whether a behavioural rule that is adapted to a stochastically changing world can cause inactivity which appears similar to the effect of depression, in that it persists after the situation has improved. METHODOLOGY: We develop an adaptive learning model in which an individual has repeated choices of whether to invest costly effort that may result in a net benefit. Investing effort also provides information about the current conditions and rates of change of the conditions. RESULTS: An individual following the optimal behavioural strategy may sometimes remain inactive when conditions are favourable (i.e. when it would be better to invest effort) when it is poorly informed about the current environmental state. Initially benign conditions can predispose an individual to inactivity after a relatively brief period of negative experiences. CONCLUSIONS AND IMPLICATIONS: Our approach suggests that the antecedent factors causing depressed behaviour could go much further back in an individual s history than is currently appreciated. The insights from our approach have implications for the ongoing debate about best treatment options for patients with depressive symptoms.This work was supported by the European Research Council (Evomech Advanced Grant 250 209 to A.I.H.)

The influence of the food–predation trade-off on the foraging behaviour of central-place foragers

PublishedThis is the author accepted manuscript. The final version is available from Springer via the DOI in this record.Social animals that bring resources to a central place have commonly been used to test the predictions of optimal foraging models. Such animals are amenable to test because they do not themselves reproduce, and so we might expect them to be selected to maximise some measure of input of food to the colony. Several currencies have been proposed to predict behaviour, such as net rate, efficiency, and the ratio of the mortality rate to energy gain rate. Observations on social animals, especially bees, show mixed support for each currency. Here, we examine how these currencies can be united by considering the expected lifetime input of energy to the colony in a representative study of patch residence time. This currency explains partial loads because it leads to the prediction that the energy that a forager delivers to the colony over its lifetime is maximised by returning to the colony after a critical amount has been collected, even if energy is gained at a constant rate. We show that the extent to which foraging carries a greater mortality risk than travelling controls whether this currency makes similar predictions to net rate or to efficiency. We assess the evidence that bee behaviour actually maximises this currency and argue that mortality risk at resource sites is likely to be a critical determinant of foraging strategies in central-place foragers

Fatness and fitness: Exposing the logic of evolutionary explanations for obesity

To explore the logic of evolutionary explanations of obesity we modelled food consumption in an animal that minimizes mortality (starvation plus predation) by switching between activities that differ in energy gain and predation. We show that if switching does not incur extra predation risk, the animal should have a single threshold level of reserves above which it performs the safe activity and below which it performs the dangerous activity. The value of the threshold is determined by the environmental conditions, implying that animals should have variable ‘set points’. Selection pressure to prevent energy stores exceeding the optimal level is usually weak, suggesting that immediate rewards might easily overcome the controls against becoming overweight. The risk of starvation can have a strong influence on the strategy even when starvation is extremely uncommon, so the incidence of mortality during famine in human history may be unimportant for explanations for obesity. If there is an extra risk of switching between activities, the animal should have two distinct thresholds: one to initiate weight gain and one to initiate weight loss. Contrary to the dual intervention point model, these thresholds will be inter-dependent, such that altering the predation risk alters the location of both thresholds; a result that undermines the evolutionary basis of the drifty genes hypothesis. Our work implies that understanding the causes of obesity can benefit from a better understanding of how evolution shapes the mechanisms that control body weight

Behavioural specialization and learning in social networks

This is the final version. Available on open access from the Royal Society via the DOI in this recordData accessibility: C++ source code for the individual-based simulations is available at GitHub, together with instructions for compilation on a Linux operating system: https://github.com/oleimar/behavspec. Electronic supplementary material is available online [46].Interactions in social groups can promote behavioural specialization. One way this can happen is when individuals engage in activities with two behavioural options and learn which option to choose. We analyse interactions in groups where individuals learn from playing games with two actions and negatively frequency-dependent payoffs, such as producer-scrounger, caller-satellite, or hawk-dove games. Group members are placed in social networks, characterized by the group size and the number of neighbours to interact with, ranging from just a few neighbours to interactions between all group members. The networks we analyse include ring lattices and the much-studied small-world networks. By implementing two basic reinforcement-learning approaches, action-value learning and actor-critic learning, in different games, we find that individuals often show behavioural specialization. Specialization develops more rapidly when there are few neighbours in a network and when learning rates are high. There can be learned specialization also with many neighbours, but we show that, for action-value learning, behavioural consistency over time is higher with a smaller number of neighbours. We conclude that frequency-dependent competition for resources is a main driver of specialization. We discuss our theoretical results in relation to experimental and field observations of behavioural specialization in social situations.Swedish Research Counci

The Perfect Family: Decision Making in Biparental Care

Background Previous theoretical work on parental decisions in biparental care has emphasized the role of the conflict between evolutionary interests of parents in these decisions. A prominent prediction from this work is that parents should compensate for decreases in each other\u27s effort, but only partially so. However, experimental tests that manipulate parents and measure their responses fail to confirm this prediction. At the same time, the process of parental decision making has remained unexplored theoretically. We develop a model to address the discrepancy between experiments and the theoretical prediction, and explore how assuming different decision making processes changes the prediction from the theory. Model Description We assume that parents make decisions in behavioral time. They have a fixed time budget, and allocate it between two parental tasks: provisioning the offspring and defending the nest. The proximate determinant of the allocation decisions are parents\u27 behavioral objectives. We assume both parents aim to maximize the offspring production from the nest. Experimental manipulations change the shape of the nest production function. We consider two different scenarios for how parents make decisions: one where parents communicate with each other and act together (the perfect family), and one where they do not communicate, and act independently (the almost perfect family). Conclusions/Significance The perfect family model is able to generate all the types of responses seen in experimental studies. The kind of response predicted depends on the nest production function, i.e. how parents\u27 allocations affect offspring production, and the type of experimental manipulation. In particular, we find that complementarity of parents\u27 allocations promotes matching responses. In contrast, the relative responses do not depend on the type of manipulation in the almost perfect family model. These results highlight the importance of the interaction between nest production function and how parents make decisions, factors that have largely been overlooked in previous models

Male age mediates reproductive investment and response to paternity assurance.

addresses: Centre for Ecology and Conservation, College of Life and Environmental Sciences, University of Exeter, , Cornwall Campus, Penryn TR10 9EZ, UK, Department of Genetics, University of Georgia, , Athens, GA 30602, USA.types: Journal ArticleTheory predicts that male response to reduced paternity will depend on male state and interactions between the sexes. If there is little chance of reproducing again, then males should invest heavily in current offspring, regardless of their share in paternity. We tested this by manipulating male age and paternity assurance in the burying beetle Nicrophorus vespilloides. We found older males invested more in both mating effort and parental effort than younger males. Furthermore, male age, a component of male state, mediated male response to perceived paternity. Older males provided more prenatal care, whereas younger males provided less prenatal care, when perceived paternity was low. Adjustments in male care, however, did not influence selection acting indirectly on parents, through offspring performance. This is because females adjusted their care in response to the age of their partner, providing less care when paired with older males than younger males. As a result offspring, performance did not differ between treatments. Our study shows, for the first time, that a male state variable is an important modifier of paternity-parental care trade-offs and highlights the importance of social interactions between males and females during care in determining male response to perceived paternity