A Rescorla-Wagner Drift-Diffusion Model of Conditioning and Timing

Computational models of classical conditioning have made significant contributions to the theoretic understanding of associative learning, yet they still struggle when the temporal aspects of conditioning are taken into account. Interval timing models have contributed a rich variety of time representations and provided accurate predictions for the timing of responses, but they usually have little to say about associative learning. In this article we present a unified model of conditioning and timing that is based on the influential Rescorla-Wagner conditioning model and the more recently developed Timing Drift-Diffusion model. We test the model by simulating 10 experimental phenomena and show that it can provide an adequate account for 8, and a partial account for the other 2. We argue that the model can account for more phenomena in the chosen set than these other similar in scope models: CSC-TD, MS-TD, Learning to Time and Modular Theory. A comparison and analysis of the mechanisms in these models is provided, with a focus on the types of time representation and associative learning rule used

Human Factor and Energy Efficiency in Buildings: Motivating End-Users Behavioural Change

Energy efficiency in buildings does not only rely on efficient technical solutions and design of the building features, but is also highly dependent on how occupants decide to set their comfort criteria, as well as on their energy-related and environmental lifestyles. In this perspective, raising user awareness among occupants by training them to adopt a more “green” and energy-friendly behaviour has become a crucial aspect for reaching energy efficiency goals in buildings. Motivating occupants to change their behaviour can become a challenging task, especially if they are expected to internalize and adopt the new behaviour on a long term. This means that information and feedback provided to the occupants must be stimulating, easy to understand, and easy to adopt in the daily routine. In this context, first methodological progresses are here presented within an European project, designed to raise user awareness, reduce energy consumptions and improve health and IEQ conditions in different typologies of demonstration case studies by providing combined feedback on energy, indoor environmental quality, and health. In particular, this paper presents one out of five MOBISTYLE demonstration testbeds – a residence hotel - located in central Turin (IT). In detail, this paper describes the setup of a tailored engagement campaign for hotel apartments and the reception area. Based on selected monitored variables, user-friendly feedback was defined to provide the users with real-time information on energy use and environmental quality, as well as guidance on how to save energy and optimize consumption profiles while creating an acceptably comfortable and healthy indoor environment

Temporal-Difference Reinforcement Learning with Distributed Representations

Temporal-difference (TD) algorithms have been proposed as models of reinforcement learning (RL). We examine two issues of distributed representation in these TD algorithms: distributed representations of belief and distributed discounting factors. Distributed representation of belief allows the believed state of the world to distribute across sets of equivalent states. Distributed exponential discounting factors produce hyperbolic discounting in the behavior of the agent itself. We examine these issues in the context of a TD RL model in which state-belief is distributed over a set of exponentially-discounting “micro-Agents”, each of which has a separate discounting factor (γ). Each µAgent maintains an independent hypothesis about the state of the world, and a separate value-estimate of taking actions within that hypothesized state. The overall agent thus instantiates a flexible representation of an evolving world-state. As with other TD models, the value-error (δ) signal within the model matches dopamine signals recorded from animals in standard conditioning reward-paradigms. The distributed representation of belief provides an explanation for the decrease in dopamine at the conditioned stimulus seen in overtrained animals, for the differences between trace and delay conditioning, and for transient bursts of dopamine seen at movement initiation. Because each µAgent also includes its own exponential discounting factor, the overall agent shows hyperbolic discounting, consistent with behavioral experiments

A Program for At-Risk High School Students Informed by Evolutionary Science

Improving the academic performance of at-risk high school students has proven difficult, often calling for an extended day, extended school year, and other expensive measures. Here we report the results of a program for at-risk 9th and 10th graders in Binghamton, New York, called the Regents Academy that takes place during the normal school day and year. The design of the program is informed by the evolutionary dynamics of cooperation and learning, in general and for our species as a unique product of biocultural evolution. Not only did the Regents Academy students outperform their comparison group in a randomized control design, but they performed on a par with the average high school student in Binghamton on state-mandated exams. All students can benefit from the social environment provided for at-risk students at the Regents Academy, which is within the reach of most public school districts

Meta-analysis of the effects of predation on animal prey abundance: evidence from UK vertebrates

Background: Controlling vertebrate predators is one of the most widespread forms of wildlife management and it continues to cause conflict between stakeholders worldwide. It is important for managers and policy-makers to make decisions on this issue that are based on the best available scientific evidence. Therefore, it is first important to understand if there is indeed an impact of vertebrate predators on prey, and then to quantify this impact. Methodology/Principal Findings: Using the UK as a case study, we use a meta-analytical approach to review the available evidence to assess the effect of vertebrate predation on animal prey abundance. We find a significant effect of predators on prey abundance across our studies. On average, there is a 1.6 fold increase in prey abundance in the absence of predation. However, we show significant heterogeneity in effect sizes, and discuss how the method of predator control, whether the predator is native or non-native, and aspects of study design, may be potential causes. Conclusions/Significance: Our results allow some cautious policy recommendations to be made regarding the management of predator and prey populations. Meta-analysis is an important tool for understanding general patterns in the effect of predators on prey abundance across studies. Such an approach is especially valuable where management decisions need to be made in the absence of site-specific information

Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits

Background Over the last several years, it has become apparent that there are critical problems with the hypothesis that brain dopamine (DA) systems, particularly in the nucleus accumbens, directly mediate the rewarding or primary motivational characteristics of natural stimuli such as food. Hypotheses related to DA function are undergoing a substantial restructuring, such that the classic emphasis on hedonia and primary reward is giving way to diverse lines of research that focus on aspects of instrumental learning, reward prediction, incentive motivation, and behavioral activation. Objective The present review discusses dopaminergic involvement in behavioral activation and, in particular, emphasizes the effort-related functions of nucleus accumbens DA and associated forebrain circuitry. Results The effects of accumbens DA depletions on food-seeking behavior are critically dependent upon the work requirements of the task. Lever pressing schedules that have minimal work requirements are largely unaffected by accumbens DA depletions, whereas reinforcement schedules that have high work (e.g., ratio) requirements are substantially impaired by accumbens DA depletions. Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related decision making. Rats with accumbens DA depletions reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and instead, these rats select a less-effortful type of food-seeking behavior. Conclusions Along with prefrontal cortex and the amygdala, nucleus accumbens is a component of the brain circuitry regulating effort-related functions. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as energy-related disorders such as psychomotor slowing, fatigue, or anergia in depression