Pavlovian control of escape and avoidance

Contains fulltext : 195168.pdf (publisher's version ) (Open Access)To survive in complex environments, animals need to have mechanisms to select effective actions quickly, with minimal computational costs. As perhaps the computationally most parsimonious of these systems, Pavlovian control accomplishes this by hardwiring specific stereotyped responses to certain classes of stimuli. It is well documented that appetitive cues initiate a Pavlovian bias toward vigorous approach; however, Pavlovian responses to aversive stimuli are less well understood. Gaining a deeper understanding of aversive Pavlovian responses, such as active avoidance, is important given the critical role these behaviors play in several psychiatric conditions. The goal of the current study was to establish a behavioral and computational framework to examine aversive Pavlovian responses (activation vs. inhibition) depending on the proximity of an aversive state (escape vs. avoidance). We introduce a novel task in which participants are exposed to primary aversive (noise) stimuli and characterized behavior using a novel generative computational model. This model combines reinforcement learning and drift-diffusion models so as to capture effects of invigoration/inhibition in both explicit choice behavior as well as changes in RT. Choice and RT results both suggest that escape is associated with a bias for vigorous action, whereas avoidance is associated with behavioral inhibition. These results lay a foundation for future work that promise to provide insights into typical and atypical aversive Pavlovian responses involved in psychiatric disorders, allowing us to quantify both implicit and explicit indices of vigorous choice behavior in the context of aversion.12 p

From extinction learning to anxiety treatment: mind the gap

Laboratory models of extinction learning in animals and humans have the potential to illuminate methods for improving clinical treatment of fear-based clinical disorders. However, such translational research often neglects important differences between threat responses in animals and fear learning in humans, particularly as it relates to the treatment of clinical disorders. Specifically, the conscious experience of fear and anxiety, along with the capacity to deliberately engage top-down cognitive processes to modulate that experience, involves distinct brain circuitry and is measured and manipulated using different methods than typically used in laboratory research. This paper will identify how translational research that investigates methods of enhancing extinction learning can more effectively model such elements of human fear learning, and how doing so will enhance the relevance of this research to the treatment of fear-based psychological disorders.Published versio

Obstacle Avoidance by Means of an Operant Conditioning Model

This paper describes the application of a model of operant conditioning to the problem of obstacle avoidance with a wheeled mobile robot. The main characteristic of the applied model is that the robot learns to avoid obstacles through a learning-by-doing cycle without external supervision. A series of ultrasonic sensors act as Conditioned Stimuli (CS), while collisions act as an Unconditioned Stimulus (UCS). By experiencing a series of movements in a cluttered environment, the robot learns to avoid sensor activation patterns that predict collisions, thereby learning to avoid obstacles. Learning generalizes to arbitrary cluttered environments. In this work we describe our initial implementation using a computer simulation

Pain: A Precision Signal for Reinforcement Learning and Control.

Since noxious stimulation usually leads to the perception of pain, pain has traditionally been considered sensory nociception. But its variability and sensitivity to a broad array of cognitive and motivational factors have meant it is commonly viewed as inherently imprecise and intangibly subjective. However, the core function of pain is motivational-to direct both short- and long-term behavior away from harm. Here, we illustrate that a reinforcement learning model of pain offers a mechanistic understanding of how the brain supports this, illustrating the underlying computational architecture of the pain system. Importantly, it explains why pain is tuned by multiple factors and necessarily supported by a distributed network of brain regions, recasting pain as a precise and objectifiable control signal

A Model of Operant Conditioning for Adaptive Obstacle Avoidance

We have recently introduced a self-organizing adaptive neural controller that learns to control movements of a wheeled mobile robot toward stationary or moving targets, even when the robot's kinematics arc unknown, or when they change unexpectedly during operation. The model has been shown to outperform other traditional controllers, especially in noisy environments. This article describes a neural network module for obstacle avoidance that complements our previous work. The obstacle avoidance module is based on a model of classical and operant conditioning first proposed by Grossberg ( 1971). This module learns the patterns of ultrasonic sensor activation that predict collisions as the robot navigates in an unknown cluttered environment. Along with our original low-level controller, this work illustrates the potential of applying biologically inspired neural networks to the areas of adaptive robotics and control.Office of Naval Research (N00014-95-1-0409, Young Investigator Award

An automatic recording system for the study of escape from fear in rats

Escape from fear (EFF) is an active response to a conditioned stimulus (CS) previously paired with an unconditioned fearful stimulus (US), which typically leads to the termination of the CS. In this paradigm, animals acquire two distinct associations: S-S [CS-US] and R-O [responseoutcome] through Pavlovian and instrumental conditioning, respectively. The present study describes a computer controlled automatic recording system that captures the development of EFF and allows the determination of the respective roles of S-S and R-O associations in this process. We validated this system by showing that only rats subjected to a simultaneous CS-US conditioning (i.e., CS and US occur together at the beginning of each trial) acquired EFF, not those subjected to an unpaired CS-US conditioning. Paired rats had a progressively increased number of EFF and significantly shorter escape latencies than unpaired rats across the 5-trial blocks on the test day. However, during the conditioning phase, the unpaired rats emitted more 22 kHz ultrasonic vocalizations, a validated measure of conditioned reactive fear responses. Our results demonstrate that the acquisition of EFF is contingent upon pairing of the CS with the US, not simply the consequence of a high level of generalized fear. Because this commercially available system is capable of examining both conditioned active and reactive fear responses in a single setup, it could be used to determine the relative roles of S-S and R-O associations in EFF, the neurobiology of conditioned active fear response and neuropharmacology of psychotherapeutic drugs

An automatic recording system for the study of escape from fear in rats

Escape from fear (EFF) is an active response to a conditioned stimulus (CS) previously paired with an unconditioned fearful stimulus (US), which typically leads to the termination of the CS. In this paradigm, animals acquire two distinct associations: S-S [CS-US] and R-O [responseoutcome] through Pavlovian and instrumental conditioning, respectively. The present study describes a computer controlled automatic recording system that captures the development of EFF and allows the determination of the respective roles of S-S and R-O associations in this process. We validated this system by showing that only rats subjected to a simultaneous CS-US conditioning (i.e., CS and US occur together at the beginning of each trial) acquired EFF, not those subjected to an unpaired CS-US conditioning. Paired rats had a progressively increased number of EFF and significantly shorter escape latencies than unpaired rats across the 5-trial blocks on the test day. However, during the conditioning phase, the unpaired rats emitted more 22 kHz ultrasonic vocalizations, a validated measure of conditioned reactive fear responses. Our results demonstrate that the acquisition of EFF is contingent upon pairing of the CS with the US, not simply the consequence of a high level of generalized fear. Because this commercially available system is capable of examining both conditioned active and reactive fear responses in a single setup, it could be used to determine the relative roles of S-S and R-O associations in EFF, the neurobiology of conditioned active fear response and neuropharmacology of psychotherapeutic drugs