SOVEREIGN: An Autonomous Neural System for Incrementally Learning Planned Action Sequences to Navigate Towards a Rewarded Goal

How do reactive and planned behaviors interact in real time? How are sequences of such behaviors released at appropriate times during autonomous navigation to realize valued goals? Controllers for both animals and mobile robots, or animats, need reactive mechanisms for exploration, and learned plans to reach goal objects once an environment becomes familiar. The SOVEREIGN (Self-Organizing, Vision, Expectation, Recognition, Emotion, Intelligent, Goaloriented Navigation) animat model embodies these capabilities, and is tested in a 3D virtual reality environment. SOVEREIGN includes several interacting subsystems which model complementary properties of cortical What and Where processing streams and which clarify similarities between mechanisms for navigation and arm movement control. As the animat explores an environment, visual inputs are processed by networks that are sensitive to visual form and motion in the What and Where streams, respectively. Position-invariant and sizeinvariant recognition categories are learned by real-time incremental learning in the What stream. Estimates of target position relative to the animat are computed in the Where stream, and can activate approach movements toward the target. Motion cues from animat locomotion can elicit head-orienting movements to bring a new target into view. Approach and orienting movements are alternately performed during animat navigation. Cumulative estimates of each movement are derived from interacting proprioceptive and visual cues. Movement sequences are stored within a motor working memory. Sequences of visual categories are stored in a sensory working memory. These working memories trigger learning of sensory and motor sequence categories, or plans, which together control planned movements. Predictively effective chunk combinations are selectively enhanced via reinforcement learning when the animat is rewarded. Selected planning chunks effect a gradual transition from variable reactive exploratory movements to efficient goal-oriented planned movement sequences. Volitional signals gate interactions between model subsystems and the release of overt behaviors. The model can control different motor sequences under different motivational states and learns more efficient sequences to rewarded goals as exploration proceeds.Riverside Reserach Institute; Defense Advanced Research Projects Agency (N00014-92-J-4015); Air Force Office of Scientific Research (F49620-92-J-0225); National Science Foundation (IRI 90-24877, SBE-0345378); Office of Naval Research (N00014-92-J-1309, N00014-91-J-4100, N00014-01-1-0624, N00014-01-1-0624); Pacific Sierra Research (PSR 91-6075-2

Motivation and the theory of the firm

This paper proposes to revisit the debate on the theory of the firm using motivation theory as the primary analytical tool.theory of the firm; motivation theory

Frontostriatal Maturation Predicts Cognitive Control Failure to Appetitive Cues in Adolescents

Adolescent risk-taking is a public health issue that increases the odds of poor lifetime outcomes. One factor thought to influence adolescents' propensity for risk-taking is an enhanced sensitivity to appetitive cues, relative to an immature capacity to exert sufficient cognitive control. We tested this hypothesis by characterizing interactions among ventral striatal, dorsal striatal, and prefrontal cortical regions with varying appetitive load using fMRI scanning. Child, teen, and adult participants performed a go/no-go task with appetitive (happy faces) and neutral cues (calm faces). Impulse control to neutral cues showed linear improvement with age, whereas teens showed a nonlinear reduction in impulse control to appetitive cues. This performance decrement in teens was paralleled by enhanced activity in the ventral striatum. Prefrontal cortical recruitment correlated with overall accuracy and showed a linear response with age for no-go versus go trials. Connectivity analyses identified a ventral frontostriatal circuit including the inferior frontal gyrus and dorsal striatum during no-go versus go trials. Examining recruitment developmentally showed that teens had greater between-subject ventral-dorsal striatal coactivation relative to children and adults for happy no-go versus go trials. These findings implicate exaggerated ventral striatal representation of appetitive cues in adolescents relative to an intermediary cognitive control response. Connectivity and coactivity data suggest these systems communicate at the level of the dorsal striatum differentially across development. Biased responding in this system is one possible mechanism underlying heightened risk-taking during adolescence

Neural Dynamics of Autistic Behaviors: Cognitive, Emotional, and Timing Substrates

What brain mechanisms underlie autism and how do they give rise to autistic behavioral symptoms? This article describes a neural model, called the iSTART model, which proposes how cognitive, emotional, timing, and motor processes may interact together to create and perpetuate autistic symptoms. These model processes were originally developed to explain data concerning how the brain controls normal behaviors. The iSTART model shows how autistic behavioral symptoms may arise from prescribed breakdowns in these brain processes.Air Force Office of Scientific Research (F49620-01-1-0397); Office of Naval Research (N00014-01-1-0624

Dopaminergic and Non-Dopaminergic Value Systems in Conditioning and Outcome-Specific Revaluation

Animals are motivated to choose environmental options that can best satisfy current needs. To explain such choices, this paper introduces the MOTIVATOR (Matching Objects To Internal Values Triggers Option Revaluations) neural model. MOTIVATOR describes cognitiveemotional interactions between higher-order sensory cortices and an evaluative neuraxis composed of the hypothalamus, amygdala, and orbitofrontal cortex. Given a conditioned stimulus (CS), the model amygdala and lateral hypothalamus interact to calculate the expected current value of the subjective outcome that the CS predicts, constrained by the current state of deprivation or satiation. The amygdala relays the expected value information to orbitofrontal cells that receive inputs from anterior inferotemporal cells, and medial orbitofrontal cells that receive inputs from rhinal cortex. The activations of these orbitofrontal cells code the subjective values of objects. These values guide behavioral choices. The model basal ganglia detect errors in CS-specific predictions of the value and timing of rewards. Excitatory inputs from the pedunculopontine nucleus interact with timed inhibitory inputs from model striosomes in the ventral striatum to regulate dopamine burst and dip responses from cells in the substantia nigra pars compacta and ventral tegmental area. Learning in cortical and striatal regions is strongly modulated by dopamine. The model is used to address tasks that examine food-specific satiety, Pavlovian conditioning, reinforcer devaluation, and simultaneous visual discrimination. Model simulations successfully reproduce discharge dynamics of known cell types, including signals that predict saccadic reaction times and CS-dependent changes in systolic blood pressure.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); National Institutes of Health (R29-DC02952, R01-DC007683); National Science Foundation (IIS-97-20333, SBE-0354378); Office of Naval Research (N00014-01-1-0624

Motivation

The ability of humans and animals to survive in a constantly changing environment is a testament to the power of biological processes. At any given instant in our lives, we are faced with an enormous number of sensory stimuli, and we can typically generate an equally large number of behaviors. How do we learn to ignore irrelevant information and suppress inappropriate behavior so that we may function in a complex environment? In this chapter we discuss motivation, the internal force that produces actions reflecting the interactions between our needs and the demands of our environment. We will first discuss what psychologists mean when they refer to motivation, and then review neural network theories that can expbin how motivation arises within biological nervous systems.Sloan Fellowship (BR-3122): Air Force Office of Scientific Research (F49620-92-J-0499, F49620-92-J-0334

Structures, inner values, hierarchies and stages: essentials for developmental robot architectures

In this paper we try to locate the essential components needed for a developmental robot architecture. We take the vocabulary and the main concepts from Piaget’s genetic epistemology and Vygotsky’s activity theory. After proposing an outline for a general developmental architecture, we describe the architectures that we have been developing in the recent years - Petitagé and Vygovorotsky. According to this outline, various contemporary works in autonomous agents can be classified, in an attempt to get a glimpse into the big picture and make the advances and open problems visible