Layered control architectures in natural and artificial systems

We review recent research in robotics and neuroscience with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ [9] subsumption architecture for robot control with a part of the neuroscience literature that can be interpreted as demonstrating hierarchical control systems in animal brains. We focus first on work that follows the tradition of Hughlings Jackson [23] who, in neuroscience and neuropsychology, is particularly associated with the notion of layered competence. From this perspective we further argue that recent work on the defense system of the rat can be interpreted by analogy to Brooks’ subsumption architecture. An important focus is the role of multiple learning systems in the brain, and of hierarchical learning mechanisms in the rat defense system

Layered control architectures in natural and artificial systems

We review recent research in robotics and neuroscience with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ [9] subsumption architecture for robot control with a part of the neuroscience literature that can be interpreted as demonstrating hierarchical control systems in animal brains. We focus first on work that follows the tradition of Hughlings Jackson [23] who, in neuroscience and neuropsychology, is particularly associated with the notion of layered competence. From this perspective we further argue that recent work on the defense system of the rat can be interpreted by analogy to Brooks’ subsumption architecture. An important focus is the role of multiple learning systems in the brain, and of hierarchical learning mechanisms in the rat defense system

The basal ganglia: A vertebrate solution to the selection problem?

A selection problem arises whenever two or more competing systems seek simultaneous access to a restricted resource. Consideration of several selection architectures suggests there are significant advantages for systems which incorporate a central switching mechanism. We propose that the vertebrate basal ganglia have evolved as a centralized selection device, specialized to resolve conflicts over access to limited motor and cognitive resources. Analysis of basal ganglia functional architecture and its position within a wider anatomical framework suggests it can satisfy many of the requirements expected of an efficient selection mechanism

Is the short-latency dopamine response too short to signal reward error?

Unexpected stimuli that are behaviourally significant have the capacity to elicit a short-latency, short-duration burst of firing in mesencephalic dopaminergic neurones. An influential interpretation of the experimental data that characterize this response proposes that dopaminergic neurones have a crucial role in reinforcement learning because they signal error in the prediction of future reward. In this article we propose a different functional role for this ‘short-latency dopamine response’ in the mechanisms that underlie associative learning. We suggest that the initial burst of dopaminergic-neurone firing could represent an essential component in the process of switching attentional and behavioural selections to unexpected, behaviourally important stimuli. This switching response could be a crucial prerequisite for associative learning and might be part of a general short-latency response that is mediated by catecholamines and prepares the organism for an appropriate reaction to biologically significant events. Any act which in a given situation produces satisfaction becomes associated with that situation so that when the situation recurs the act is more likely than before to recur also. E.L. Thorndike (1911)

The basal ganglia viewed as an action selection device

The action selection problem describes the task of resolving conflicts between the different functional systems that can control behavior. This paper reviews the role of the basal ganglia (BG) summarising evidence that they function within the vertebrate brain architecture as a specialized action selection device. There is a rich connectivity within the BG whose function is not well understood. We outline a new computational model of BG intrinsic pathways which demonstrates that these circuits could allow the BG to implement clean switching between competing functional systems

Empirically inspired simulated electro-mechanical model of the rat mystacial follicle-sinus complex

In whiskered animals, activity is evoked in the primary sensory afferent cells (trigeminal nerve) by mechanical stimulation of the whiskers. In some cell populations this activity is correlated well with continuous stimulus parameters such as whisker deflection magnitude, but in others it is observed to represent events such as whisker-stimulator contact or detachment. The transduction process is mediated by the mechanics of the whisker shaft and follicle-sinus complex (FSC), and the mechanics and electro-chemistry of mechanoreceptors within the FSC. An understanding of this transduction process and the nature of the primary neural codes generated is crucial for understanding more central sensory processing in the thalamus and cortex. However, the details of the peripheral processing are currently poorly understood. To overcome this deficiency in our knowledge, we constructed a simulated electro-mechanical model of the whisker-FSC-mechanoreceptor system in the rat and tested it against a variety of data drawn from the literature. The agreement was good enough to suggest that the model captures many of the key features of the peripheral whisker system in the rat

Emergence of visually-evoked reward expectation signals in dopamine neurons via the superior colliculus in V1 lesioned monkeys

Responses of midbrain dopamine (DA) neurons reflecting expected reward from sensory cues are critical for reward-based associative learning. However, critical pathways by which reward-related visual information is relayed to DA neurons remain unclear. To address this question, we investigated Pavlovian conditioning in macaque monkeys with unilateral primary visual cortex (V1) lesions (an animal model of 'blindsight'). Anticipatory licking responses to obtain juice drops were elicited in response to visual conditioned stimuli (CS) in the affected visual field. Subsequent pharmacological inactivation of the superior colliculus (SC) suppressed the anticipatory licking. Concurrent single unit recordings indicated that DA responses reflecting the reward expectation could be recorded in the absence of V1, and that these responses were also suppressed by SC inactivation. These results indicate that the subcortical visual circuit can relay reward-predicting visual information to DA neurons and integrity of the SC is necessary for visually-elicited classically conditioned responses after V1 lesion

A robot model of the basal ganglia: Behavior and intrinsic processing

The existence of multiple parallel loops connecting sensorimotor systems to the basal ganglia has given rise to proposals that these nuclei serve as a selection mechanism resolving competitions between the alternative actions available in a given context. A strong test of this hypothesis is to require a computational model of the basal ganglia to generate integrated selection sequences in an autonomous agent, we therefore describe a robot architecture into which such a model is embedded, and require it to control action selection in a robotic task inspired by animal observations. Our results demonstrate effective action selection by the embedded model under a wide range of sensory and motivational conditions. When confronted with multiple, high salience alternatives, the robot also exhibits forms of behavioral disintegration that show similarities to animal behavior in conflict situations. The model is shown to cast light on recent neurobiological findings concerning behavioral switching and sequencing

The robot basal ganglia : action selection by an embedded model of the basal ganglia

Action selection is the task of resolving conflicts between multiple sensorimotor systems seeking access to the final common motor path. Recently,1,2 we proposed that the basal ganglia may act to provide a biological solution to the problem of selection. To test this notion we have implemented a high level computational model of intrinsic basal ganglia circuitry and its interactions with simulated thalamocortical connections.3,4 The computational model was then exposed to the rigors of `real world’ action selection by embedding it within the control architecture of a small mobile robot.5 In a mock foraging task, the robot was required to select appropriate actions under changing sensory and motivational conditions, thereby generating sequences of integrated behavior. Our results demonstrate: (i) the computational model of basal ganglia switches effectively between competing channels depending on the dynamics of relative input ‘salience’; (ii) its performance is enhanced by inclusion of anatomically inspired thalamocortical circuitry; (iii) in the robot, the model demonstrates appropriate and clean switching between different actions and is able to generate coherent sequences of behavior