Rewards have a transient and task-specific effect on saccade latency and accuracy

The focus of this thesis was on investigating the key questions regarding the effectiveness of monetary rewards as a tool for behaviour change in rehabilitation. Firstly, do rewards consistently influence the eye movement behaviour in a neuro-typical human population? Secondly, do these effects persist once rewards are withdrawn? Finally, do these effects transfer to other unrewarded eye movement tasks? Nine experiments investigated the influence of monetary rewards on oculomotor function and attention in humans. Monetary rewards were found to consistently influence human saccadic behaviour such that faster eye movements were generated to rewarded locations compared to unrewarded locations. These effects persisted for a short period of time after rewards were withdrawn before extinguishing quickly. However, these hemifield-specific effects failed to transfer to any secondary unrewarded eye movement task, but instead produced a more general effect of reward in one experiment conducted. The present set of experiments have established a reward paradigm able to consistently produce behaviour change when rewards are present; however these effects were found to be context and task-specific. The findings of the present set of experiments have highlighted the transient nature of the effects of reward and provide a framework for the future use of monetary rewards as a tool for behaviour change. The findings provided by the present set of experiments can be harnessed in future to guide the effectiveness of monetary reinforcers in a neuro-atypical population

Genetic dissection of circuits underlying the modular structure of the Superior Colliculus

In order to successfully interact with the environment, animals need to produce accurate movements towards specific positions in space. A crucial region of the brain that guides such goal-oriented movements is the superior colliculus (SC), an evolutionary conserved structure of the midbrain. While several lines of research in different model organisms have confirmed that the SC contributes to the initiation of orienting movements, how functionally distinct neuronal groups within the SC are organized to support the production of such motor outputs is poorly understood. One of the reasons why the intrinsic circuit organization of the SC remains elusive is the lack of genetic characterization of the neuronal populations of the motor SC. Here, we performed RNAseq to screen for genetic markers for neuronal subpopulations in the motor SC. We identified a transcription factor, Pitx2, which is exclusively expressed in a subpopulation of glutamatergic neurons in the motor domain of the SC. Strikingly, this population of neurons displays a non-homogenous distribution within the motor layer of the SC, being organised in clusters along the mediolateral and anteroposterior axis. We mapped the pre-synaptic network and the post-synaptic targets of Pitx2ON neurons, unveiling that this modular population receives direct inputs from motor and sensory cortical regions, as well as several midbrain nuclei involved in movement control, and sends projection along the cephalomotor pathway. We then asked whether these modules may act as functional units, each integrating multimodal sensory information and encoding a specific feature of head movement, the main ethologically relevant orienting behaviour in rodents. Optogenetic activation of this modular population in freely moving animals produced a stereotyped, robust head motion characterised by a pronounced quantal nature; furthermore, the amplitude of the elicited head movement varied based on the modular unit activated. Our results suggest that distinct clusters of genetically defined neurons produce head displacement along a characteristic vector. In conclusion, we found that a population of premotor neurons in the SC is organised in a modular conformation and we suggest that such modularity may represent a physical implementation of a discontinuous motor map for orienting movements encoded in the mouse SC. Our work complements previous observations of periodicity in SC circuitry, as well as its afferent and efferent systems. Exploiting the genetic toolkit available in the mouse, our work begins to address the functional relevance of this modularity and paves the way for future experiments to investigate principles of sensorimotor integration in SC circuits.MR

Behavioral Learning in a Cognitive Neuromorphic Robot: An Integrative Approach

We present here a learning system using the iCub humanoid robot and the SpiNNaker neuromorphic chip to solve the real-world task of object-specific attention. Integrating spiking neural networks with robots introduces considerable complexity for questionable benefit if the objective is simply task performance. But, we suggest, in a cognitive robotics context, where the goal is understanding how to compute, such an approach may yield useful insights to neural architecture as well as learned behavior, especially if dedicated neural hardware is available. Recent advances in cognitive robotics and neuromorphic processing now make such systems possible. Using a scalable, structured, modular approach, we build a spiking neural network where the effects and impact of learning can be predicted and tested, and the network can be scaled or extended to new tasks automatically. We introduce several enhancements to a basic network and show how they can be used to direct performance toward behaviorally relevant goals. Results show that using a simple classical spike-timing-dependent plasticity (STDP) rule on selected connections, we can get the robot (and network) to progress from poor task-specific performance to good performance. Behaviorally relevant STDP appears to contribute strongly to positive learning: “do this” but less to negative learning: “don't do that.” In addition, we observe that the effect of structural enhancements tends to be cumulative. The overall system suggests that it is by being able to exploit combinations of effects, rather than any one effect or property in isolation, that spiking networks can achieve compelling, task-relevant behavior

A sensory system for robots using evolutionary artificial neural networks.

The thesis presents the research involved with developing an Intelligent Vision System for an animat that can analyse a visual scene in uncontrolled environments. Inspiration was drawn both from Biological Visual Systems and Artificial Image Recognition Systems. Several Biological Systems including the Insect, Toad and Human Visual Systems were studied alongside popular Pattern Recognition Systems such as fully connected Feedforward Networks, Modular Neural Networks and the Neocognitron. The developed system, called the Distributed Neural Network (DNN) was based on the sensory-motor connections in the common toad, Bufo Bufo. The sparsely connected network architecture has features of modularity enhanced by the presence of lateral inhibitory connections. It was implemented using Evolutionary Artificial Neural Networks (EANN). A novel method called FUSION was used to train the DNN, which is an amalgamation of several concepts of learning in Artificial Neural Networks such as Unsupervised Learning, Supervised Learning, Reinforcement Learning, Competitive Learning, Self-organisation and Fuzzy Logic. The DNN has unique feature detecting capabilities. When the DNN was tested using images that comprised of combination of features used in the training set, the DNN was successful in recognising individual features. The combinations of features were never used in the training set. This is a unique feature of the DNN trained using Fusion that cannot be matched by any other popular ANN architecture or training method. The system proved to be robust in dealing with New and Noisy Images. The unique features of the DNN make the network suitable for applications in robotics such as obstacle avoidance and terrain recognition, where the environment is unpredictable. The network can also be used in the field of Medical Imaging, Biometrics (Face and Finger Print Recognition) and Quality Inspection in the Food Processing Industry and applications in other uncontrolled environments

Artificial ontogenesis: a connectionist model of development

This thesis suggests that ontogenetic adaptive processes are important for generating intelligent beha- viour. It is thus proposed that such processes, as they occur in nature, need to be modelled and that such a model could be used for generating artificial intelligence, and specifically robotic intelligence. Hence, this thesis focuses on how mechanisms of intelligence are specified.A major problem in robotics is the need to predefine the behaviour to be followed by the robot. This makes design intractable for all but the simplest tasks and results in controllers that are specific to that particular task and are brittle when faced with unforeseen circumstances. These problems can be resolved by providing the robot with the ability to adapt the rules it follows and to autonomously create new rules for controlling behaviour. This solution thus depends on the predefinition of how rules to control behaviour are to be learnt rather than the predefinition of rules for behaviour themselves.Learning new rules for behaviour occurs during the developmental process in biology. Changes in the structure of the cerebral 'cortex underly behavioural and cognitive development throughout infancy and beyond. The uniformity of the neocortex suggests that there is significant computational uniformity across the cortex resulting from uniform mechanisms of development, and holds out the possibility of a general model of development. Development is an interactive process between genetic predefinition and environmental influences. This interactive process is constructive: qualitatively new behaviours are learnt by using simple abilities as a basis for learning more complex ones. The progressive increase in competence, provided by development, may be essential to make tractable the process of acquiring higher -level abilities.While simple behaviours can be triggered by direct sensory cues, more complex behaviours require the use of more abstract representations. There is thus a need to find representations at the correct level of abstraction appropriate to controlling each ability. In addition, finding the correct level of abstrac- tion makes tractable the task of associating sensory representations with motor actions. Hence, finding appropriate representations is important both for learning behaviours and for controlling behaviours. Representations can be found by recording regularities in the world or by discovering re- occurring pat- terns through repeated sensory -motor interactions. By recording regularities within the representations thus formed, more abstract representations can be found. Simple, non -abstract, representations thus provide the basis for learning more complex, abstract, representations.A modular neural network architecture is presented as a basis for a model of development. The pat- tern of activity of the neurons in an individual network constitutes a representation of the input to that network. This representation is formed through a novel, unsupervised, learning algorithm which adjusts the synaptic weights to improve the representation of the input data. Representations are formed by neurons learning to respond to correlated sets of inputs. Neurons thus became feature detectors or pat- tern recognisers. Because the nodes respond to patterns of inputs they encode more abstract features of the input than are explicitly encoded in the input data itself. In this way simple representations provide the basis for learning more complex representations. The algorithm allows both more abstract represent- ations to be formed by associating correlated, coincident, features together, and invariant representations to be formed by associating correlated, sequential, features together.The algorithm robustly learns accurate and stable representations, in a format most appropriate to the structure of the input data received: it can represent both single and multiple input features in both the discrete and continuous domains, using either topologically or non -topologically organised nodes. The output of one neural network is used to provide inputs for other networks. The robustness of the algorithm enables each neural network to be implemented using an identical algorithm. This allows a modular `assembly' of neural networks to be used for learning more complex abilities: the output activations of a network can be used as the input to other networks which can then find representations of more abstract information within the same input data; and, by defining the output activations of neurons in certain networks to have behavioural consequences it is possible to learn sensory -motor associations, to enable sensory representations to be used to control behaviour

Continuous Reinforced Snap-Drift Learning in a Neural Architecture for Proxylet Selection in Active Computer Networks

A new continuous learning method is used to optimise the selection of services in response to user requests in an active computer network simulation environment. The learning is an enhanced version of the ‘snap-drift’ algorithm, which employs the complementary concepts of fast, minimalist (snap) learning and slower drift (towards the input patterns) learning, in a non-stationary environment where new patterns arrive continually. Snap is based on Adaptive Resonance Theory, and drift on Learning Vector Quantisation. The new algorithm swaps its learning style between these two self-organisational modes when declining performance is detected, but maintains the same learning mode during episodes of improved performance. Performance updates occur at the end of each epoch. Reinforcement is implemented by enabling learning on any given pattern with a probability that increases linearly with declining performance. This method, which is capable of rapid re-learning, is used in the design of a modular neural network system: Performance-guided Adaptive Resonance Theory (P-ART). Simulations involving a requirement to continuously adapt to make appropirate decisions within a BT active computer network environment, demonstrate the learning is stable, and able to discover alternative solutions in rapid response to new performance requirements or significant changes in the stream of input patterns

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task