Towards a continuous dynamic model of the Hopfield theory on neuronal interaction and memory storage

The purpose of this work is to study the Hopfield model for neuronal interaction and memory storage, in particular the convergence to the stored patterns. Since the hypothesis of symmetric synapses is not true for the brain, we will study how we can extend it to the case of asymmetric synapses using a probabilistic approach. We then focus on the description of another feature of the memory process and brain: oscillations. Using the Kuramoto model we will be able to describe them completely, gaining the presence of synchronization between neurons. Our aim is therefore to understand how and why neurons can be seen as oscillators and to establish a strong link between this model and the Hopfield approach

Coding and learning of chemosensor array patterns in a neurodynamic model of the olfactory system

Arrays of broadly-selective chemical sensors, also known as electronic noses, have been developed during the past two decades as a low-cost and high-throughput alternative to analytical instruments for the measurement of odorant chemicals. Signal processing in these gas-sensor arrays has been traditionally performed by means of statistical and neural pattern recognition techniques. The objective of this dissertation is to develop new computational models to process gas sensor array signals inspired by coding and learning mechanisms of the biological olfactory system. We have used a neurodynamic model of the olfactory system, the KIII, to develop and demonstrate four odor processing computational functions: robust recovery of overlapping patterns, contrast enhancement, background suppression, and novelty detection. First, a coding mechanism based on the synchrony of neural oscillations is used to extract information from the associative memory of the KIII model. This temporal code allows the KIII to recall overlapping patterns in a robust manner. Second, a new learning rule that combines Hebbian and anti-Hebbian terms is proposed. This learning rule is shown to achieve contrast enhancement on gas-sensor array patterns. Third, a new local learning mechanism based on habituation is proposed to perform odor background suppression. Combining the Hebbian/anti-Hebbian rule and the local habituation mechanism, the KIII is able to suppress the response to continuously presented odors, facilitating the detection of the new ones. Finally, a new learning mechanism based on anti-Hebbian learning is proposed to perform novelty detection. This learning mechanism allows the KIII to detect the introduction of new odors even in the presence of strong backgrounds. The four computational models are characterized with synthetic data and validated on gas sensor array patterns obtained from an e-nose prototype developed for this purpose

Multiuser detection employing recurrent neural networks for DS-CDMA systems.

Thesis (M.Sc.Eng.)-University of KwaZulu-Natal, 2006.Over the last decade, access to personal wireless communication networks has evolved to a point of necessity. Attached to the phenomenal growth of the telecommunications industry in recent times is an escalating demand for higher data rates and efficient spectrum utilization. This demand is fuelling the advancement of third generation (3G), as well as future, wireless networks. Current 3G technologies are adding a dimension of mobility to services that have become an integral part of modem everyday life. Wideband code division multiple access (WCDMA) is the standardized multiple access scheme for 3G Universal Mobile Telecommunication System (UMTS). As an air interface solution, CDMA has received considerable interest over the past two decades and a great deal of current research is concerned with improving the application of CDMA in 3G systems. A factoring component of CDMA is multiuser detection (MUD), which is aimed at enhancing system capacity and performance, by optimally demodulating multiple interfering signals that overlap in time and frequency. This is a major research problem in multipoint-to-point communications. Due to the complexity associated with optimal maximum likelihood detection, many different sub-optimal solutions have been proposed. This focus of this dissertation is the application of neural networks for MUD, in a direct sequence CDMA (DS-CDMA) system. Specifically, it explores how the Hopfield recurrent neural network (RNN) can be employed to give yet another suboptimal solution to the optimization problem of MUD. There is great scope for neural networks in fields encompassing communications. This is primarily attributed to their non-linearity, adaptivity and key function as data classifiers. In the context of optimum multiuser detection, neural networks have been successfully employed to solve similar combinatorial optimization problems. The concepts of CDMA and MUD are discussed. The use of a vector-valued transmission model for DS-CDMA is illustrated, and common linear sub-optimal MUD schemes, as well as the maximum likelihood criterion, are reviewed. The performance of these sub-optimal MUD schemes is demonstrated. The Hopfield neural network (HNN) for combinatorial optimization is discussed. Basic concepts and techniques related to the field of statistical mechanics are introduced and it is shown how they may be employed to analyze neural classification. Stochastic techniques are considered in the context of improving the performance of the HNN. A neural-based receiver, which employs a stochastic HNN and a simulated annealing technique, is proposed. Its performance is analyzed in a communication channel that is affected by additive white Gaussian noise (AWGN) by way of simulation. The performance of the proposed scheme is compared to that of the single-user matched filter, linear decorrelating and minimum mean-square error detectors, as well as the classical HNN and the stochastic Hopfield network (SHN) detectors. Concluding, the feasibility of neural networks (in this case the HNN) for MUD in a DS-CDMA system is explored by quantifying the relative performance of the proposed model using simulation results and in view of implementation issues

Neural processes underpinning episodic memory

Episodic memory is the memory for our personal past experiences. Although numerous functional magnetic resonance imaging (fMRI) studies investigating its neural basis have revealed a consistent and distributed network of associated brain regions, surprisingly little is known about the contributions individual brain areas make to the recollective experience. In this thesis I address this fundamental issue by employing a range of different experimental techniques including neuropsychological testing, virtual reality environments, whole brain and high spatial resolution fMRI, and multivariate pattern analysis. Episodic memory recall is widely agreed to be a reconstructive process, one that is known to be critically reliant on the hippocampus. I therefore hypothesised that the same neural machinery responsible for reconstruction might also support ‘constructive’ cognitive functions such as imagination. To test this proposal, patients with focal damage to the hippocampus bilaterally were asked to imagine new experiences and were found to be impaired relative to matched control participants. Moreover, driving this deficit was a lack of spatial coherence in their imagined experiences, pointing to a role for the hippocampus in binding together the disparate elements of a scene. A subsequent fMRI study involving healthy participants compared the recall of real memories with the construction of imaginary memories. This revealed a fronto-temporo-parietal network in common to both tasks that included the hippocampus, ventromedial prefrontal, retrosplenial and parietal cortices. Based on these results I advanced the notion that this network might support the process of ‘scene construction’, defined as the generation and maintenance of a complex and coherent spatial context. Furthermore, I argued that this scene construction network might underpin other important cognitive functions besides episodic memory and imagination, such as navigation and thinking about the future. It is has been proposed that spatial context may act as the scaffold around which episodic memories are built. Given the hippocampus appears to play a critical role in imagination by supporting the creation of a rich coherent spatial scene, I sought to explore the nature of this hippocampal spatial code in a novel way. By combining high spatial resolution fMRI with multivariate pattern analysis techniques it proved possible to accurately determine where a subject was located in a virtual reality environment based solely on the pattern of activity across hippocampal voxels. For this to have been possible, the hippocampal population code must be large and non-uniform. I then extended these techniques to the domain of episodic memory by showing that individual memories could be accurately decoded from the pattern of activity across hippocampal voxels, thus identifying individual memory traces. I consider these findings together with other recent advances in the episodic memory field, and present a new perspective on the role of the hippocampus in episodic recollection. I discuss how this new (and preliminary) framework compares with current prevailing theories of hippocampal function, and suggest how it might account for some previously contradictory data

Artificial general intelligence: Proceedings of the Second Conference on Artificial General Intelligence, AGI 2009, Arlington, Virginia, USA, March 6-9, 2009

Artificial General Intelligence (AGI) research focuses on the original and ultimate goal of AI – to create broad human-like and transhuman intelligence, by exploring all available paths, including theoretical and experimental computer science, cognitive science, neuroscience, and innovative interdisciplinary methodologies. Due to the difficulty of this task, for the last few decades the majority of AI researchers have focused on what has been called narrow AI – the production of AI systems displaying intelligence regarding specific, highly constrained tasks. In recent years, however, more and more researchers have recognized the necessity – and feasibility – of returning to the original goals of the field. Increasingly, there is a call for a transition back to confronting the more difficult issues of human level intelligence and more broadly artificial general intelligence

Training issues and learning algorithms for feedforward and recurrent neural networks

Ph.DDOCTOR OF PHILOSOPH

A Multiple-Systems Approach in the Symbolic Modelling of Human Vision

For most of the thirty years or so of machine vision research, activity has been concentrated mainly in the domain of metric-based approaches: there has been negligible attention to the psychological factors in human vision. With the recent resurgence of interest in neural systems, that is now changing. This thesis discusses relevant aspects of basic visual neuroanatomy, and psychological phenomena, in an attempt to relate the concepts to a model of human vision and the prospective goals of future machine vision systems. It is suggested that, while biological vision is complex, the underlying mechanisms of human vision are more tractable than is often believed. We also argue here that the controversial subject of direct vision plays a crucial role in natural vision, and we attempt to relate this to the model. The recognition of massive parallelism in natural vision has led to proposals for emulating aspects of neural networks in technology. The systems model developed in this work demonstrates software-simulated cellular automata (CAs) in the role of mainly low-level image processing. It is shown that CAs are able to efficiently provide both conventional and neurally-inspired vision functions. The thesis also discusses the use of Prolog as the means of realising higher level image understanding. The symbolic processing developed is basic, but is nevertheless sufficient for the purposes of the present. demonstrations. Extensions to the concepts can be easily achieved. The modular systems approach adopted blends together several ideas and processes, and results in a more robust model of human vision that is able to translate a noisy real image into an accessible symbolic form for expert-domain interpretation