Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex

Neocortical neurons have thousands of excitatory synapses. It is a mystery how neurons integrate the input from so many synapses and what kind of large-scale network behavior this enables. It has been previously proposed that non-linear properties of dendrites enable neurons to recognize multiple patterns. In this paper we extend this idea by showing that a neuron with several thousand synapses arranged along active dendrites can learn to accurately and robustly recognize hundreds of unique patterns of cellular activity, even in the presence of large amounts of noise and pattern variation. We then propose a neuron model where some of the patterns recognized by a neuron lead to action potentials and define the classic receptive field of the neuron, whereas the majority of the patterns recognized by a neuron act as predictions by slightly depolarizing the neuron without immediately generating an action potential. We then present a network model based on neurons with these properties and show that the network learns a robust model of time-based sequences. Given the similarity of excitatory neurons throughout the neocortex and the importance of sequence memory in inference and behavior, we propose that this form of sequence memory is a universal property of neocortical tissue. We further propose that cellular layers in the neocortex implement variations of the same sequence memory algorithm to achieve different aspects of inference and behavior. The neuron and network models we introduce are robust over a wide range of parameters as long as the network uses a sparse distributed code of cellular activations. The sequence capacity of the network scales linearly with the number of synapses on each neuron. Thus neurons need thousands of synapses to learn the many temporal patterns in sensory stimuli and motor sequences.Comment: Submitted for publicatio

An analogue approach for the processing of biomedical signals

Constant device scaling has signifcantly boosted electronic systems design in the digital domain enabling incorporation of more functionality within small silicon area and at the same time allows high-speed computation. This trend has been exploited for developing high-performance miniaturised systems in a number of application areas like communication, sensor network, main frame computers, biomedical information processing etc. Although successful, the associated cost comes in the form of high leakage power dissipation and systems reliability. With the increase of customer demands for smarter and faster technologies and with the advent of pervasive information processing, these issues may prove to be limiting factors for application of traditional digital design techniques. Furthermore, as the limit of device scaling is nearing, performance enhancement for the conventional digital system design methodology cannot be achieved any further unless innovations in new materials and new transistor design are made. To this end, an alternative design methodology that may enable performance enhancement without depending on device scaling is much sought today.Analogue design technique is one of these alternative techniques that have recently gained considerable interests. Although it is well understood that there are several roadblocks still to be overcome for making analogue-based system design for information processing as the main-stream design technique (e.g., lack of automated design tool, noise performance, efficient passive components implementation on silicon etc.), it may offer a faster way of realising a system with very few components and therefore may have a positive implication on systems performance enhancement. The main aim of this thesis is to explore possible ways of information processing using analogue design techniques in particular in the field of biomedical systems