Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex

Neocortical neurons have thousands of excitatory synapses. It is a mystery how neurons integrate the input from so many synapses and what kind of large-scale network behavior this enables. It has been previously proposed that non-linear properties of dendrites enable neurons to recognize multiple patterns. In this paper we extend this idea by showing that a neuron with several thousand synapses arranged along active dendrites can learn to accurately and robustly recognize hundreds of unique patterns of cellular activity, even in the presence of large amounts of noise and pattern variation. We then propose a neuron model where some of the patterns recognized by a neuron lead to action potentials and define the classic receptive field of the neuron, whereas the majority of the patterns recognized by a neuron act as predictions by slightly depolarizing the neuron without immediately generating an action potential. We then present a network model based on neurons with these properties and show that the network learns a robust model of time-based sequences. Given the similarity of excitatory neurons throughout the neocortex and the importance of sequence memory in inference and behavior, we propose that this form of sequence memory is a universal property of neocortical tissue. We further propose that cellular layers in the neocortex implement variations of the same sequence memory algorithm to achieve different aspects of inference and behavior. The neuron and network models we introduce are robust over a wide range of parameters as long as the network uses a sparse distributed code of cellular activations. The sequence capacity of the network scales linearly with the number of synapses on each neuron. Thus neurons need thousands of synapses to learn the many temporal patterns in sensory stimuli and motor sequences.Comment: Submitted for publicatio

Cognitive networks: brains, internet, and civilizations

In this short essay, we discuss some basic features of cognitive activity at several different space-time scales: from neural networks in the brain to civilizations. One motivation for such comparative study is its heuristic value. Attempts to better understand the functioning of "wetware" involved in cognitive activities of central nervous system by comparing it with a computing device have a long tradition. We suggest that comparison with Internet might be more adequate. We briefly touch upon such subjects as encoding, compression, and Saussurean trichotomy langue/langage/parole in various environments.Comment: 16 page

HTM-MAT: An online prediction software toolbox based on cortical machine learning algorithm

HTM-MAT is a MATLAB based toolbox for implementing cortical learning algorithms (CLA) including related cortical-like algorithms that possesses spatiotemporal properties. CLA is a suite of predictive machine learning algorithms developed by Numenta Inc. and is based on the hierarchical temporal memory (HTM). This paper presents an implementation of HTM-MAT with several illustrative examples including several toy datasets and compared with two sequence learning applications employing state-of-the-art algorithms - the recurrentjs based on the Long Short-Term Memory (LSTM) algorithm and OS-ELM which is based on an online sequential version of the Extreme Learning Machine. The performance of HTM-MAT using two historical benchmark datasets and one real world dataset is also compared with one of the existing sequence learning applications, the OS-ELM. The results indicate that HTM-MAT predictions are indeed competitive and can outperform OS-ELM in sequential prediction tasks.Comment: This research is currently under review in a Journal. Contents might vary from final published versio

The use of computer vision for automated control of gypsum feed and conveyor belt movement in the production of a gypsum plasterboard

The paper presents a description of the gypsum dough spill analysis method on the basis of computer vision for automated control of gypsum feed and conveyor belt speed. The task of computer vision is to control the form of gypsum dough spill on the basis of optical analysis of video data in real time. © Published under licence by IOP Publishing Ltd