150 research outputs found
Scalable Digital Architecture of Hierarchical Temporal Memory Spatial Pooler
Hierarchical Temporal memory is an unsupervised machine learning algorithm. Inspired by the structural and functional properties of the human brain, it is capable of processing spatio-temporal signals which are used for data storage and predictions. The algorithm is composed of two main components; the Spatial Pooler and the Temporal Memory. The spatial pooler produces a sparse distribution representation for the given pattern. These generalized representations are used by the temporal memory to make predictions. Therefore, it is important to ensure that more generalized sparse distribution representations are obtained for the spatio-temporal data
patterns.
This work presents the digital design of spatial pooler implementation for an existing mathematical algorithm along with an analysis of its scalability for the target FPGA device. The digital design is implemented in two ways; Conventional and Parallel architectures. The architectures are compared in terms of speedup, area and power consumption. Based on the analysis of results, it is seen that the parallel approach is more efficient in terms of speed and power, with a negligible increase in device utilization. The spatial pooler design is evaluated against the standard MNIST dataset, obtaining up to 90% and 88% classication accuracy for the train and test data, respectively. Additionally, the designs are tested on the MNIST dataset, in the presence of noise, to determine its robustness. Fluctuations of up to 10% of the peak accuracy are observed during classication, and are noted in the classication accuracy plots for the dataset with noise. The design is synthesized for the Xilinx Virtex 7 family with a total power consumption of up to 260 mW
Memristor-Based HTM Spatial Pooler with On-Device Learning for Pattern Recognition
This article investigates hardware implementation of hierarchical temporal memory (HTM), a brain-inspired machine learning algorithm that mimics the key functions of the neocortex and is applicable to many machine learning tasks. Spatial pooler (SP) is one of the main parts of HTM, designed to learn the spatial information and obtain the sparse distributed representations (SDRs) of input patterns. The other part is temporal memory (TM) which aims to learn the temporal information of inputs. The memristor, which is an appropriate synapse emulator for neuromorphic systems, can be used as the synapse in SP and TM circuits. In this article, a memristor-based SP (MSP) circuit structure is designed to accelerate the execution of the SP algorithm. The presented MSP has properties of modeling both the synaptic permanence and the synaptic connection state within a single synapse, and on-device and parallel learning. Simulation results of statistic metrics and classification tasks on several real-world datasets substantiate the validity of MSP
ToyArchitecture: Unsupervised Learning of Interpretable Models of the World
Research in Artificial Intelligence (AI) has focused mostly on two extremes:
either on small improvements in narrow AI domains, or on universal theoretical
frameworks which are usually uncomputable, incompatible with theories of
biological intelligence, or lack practical implementations. The goal of this
work is to combine the main advantages of the two: to follow a big picture
view, while providing a particular theory and its implementation. In contrast
with purely theoretical approaches, the resulting architecture should be usable
in realistic settings, but also form the core of a framework containing all the
basic mechanisms, into which it should be easier to integrate additional
required functionality.
In this paper, we present a novel, purposely simple, and interpretable
hierarchical architecture which combines multiple different mechanisms into one
system: unsupervised learning of a model of the world, learning the influence
of one's own actions on the world, model-based reinforcement learning,
hierarchical planning and plan execution, and symbolic/sub-symbolic integration
in general. The learned model is stored in the form of hierarchical
representations with the following properties: 1) they are increasingly more
abstract, but can retain details when needed, and 2) they are easy to
manipulate in their local and symbolic-like form, thus also allowing one to
observe the learning process at each level of abstraction. On all levels of the
system, the representation of the data can be interpreted in both a symbolic
and a sub-symbolic manner. This enables the architecture to learn efficiently
using sub-symbolic methods and to employ symbolic inference.Comment: Revision: changed the pdftitl
Design and Analysis of a Reconfigurable Hierarchical Temporal Memory Architecture
Self-learning hardware systems, with high-degree of plasticity, are critical in performing spatio-temporal tasks in next-generation computing systems. To this end, hierarchical temporal memory (HTM) offers time-based online-learning algorithms that store and recall temporal and spatial patterns. In this work, a reconfigurable and scalable HTM architecture is designed with unique pooling realizations. Virtual synapse design is proposed to address the dynamic interconnections occurring in the learning process. The architecture is interweaved with parallel cells and columns that enable high processing speed for the cortical learning algorithm.
HTM has two core operations, spatial and temporal pooling. These operations are verified for two different datasets: MNIST and European number plate font. The spatial pooling operation is independently verified for classification with and without the presence of noise. The temporal pooling is verified for simple prediction. The spatial pooler architecture is ported onto an Altera cyclone II fabric and the entire architecture is synthesized for Xilinx Virtex IV. The results show that 91% classification accuracy is achieved with MNIST database and 90% accuracy for the European number plate font numbers with the presence of Gaussian and Salt & Pepper noise. For the prediction, first and second order predictions are observed for a 5-number long sequence generated from European number plate font and ~95% accuracy is obtained. Moreover, the proposed hardware architecture offers 3902X speedup over the software realization. These results indicate that the proposed architecture can serve as a core to build the HTM in hardware and eventually as a standalone self-learning hardware system
Neuro-memristive Circuits for Edge Computing: A review
The volume, veracity, variability, and velocity of data produced from the
ever-increasing network of sensors connected to Internet pose challenges for
power management, scalability, and sustainability of cloud computing
infrastructure. Increasing the data processing capability of edge computing
devices at lower power requirements can reduce several overheads for cloud
computing solutions. This paper provides the review of neuromorphic
CMOS-memristive architectures that can be integrated into edge computing
devices. We discuss why the neuromorphic architectures are useful for edge
devices and show the advantages, drawbacks and open problems in the field of
neuro-memristive circuits for edge computing
A Novel FPGA Implementation of Hierarchical Temporal Memory Spatial Pooler
There is currently a strong focus across the technological landscape to create machines capable of performing complex, objective based tasks in a manner similar to, or superior to a human. Many of the methods being explored in the machine intelligence space require large sets of labeled data to first train, and then classify inputs. Hierarchical Temporal Memory (HTM) is a biologically inspired machine intelligence framework which aims to classify and interpret streaming unlabeled data, without supervision, and be able to detect anomalies in such data.
In software HTM models, increasing the number of “columns” or processing elements to the levels required to make meaningful predictions in complex data can be prohibitive to analyzing in real time. There exists a need to improve the throughput of such systems. HTMs require large amounts of data available to be accessed randomly, and then processed independently. FPGAs provide a reconfigurable, and easily scalable platform ideal for these types of operations. One of the two main components of the HTM architecture is the “spatial pooler”. This thesis explores a novel hardware implementation of an HTM spatial pooler, with a boosting algorithm to increase homeostasis, and a novel classification algorithm to interpret input data in real time. This implementation shows a significant speedup in data processing, and provides a framework to scale the implementation based on the available hardware resources of the FPGA
Hierarchical Temporal Memory using Memristor Networks: A Survey
This paper presents a survey of the currently available hardware designs for
implementation of the human cortex inspired algorithm, Hierarchical Temporal
Memory (HTM). In this review, we focus on the state of the art advances of
memristive HTM implementation and related HTM applications. With the advent of
edge computing, HTM can be a potential algorithm to implement on-chip near
sensor data processing. The comparison of analog memristive circuit
implementations with the digital and mixed-signal solutions are provided. The
advantages of memristive HTM over digital implementations against performance
metrics such as processing speed, reduced on-chip area and power dissipation
are discussed. The limitations and open problems concerning the memristive HTM,
such as the design scalability, sneak currents, leakage, parasitic effects,
lack of the analog learning circuits implementations and unreliability of the
memristive devices integrated with CMOS circuits are also discussed
- …