High-capacity exponential associative memories

A generalized associative memory model with potentially high capacity is presented. A memory of this kind with M stored vectors of length N, can be implemented with M nonlinear neurons, N ordinary thresholding neurons, and 2MN binary synapses. It is shown that special cases of this model include the Hopfield and high-order correlation memories. A special case of the model, based on a neuron which can implement the subthreshold region, is presented. The authors analyze the capacity of this exponentially associative memory and show that it scales exponentially with N. In any practical realization, however, the dynamic range of the exponentiators is constrained. They show that the capacity for networks with fixed dynamic range exponential circuits is proportional to the dynamic range

VLSI Implementation of a High-Capacity Neural Network Associative Memory

In this paper we describe the VLSI design and testing of a high capacity associative memory which we call the exponential correlation associative memory (ECAM). The prototype 3µ-CMOS programmable chip is capable of storing 32 memory patterns of 24 bits each. The high capacity of the ECAM is partly due to the use of special exponentiation neurons, which are implemented via sub-threshold MOS transistors in this design. The prototype chip is capable of performing one associative recall in 3 µS

The Kanerva memory is stable

The Kanerva memory is a simple yet important model of the cerebellar cortex. Its power has been demonstrated by its huge storage capacity as an associative memory. In the present work, the Kanerva memory is briefly introduced and it is shown to be asymptotically stable in both the parallel update and sequential update modes. Its asymptotic stability is proved by introducing a Lyapunov function and showing that the function follows a descent trajectory as the Kanerva memory evolves

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 µs. An application of the ECAM chip to vector quantization is also described

A static RAM chip with on-chip error correction

This paper describes a 2-kb CMOS static RAM with on-chip error-correction capability (ECCRAM chip). The chip employs the linear sum code (LSC) technique to perform error detection and correction. The ECCRAM chip has been fabricated in a double-metal scalable CMOS process with 3-µm feature size. Testing results of the actual chip shows a significant improvement in random error tolerance

High-capacity exponential associative memories

A generalized associative memory model with potentially high capacity is presented. A memory of this kind with M stored vectors of length N, can be implemented with M nonlinear neurons, N ordinary thresholding neurons, and 2MN binary synapses. It is shown that special cases of this model include the Hopfield and high-order correlation memories. A special case of the model, based on a neuron which can implement the subthreshold region, is presented. The authors analyze the capacity of this exponentially associative memory and show that it scales exponentially with N. In any practical realization, however, the dynamic range of the exponentiators is constrained. They show that the capacity for networks with fixed dynamic range exponential circuits is proportional to the dynamic range

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 µs. An application of the ECAM chip to vector quantization is also described