SkyNet: Memristor-based 3D IC for Artificial Neural Networks

Hardware implementations of artificial neural networks (ANNs) have become feasible due to the advent of persistent 2-terminal devices such as memristor, phase change memory, MTJs, etc. Hybrid memristor crossbar/CMOS systems have been studied extensively and demonstrated experimentally. In these circuits, memristors located at each cross point in a crossbar are, however, stacked on top of CMOS circuits using back end of line processing (BOEL), limiting scaling. Each neuron’s functionality is spread across layers of CMOS and memristor crossbar and thus cannot support the required connectivity to implement large-scale multi-layered ANNs. This work proposes a new fine-grained 3D integrated circuit technology for ANNs that is one of the first IC technologies for this purpose. Synaptic weights implemented with devices are incorporated in a uniform vertical nanowire template co-locating the memory and computation requirements of ANNs within each neuron. Novel 3D routing features are used for interconnections in all three dimensions between the devices enabling high connectivity without the need for special pins or metal vias. To demonstrate the proof of concept of this fabric, classification of binary images using a perceptron-based feed forward neural network is shown. Bottom-up evaluations for the proposed fabric considering 3D implementation of fabric components reveal up to 19x density, 1.2x power benefits when compared to 16nm hybrid memristor/CMOS technology

FPGA implementation of artificial neural networks

As the title suggests our project deals with a hardware implementation of artificial neural networks, specifically a FPGA implementation. During the course of this project we learnt about ANNs and the uses of such soft computing approaches, FPGAs, VHDL and use of various tools like Xilinx ISE Project Navigator and ModelSim. As numerous hardware implementations of ANNs already exist our aim was to come up with an approach that would facilitate topology evolution of the ANN as well

Training Artificial Neural Networks by PSO to Perform Digital Circuits Using Xilinx FPGA

One of the major constraints on hardware implementations of Artificial Neural Networks (ANNs) is the amount of circuitry required to perform the multiplication process of each input by its corresponding weight and there subsequent addition. Field Programmable Gate Array (FPGA) is a suitable hardware IC for Neural Network (NN) implementation as it preserves the parallel architecture of the neurons in a layer and offers flexibility in reconfiguration and cost issues. In this paper the adaption of the ANN weights is proposed using Particle Swarm Optimization (PSO) as a mechanism to improve the performance of ANN and also for the reduction in the ANN hardware. For this purpose we modified the MATLAB PSO toolbox to be suitable for the taken application. In the proposed design training is done off chip then the fully trained design is download into the chip, in this way less circuitry is required. This paper executes four bit Arithmetic Logic Unit (ALU) implemented using Xilinx schematic design entry tools as an example for the implementation of digital circuits using ANN trained by PSO algorithm

Innovation and application of ANN in Europe demonstrated by Kohonen maps

One of the most important contributions to neural networks comes from Kohonen, Helsinki/Espoo, Finland, who had the idea of self-organizating maps in 1981. He verified his idea by an algorithm of which many applications make use of. The impetus for this idea came from biology, a field where the Europeans have always been very active at several research laboratories. The challenge was to model the self-organization found in the brain. Today one goal is the development of more sophisticated neurons which model the biological neurons more exactly. They should come to a better performance of neural nets with only a few complex neurons instead of many simple ones. A lot of application concepts arise from this idea: Kohonen himself applied it to speech recognition, but the project did not overcome much more than the recognition of the numerals one to ten at that time. A more promising application for self-organizing maps is process control and process monitoring. Several proposals were made which concern parameter classification of semiconductor technologies, design of integrated circuits, and control of chemical processes. Self-organizing maps were applied to robotics. The neural concept was introduced into electric power systems. At Dortmund we are working on a system which has to monitor the quality and the reliability of gears and electrical motors in equipment installed in coal mines. The results are promising and the probability to apply the system in the field is very high. A special feature of the system is that linguistic rules which are embedded in a fuzzy controller analyze the data of the self-organizing map in regard to life expectation of the gears. It seems that the fuzzy technique will introduce the technology of neural networks in a tandem mode. These technologies together with the genetic algorithms start to form the attractive field of computational intelligence

Capacity, Fidelity, and Noise Tolerance of Associative Spatial-Temporal Memories Based on Memristive Neuromorphic Network

We have calculated the key characteristics of associative (content-addressable) spatial-temporal memories based on neuromorphic networks with restricted connectivity - "CrossNets". Such networks may be naturally implemented in nanoelectronic hardware using hybrid CMOS/memristor circuits, which may feature extremely high energy efficiency, approaching that of biological cortical circuits, at much higher operation speed. Our numerical simulations, in some cases confirmed by analytical calculations, have shown that the characteristics depend substantially on the method of information recording into the memory. Of the four methods we have explored, two look especially promising - one based on the quadratic programming, and the other one being a specific discrete version of the gradient descent. The latter method provides a slightly lower memory capacity (at the same fidelity) then the former one, but it allows local recording, which may be more readily implemented in nanoelectronic hardware. Most importantly, at the synchronous retrieval, both methods provide a capacity higher than that of the well-known Ternary Content-Addressable Memories with the same number of nonvolatile memory cells (e.g., memristors), though the input noise immunity of the CrossNet memories is somewhat lower

Field programmable gate array based sigmoid function implementation using differential lookup table and second order nonlinear function

Artificial neural network (ANN) is an established artificial intelligence technique that is widely used for solving numerous problems such as classification and clustering in various fields. However, the major problem with ANN is a factor of time. ANN takes a longer time to execute a huge number of neurons. In order to overcome this, ANN is implemented into hardware namely field-programmable-gate-array (FPGA). However, implementing the ANN into a field-programmable gate array (FPGA) has led to a new problem related to the sigmoid function implementation. Often used as the activation function for ANN, a sigmoid function cannot be directly implemented in FPGA. Owing to its accuracy, the lookup table (LUT) has always been used to implement the sigmoid function in FPGA. In this case, obtaining the high accuracy of LUT is expensive particularly in terms of its memory requirements in FPGA. Second-order nonlinear function (SONF) is an appealing replacement for LUT due to its small memory requirement. Although there is a trade-off between accuracy and memory size. Taking the advantage of the aforementioned approaches, this thesis proposed a combination of SONF and a modified LUT namely differential lookup table (dLUT). The deviation values between SONF and sigmoid function are used to create the dLUT. SONF is used as the first step to approximate the sigmoid function. Then it is followed by adding or deducting with the value that has been stored in the dLUT as a second step as demonstrated via simulation. This combination has successfully reduced the deviation value. The reduction value is significant as compared to previous implementations such as SONF, and LUT itself. Further simulation has been carried out to evaluate the accuracy of the ANN in detecting the object in an indoor environment by using the proposed method as a sigmoid function. The result has proven that the proposed method has produced the output almost as accurately as software implementation in detecting the target in indoor positioning problems. Therefore, the proposed method can be applied in any field that demands higher processing and high accuracy in sigmoid function outpu

Intrinsically Evolvable Artificial Neural Networks

Dedicated hardware implementations of neural networks promise to provide faster, lower power operation when compared to software implementations executing on processors. Unfortunately, most custom hardware implementations do not support intrinsic training of these networks on-chip. The training is typically done using offline software simulations and the obtained network is synthesized and targeted to the hardware offline. The FPGA design presented here facilitates on-chip intrinsic training of artificial neural networks. Block-based neural networks (BbNN), the type of artificial neural networks implemented here, are grid-based networks neuron blocks. These networks are trained using genetic algorithms to simultaneously optimize the network structure and the internal synaptic parameters. The design supports online structure and parameter updates, and is an intrinsically evolvable BbNN platform supporting functional-level hardware evolution. Functional-level evolvable hardware (EHW) uses evolutionary algorithms to evolve interconnections and internal parameters of functional modules in reconfigurable computing systems such as FPGAs. Functional modules can be any hardware modules such as multipliers, adders, and trigonometric functions. In the implementation presented, the functional module is a neuron block. The designed platform is suitable for applications in dynamic environments, and can be adapted and retrained online. The online training capability has been demonstrated using a case study. A performance characterization model for RC implementations of BbNNs has also been presented