527 research outputs found
Binary Weighted Memristive Analog Deep Neural Network for Near-Sensor Edge Processing
The memristive crossbar aims to implement analog weighted neural network,
however, the realistic implementation of such crossbar arrays is not possible
due to limited switching states of memristive devices. In this work, we propose
the design of an analog deep neural network with binary weight update through
backpropagation algorithm using binary state memristive devices. We show that
such networks can be successfully used for image processing task and has the
advantage of lower power consumption and small on-chip area in comparison with
digital counterparts. The proposed network was benchmarked for MNIST
handwritten digits recognition achieving an accuracy of approximately 90%
NACU: A Non-Linear Arithmetic Unit for Neural Networks
Reconfigurable architectures targeting neural networks are an attractive option. They allow multiple neural networks of different types to be hosted on the same hardware, in parallel or sequence. Reconfigurability also grants the ability to morph into different micro-architectures to meet varying power-performance constraints. In this context, the need for a reconfigurable non-linear computational unit has not been widely researched. In this work, we present a formal and comprehensive method to select the optimal fixed-point representation to achieve the highest accuracy against the floating-point implementation benchmark. We also present a novel design of an optimised reconfigurable arithmetic unit for calculating non-linear functions. The unit can be dynamically configured to calculate the sigmoid, hyperbolic tangent, and exponential function using the same underlying hardware. We compare our work with the state-of-the-art and show that our unit can calculate all three functions without loss of accuracy
Design Space Exploration of Neural Network Activation Function Circuits
The widespread application of artificial neural networks has prompted
researchers to experiment with FPGA and customized ASIC designs to speed up
their computation. These implementation efforts have generally focused on
weight multiplication and signal summation operations, and less on activation
functions used in these applications. Yet, efficient hardware implementations
of nonlinear activation functions like Exponential Linear Units (ELU), Scaled
Exponential Linear Units (SELU), and Hyperbolic Tangent (tanh), are central to
designing effective neural network accelerators, since these functions require
lots of resources. In this paper, we explore efficient hardware implementations
of activation functions using purely combinational circuits, with a focus on
two widely used nonlinear activation functions, i.e., SELU and tanh. Our
experiments demonstrate that neural networks are generally insensitive to the
precision of the activation function. The results also prove that the proposed
combinational circuit-based approach is very efficient in terms of speed and
area, with negligible accuracy loss on the MNIST, CIFAR-10 and IMAGENET
benchmarks. Synopsys Design Compiler synthesis results show that circuit
designs for tanh and SELU can save between 3.13-7.69 and 4.45-8:45 area
compared to the LUT/memory-based implementations, and can operate at 5.14GHz
and 4.52GHz using the 28nm SVT library, respectively. The implementation is
available at: https://github.com/ThomasMrY/ActivationFunctionDemo.Comment: 5 pages, 5 figures, 16 conferenc
An Investigation into Neuromorphic ICs using Memristor-CMOS Hybrid Circuits
The memristance of a memristor depends on the amount of charge flowing
through it and when current stops flowing through it, it remembers the state.
Thus, memristors are extremely suited for implementation of memory units.
Memristors find great application in neuromorphic circuits as it is possible to
couple memory and processing, compared to traditional Von-Neumann digital
architectures where memory and processing are separate. Neural networks have a
layered structure where information passes from one layer to another and each
of these layers have the possibility of a high degree of parallelism.
CMOS-Memristor based neural network accelerators provide a method of speeding
up neural networks by making use of this parallelism and analog computation. In
this project we have conducted an initial investigation into the current state
of the art implementation of memristor based programming circuits. Various
memristor programming circuits and basic neuromorphic circuits have been
simulated. The next phase of our project revolved around designing basic
building blocks which can be used to design neural networks. A memristor bridge
based synaptic weighting block, a operational transconductor based summing
block were initially designed. We then designed activation function blocks
which are used to introduce controlled non-linearity. Blocks for a basic
rectified linear unit and a novel implementation for tan-hyperbolic function
have been proposed. An artificial neural network has been designed using these
blocks to validate and test their performance. We have also used these
fundamental blocks to design basic layers of Convolutional Neural Networks.
Convolutional Neural Networks are heavily used in image processing
applications. The core convolutional block has been designed and it has been
used as an image processing kernel to test its performance.Comment: Bachelor's thesi
Mixed-Signal VLSI Implementation of CVNS Artificial Neural Networks
In this work, mixed-signal implementation of Continuous Valued Number System (CVNS) neural network is proposed. The proposed network resolves the limited signal processing precision issue present in mixed-signal neural networks. This is realized by the CVNS addition, the CVNS multiplication and the CVNS sigmoid function evaluation algorithms proposed in this dissertation. The proposed algorithms provide accurate results in low-resolution environment. In addition, an area-efficient low sensitivity CVNS Madaline is proposed. The proposed Madaline is more robust to input and weight errors when compared to the previously developed structures. Moreover, its area consumption is lower. Furthermore, a new approximation scheme for hyperbolic tangent activation function is proposed. Using the proposed approximation scheme results in efficient implementation of digital ASIC neural networks in terms of area, delay and power consumption
- …