1,914 research outputs found
Design Space Exploration of Neural Network Activation Function Circuits
The widespread application of artificial neural networks has prompted
researchers to experiment with FPGA and customized ASIC designs to speed up
their computation. These implementation efforts have generally focused on
weight multiplication and signal summation operations, and less on activation
functions used in these applications. Yet, efficient hardware implementations
of nonlinear activation functions like Exponential Linear Units (ELU), Scaled
Exponential Linear Units (SELU), and Hyperbolic Tangent (tanh), are central to
designing effective neural network accelerators, since these functions require
lots of resources. In this paper, we explore efficient hardware implementations
of activation functions using purely combinational circuits, with a focus on
two widely used nonlinear activation functions, i.e., SELU and tanh. Our
experiments demonstrate that neural networks are generally insensitive to the
precision of the activation function. The results also prove that the proposed
combinational circuit-based approach is very efficient in terms of speed and
area, with negligible accuracy loss on the MNIST, CIFAR-10 and IMAGENET
benchmarks. Synopsys Design Compiler synthesis results show that circuit
designs for tanh and SELU can save between 3.13-7.69 and 4.45-8:45 area
compared to the LUT/memory-based implementations, and can operate at 5.14GHz
and 4.52GHz using the 28nm SVT library, respectively. The implementation is
available at: https://github.com/ThomasMrY/ActivationFunctionDemo.Comment: 5 pages, 5 figures, 16 conferenc
An efficient hardware architecture for a neural network activation function generator
This paper proposes an efficient hardware architecture for a function generator suitable for an artificial neural network (ANN). A spline-based approximation function is designed that provides a good trade-off between accuracy and silicon area, whilst also being inherently scalable and adaptable for numerous activation functions. This has been achieved by using a minimax polynomial and through optimal placement of the approximating polynomials based on the results of a genetic algorithm. The approximation error of the proposed method compares favourably to all related research in this field. Efficient hardware multiplication circuitry is used in the implementation, which reduces the area overhead and increases the throughput
VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing
The hardware implementation of deep neural networks (DNNs) has recently
received tremendous attention: many applications in fact require high-speed
operations that suit a hardware implementation. However, numerous elements and
complex interconnections are usually required, leading to a large area
occupation and copious power consumption. Stochastic computing has shown
promising results for low-power area-efficient hardware implementations, even
though existing stochastic algorithms require long streams that cause long
latencies. In this paper, we propose an integer form of stochastic computation
and introduce some elementary circuits. We then propose an efficient
implementation of a DNN based on integral stochastic computing. The proposed
architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62%
average reductions in area and latency compared to the best reported
architecture in literature. We also synthesize the circuits in a 65 nm CMOS
technology and we show that the proposed integral stochastic architecture
results in up to 21% reduction in energy consumption compared to the binary
radix implementation at the same misclassification rate. Due to fault-tolerant
nature of stochastic architectures, we also consider a quasi-synchronous
implementation which yields 33% reduction in energy consumption w.r.t. the
binary radix implementation without any compromise on performance.Comment: 11 pages, 12 figure
Approximate FPGA-based LSTMs under Computation Time Constraints
Recurrent Neural Networks and in particular Long Short-Term Memory (LSTM)
networks have demonstrated state-of-the-art accuracy in several emerging
Artificial Intelligence tasks. However, the models are becoming increasingly
demanding in terms of computational and memory load. Emerging latency-sensitive
applications including mobile robots and autonomous vehicles often operate
under stringent computation time constraints. In this paper, we address the
challenge of deploying computationally demanding LSTMs at a constrained time
budget by introducing an approximate computing scheme that combines iterative
low-rank compression and pruning, along with a novel FPGA-based LSTM
architecture. Combined in an end-to-end framework, the approximation method's
parameters are optimised and the architecture is configured to address the
problem of high-performance LSTM execution in time-constrained applications.
Quantitative evaluation on a real-life image captioning application indicates
that the proposed methods required up to 6.5x less time to achieve the same
application-level accuracy compared to a baseline method, while achieving an
average of 25x higher accuracy under the same computation time constraints.Comment: Accepted at the 14th International Symposium in Applied
Reconfigurable Computing (ARC) 201
- âŠ