VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency compared to the best reported architecture in literature. We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation which yields 33% reduction in energy consumption w.r.t. the binary radix implementation without any compromise on performance.Comment: 11 pages, 12 figure

Projeto e treinamento de redes neurais em hardware FPGA usando computação estocástica

Trabalho de Conclusão de Curso (graduação)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2018.A utilização de redes neurais na solução de problemas em aplicações em tempo real requer o uso extensivo de circuitos paralelos e um bom equilíbrio entre alto desempenho e eficiência energética. Estudos anteriores demonstram que dispositivos FPGA satisfazem estes critérios, porém a capacidade lógica limitada dos mesmos impede a implementação de grandes redes que se beneficiem dos conceitos de Deep Learning. A Computação Estocástica permite que operações como adição e multiplicação sejam realizadas por portas lógicas individuais, simplificando extremamente o circuito neural. Este trabalho propõe a implementação de redes neurais baseadas em operações puramente estocásticas, viabilizando grandes estruturas e mantendo a paralelização completa. Ademais, apresentamos técnicas estocásticas que possibilitam o treinamento em hardware das redes implementadas de forma eficiente. Operações booleanas simples, aproximações de funções 2D e problemas de classificação são usados para verificar a eficácia da solução proposta.Solving real world problems with neural networks in real time applications requires extensive use of parallel circuitry and a good balance between high performance and energy efficiency. FPGA devices have beeen shown to meet the criteria, but their limited amount of logic resources prohibits the implementation of large networks that take advantage of deep learning techniques. Stochastic Computing allows operations like addition and multiplication to be performed by single logic gates, extremely simplifying neural circuitry. This work proposes the implementation of neural networks based on purely stochastic operations, supporting large structures while maintaining full parallelization. Furthermore, we also present stochastic techniques to enable high speed online training of these networks. Simple boolean operations, 2D function approximations and classification problems are used to verify the efficacy of the proposed solution