Overcoming Noise and Variations In Low-Precision Neural Networks

This work explores the impact of various design and training choices on the resilience of a neural network when subjected to noise and/or device variations. Simulations were performed under the expectation that the neural network would be implemented on analog hardware; this context asserts that there will be random noise within the circuit as well as variations in device characteristics between each fabricated device. The results show how noise can be added during the training process to reduce the impact of post-training noise. Architectural choices for the neural network also directly impact the performance variation between devices. The simulated neural networks were more robust to noise with a minimal architecture with fewer layers; if more neurons are needed for better fitting, networks with more neurons in shallow layers and fewer in deeper layers closer to the output tend to perform better. The paper also demonstrates that activation functions with lower slopes do a better job of suppressing noise in the neural network. It also shown that the accuracy can be made more consistent by introducing sparsity into the neural network. To that end, an evaluation is included of different methods for generating sparse architectures for smaller neural networks. A new method is proposed that consistently outperforms the most common methods used in larger, deeper networks.Ph.D

Analog Encoded Neural Network for Power Management in MPSoC

International audienceEncoded neural networks combine the principles of associative memories and errorcorrecting decoders so that they are good candidates to solve problems where decisions have to be made based on partial input information. This paper introduces an analog implementation of this new type of network to manage the power distribution in a Multiprocessor System-on-Chip(MPSoC). The proposed circuit has been designed for the 1V supply ST CMOS 65nm process, with a low complexity and low power consumption (less than 1% of the MPSoC power). Compared to a digital counterpart based on game theory, this analog solution consumes 6800 times less energy and reacts 4500 times faster. Thus, this analog circuit allows to fully exploiting DVFS circuits switching capabilities to continuously adapt the power distribution of an MPSoC. From a given energy budget, GT saves 38% while the analog ENN saves 60%