1 research outputs found
Neural Network Training with Approximate Logarithmic Computations
The high computational complexity associated with training deep neural
networks limits online and real-time training on edge devices. This paper
proposed an end-to-end training and inference scheme that eliminates
multiplications by approximate operations in the log-domain which has the
potential to significantly reduce implementation complexity. We implement the
entire training procedure in the log-domain, with fixed-point data
representations. This training procedure is inspired by hardware-friendly
approximations of log-domain addition which are based on look-up tables and
bit-shifts. We show that our 16-bit log-based training can achieve
classification accuracy within approximately 1% of the equivalent
floating-point baselines for a number of commonly used datasets