Hardware-efficient on-line learning through pipelined truncated-error backpropagation in binary-state networks

Artificial neural networks (ANNs) trained using backpropagation are powerful learning architectures that have achieved state-of-the-art performance in various benchmarks. Significant effort has been devoted to developing custom silicon devices to accelerate inference in ANNs. Accelerating the training phase, however, has attracted relatively little attention. In this paper, we describe a hardware-efficient on-line learning technique for feedforward multi-layer ANNs that is based on pipelined backpropagation. Learning is performed in parallel with inference in the forward pass, removing the need for an explicit backward pass and requiring no extra weight lookup. By using binary state variables in the feedforward network and ternary errors in truncated-error backpropagation, the need for any multiplications in the forward and backward passes is removed, and memory requirements for the pipelining are drastically reduced. Further reduction in addition operations owing to the sparsity in the forward neural and backpropagating error signal paths contributes to highly efficient hardware implementation. For proof-of-concept validation, we demonstrate on-line learning of MNIST handwritten digit classification on a Spartan 6 FPGA interfacing with an external 1Gb DDR2 DRAM, that shows small degradation in test error performance compared to an equivalently sized binary ANN trained off-line using standard back-propagation and exact errors. Our results highlight an attractive synergy between pipelined backpropagation and binary-state networks in substantially reducing computation and memory requirements, making pipelined on-line learning practical in deep networks.Comment: Now also consider 0/1 binary activations. Memory access statistics reporte

Construction of generalized Rademacher functions in terms of ternary logic: solving the problem of visibility of using Galois fields for digital signal processing

Using generalized Rademacher functions, constructed as a sequence of elements of Galois fields , and intended to find the spectral representation of signals with n levels, the expediency of using the concept of "logical imaginary unit" is substantiated. These functions form a complete basis on the interval corresponding to 3^n -1 discrete time intervals and for n=1  passing into the classical Rademacher functions. The advantage of such spectra obtained using Galois Fields Fourier Transform is that the range of variation of the spectrum amplitudes remains the same as the range of variation of the original signal, which is modeled on discrete time functions taking values in the Galois field

Using a genetic algorithm for optimizing the functional decomposition of multiple-valued functions

The genetic algorithm which determines the good functional decomposition of multiple-valued logic functions is presented. The algorithm expands the range of searching for a best decomposition, providing the optimal column multiplicity. The possible solutions are evaluated using the gain of decomposition for multiple-valued function

The Process of Consciousness and Its Evolution Arising from Granulation and Bilogic

Our objective is to seek symmetry originating in the unconscious, and that is expressed here as reporting in consciousness. The association of psychological constructs that are consciousness and unconscious are to be both understood and interpreted within Ignacio Matte-Blanco's theory and his Bilogic framework. We carry out text mining, using extensively recorded dream reports. The texts themselves, and all the content, termed word corpus, are mapped into real-valued semantic, factor space, using Correspondence Analysis that can also be termed Geometric Data Analysis. Factors can be interpreted. For granulation, this new development is carried out: for texts and words, their factor projections are ternary encoded, to express commonality and exceptionalism. Then determined are clusters of terms and of words with semantic identity. Hence an innovative approach to granulation, and mostly with linear computational complexity. The study here establishes new implementation in analytical methods, and is also to have definite reproducibility in other, relevant and associated domains