Asynchronous spiking neurons, the natural key to exploit temporal sparsity

Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms

Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity

Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms.European Union's Horizon 2020 No 687299 NeuRAMEuropean Union's Horizon 2020 No 824164 HERMESMinisterio de Economía y Competitividad TEC2015-63884-C2-1-

SpArNet: Sparse Asynchronous Neural Network execution for energy efficient inference

Biological neurons are known to have sparse and asynchronous communications using spikes. Despite our incomplete understanding of processing strategies of the brain, its low energy consumption in fulfilling delicate tasks suggests the existence of energy efficient mechanisms. Inspired by these key factors, we introduce SpArNet, a bio-inspired quantization scheme to convert a pre-trained convolutional neural network to a spiking neural network, with the aim of minimizing the computational load for execution on neuromorphic processors. The proposed scheme has significant advantages over the reference CNN in a reduced number of synaptic operations, and can be used for frequent executions of inference tasks. The computational load of SpArNet is adjusted to the spatio-temporal dynamics of the the input data. We have tested the converted network on two applications (autonomous steering and hand gesture recognition), demonstrating a significant reduction on the number of required synaptic operations

Stimuli’s parameters.

<p>Stimuli are generated on a space defined in absolute values (ranging arbitrarily from −1 to 1) and time defined from <i>t</i> = 0 to <i>t</i> = <i>T</i> (in seconds). As such, stimulus parameters are defined in these units. To avoid border effects, the spatio-temporal domain is defined as a 3-dimensional torus (that is the cartesian product of the periodic real spaces ). By convention, a speed of 1 is defined as a motion of one spatial period in one temporal period.</p

Both flash-initiated and flash-terminated conditions can be explained by the diagonal motion-based prediction (dMBP) model.

<p>With the same format as <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005068#pcbi.1005068.g003" target="_blank">Fig 3-B</a>, we plot the temporal evolution of the probability distributions of the inferred position for both the flashed (in red) and moving (in green) dots, in the (A) flash-initiated and (B) flash-terminated conditions. As in <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005068#pcbi.1005068.g003" target="_blank">Fig 3-B</a>, each curve corresponds to the five frames (respectively numbered from <i>i</i> − 2 to <i>i</i> + 2) centered on the time of the model’s maximal response to the flash. Dashed vertical lines indicate at each frame the estimated positions from the maximum a posteriori of the probability distributions for either the flash (red) or the moving (green) dot, together with the veridical position of the flashed dot (black). As expected, one can observe that the distribution of inferred positions is approximately correct for the flashed stimulus in all conditions. In the flash-initiated FLE condition, the distribution for the moving dot is biased towards its direction and develops very rapidly. Notice however that these biases are smaller than observed with the standard FLE. In the flash-terminated conditions, the bias is observed in the last frames before the maximum of the flash and then competes with another estimate with no bias which dominates near the moment of the flash’s maximum. Note that the a posteriori probability distributions around the flash’s maximum are very broad and indicate a high spatial uncertainty. Altogether, the absence of bias in the flash-terminated condition is similar to that reported psychophysically with human observers [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005068#pcbi.1005068.ref028" target="_blank">28</a>].</p

The flash-lag effect (FLE) as a motion-induced predictive shift.

<p>To follow the example given by [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1005068#pcbi.1005068.ref006" target="_blank">6</a>], a football (soccer) player that would run along a continuous path (the green path, where the gradient of color denotes the flow of time) is perceived to be ahead (the red position) of its actual position at the unexpected moment a ball is shot (red star) even if these positions are physically aligned. A referee would then signal an “offside” position. Similarly, such a flash-lag effect (FLE) is observed systematically in psychophysical experiments by showing a moving and a flashed stimuli (here, a square). By varying their characteristics (speed, relative position), one can explore the fundamental principles of the FLE.</p