3,677 research outputs found
Asynchronous spiking neurons, the natural key to exploit temporal sparsity
Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms
A Neuron as a Signal Processing Device
A neuron is a basic physiological and computational unit of the brain. While
much is known about the physiological properties of a neuron, its computational
role is poorly understood. Here we propose to view a neuron as a signal
processing device that represents the incoming streaming data matrix as a
sparse vector of synaptic weights scaled by an outgoing sparse activity vector.
Formally, a neuron minimizes a cost function comprising a cumulative squared
representation error and regularization terms. We derive an online algorithm
that minimizes such cost function by alternating between the minimization with
respect to activity and with respect to synaptic weights. The steps of this
algorithm reproduce well-known physiological properties of a neuron, such as
weighted summation and leaky integration of synaptic inputs, as well as an
Oja-like, but parameter-free, synaptic learning rule. Our theoretical framework
makes several predictions, some of which can be verified by the existing data,
others require further experiments. Such framework should allow modeling the
function of neuronal circuits without necessarily measuring all the microscopic
biophysical parameters, as well as facilitate the design of neuromorphic
electronics.Comment: 2013 Asilomar Conference on Signals, Systems and Computers, see
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=681029
- …