829 research outputs found
Asynchronous spiking neurons, the natural key to exploit temporal sparsity
Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms
Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection
Recent advances in Voice Activity Detection (VAD) are driven by artificial
and Recurrent Neural Networks (RNNs), however, using a VAD system in
battery-operated devices requires further power efficiency. This can be
achieved by neuromorphic hardware, which enables Spiking Neural Networks (SNNs)
to perform inference at very low energy consumption. Spiking networks are
characterized by their ability to process information efficiently, in a sparse
cascade of binary events in time called spikes. However, a big performance gap
separates artificial from spiking networks, mostly due to a lack of powerful
SNN training algorithms. To overcome this problem we exploit an SNN model that
can be recast into an RNN-like model and trained with known deep learning
techniques. We describe an SNN training procedure that achieves low spiking
activity and pruning algorithms to remove 85% of the network connections with
no performance loss. The model achieves state-of-the-art performance with a
fraction of power consumption comparing to other methods.Comment: 5 pages, 2 figures, 2 table
Unsupervised Heart-rate Estimation in Wearables With Liquid States and A Probabilistic Readout
Heart-rate estimation is a fundamental feature of modern wearable devices. In
this paper we propose a machine intelligent approach for heart-rate estimation
from electrocardiogram (ECG) data collected using wearable devices. The novelty
of our approach lies in (1) encoding spatio-temporal properties of ECG signals
directly into spike train and using this to excite recurrently connected
spiking neurons in a Liquid State Machine computation model; (2) a novel
learning algorithm; and (3) an intelligently designed unsupervised readout
based on Fuzzy c-Means clustering of spike responses from a subset of neurons
(Liquid states), selected using particle swarm optimization. Our approach
differs from existing works by learning directly from ECG signals (allowing
personalization), without requiring costly data annotations. Additionally, our
approach can be easily implemented on state-of-the-art spiking-based
neuromorphic systems, offering high accuracy, yet significantly low energy
footprint, leading to an extended battery life of wearable devices. We
validated our approach with CARLsim, a GPU accelerated spiking neural network
simulator modeling Izhikevich spiking neurons with Spike Timing Dependent
Plasticity (STDP) and homeostatic scaling. A range of subjects are considered
from in-house clinical trials and public ECG databases. Results show high
accuracy and low energy footprint in heart-rate estimation across subjects with
and without cardiac irregularities, signifying the strong potential of this
approach to be integrated in future wearable devices.Comment: 51 pages, 12 figures, 6 tables, 95 references. Under submission at
Elsevier Neural Network
Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays.
Resistive RAM crossbar arrays offer an attractive solution to minimize off-chip data transfer and parallelize on-chip computations for neural networks. Here, we report a hardware/software co-design approach based on low energy subquantum conductive bridging RAM (CBRAM®) devices and a network pruning technique to reduce network level energy consumption. First, we demonstrate low energy subquantum CBRAM devices exhibiting gradual switching characteristics important for implementing weight updates in hardware during unsupervised learning. Then we develop a network pruning algorithm that can be employed during training, different from previous network pruning approaches applied for inference only. Using a 512 kbit subquantum CBRAM array, we experimentally demonstrate high recognition accuracy on the MNIST dataset for digital implementation of unsupervised learning. Our hardware/software co-design approach can pave the way towards resistive memory based neuro-inspired systems that can autonomously learn and process information in power-limited settings
- …