4,625 research outputs found
Deep Neural Networks - A Brief History
Introduction to deep neural networks and their history.Comment: 14 pages, 14 figure
Unsupervised Learning with Self-Organizing Spiking Neural Networks
We present a system comprising a hybridization of self-organized map (SOM)
properties with spiking neural networks (SNNs) that retain many of the features
of SOMs. Networks are trained in an unsupervised manner to learn a
self-organized lattice of filters via excitatory-inhibitory interactions among
populations of neurons. We develop and test various inhibition strategies, such
as growing with inter-neuron distance and two distinct levels of inhibition.
The quality of the unsupervised learning algorithm is evaluated using examples
with known labels. Several biologically-inspired classification tools are
proposed and compared, including population-level confidence rating, and
n-grams using spike motif algorithm. Using the optimal choice of parameters,
our approach produces improvements over state-of-art spiking neural networks
Neural activity classification with machine learning models trained on interspike interval series data
The flow of information through the brain is reflected by the activity
patterns of neural cells. Indeed, these firing patterns are widely used as
input data to predictive models that relate stimuli and animal behavior to the
activity of a population of neurons. However, relatively little attention was
paid to single neuron spike trains as predictors of cell or network properties
in the brain. In this work, we introduce an approach to neuronal spike train
data mining which enables effective classification and clustering of neuron
types and network activity states based on single-cell spiking patterns. This
approach is centered around applying state-of-the-art time series
classification/clustering methods to sequences of interspike intervals recorded
from single neurons. We demonstrate good performance of these methods in tasks
involving classification of neuron type (e.g. excitatory vs. inhibitory cells)
and/or neural circuit activity state (e.g. awake vs. REM sleep vs. nonREM sleep
states) on an open-access cortical spiking activity dataset
Towards Accurate and High-Speed Spiking Neuromorphic Systems with Data Quantization-Aware Deep Networks
Deep Neural Networks (DNNs) have gained immense success in cognitive
applications and greatly pushed today's artificial intelligence forward. The
biggest challenge in executing DNNs is their extremely data-extensive
computations. The computing efficiency in speed and energy is constrained when
traditional computing platforms are employed in such computational hungry
executions. Spiking neuromorphic computing (SNC) has been widely investigated
in deep networks implementation own to their high efficiency in computation and
communication. However, weights and signals of DNNs are required to be
quantized when deploying the DNNs on the SNC, which results in unacceptable
accuracy loss. %However, the system accuracy is limited by quantizing data
directly in deep networks deployment. Previous works mainly focus on weights
discretize while inter-layer signals are mainly neglected. In this work, we
propose to represent DNNs with fixed integer inter-layer signals and
fixed-point weights while holding good accuracy. We implement the proposed DNNs
on the memristor-based SNC system as a deployment example. With 4-bit data
representation, our results show that the accuracy loss can be controlled
within 0.02% (2.3%) on MNIST (CIFAR-10). Compared with the 8-bit dynamic
fixed-point DNNs, our system can achieve more than 9.8x speedup, 89.1% energy
saving, and 30% area saving.Comment: 6 pages, 4 figure
Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection
Recent advances in Voice Activity Detection (VAD) are driven by artificial
and Recurrent Neural Networks (RNNs), however, using a VAD system in
battery-operated devices requires further power efficiency. This can be
achieved by neuromorphic hardware, which enables Spiking Neural Networks (SNNs)
to perform inference at very low energy consumption. Spiking networks are
characterized by their ability to process information efficiently, in a sparse
cascade of binary events in time called spikes. However, a big performance gap
separates artificial from spiking networks, mostly due to a lack of powerful
SNN training algorithms. To overcome this problem we exploit an SNN model that
can be recast into an RNN-like model and trained with known deep learning
techniques. We describe an SNN training procedure that achieves low spiking
activity and pruning algorithms to remove 85% of the network connections with
no performance loss. The model achieves state-of-the-art performance with a
fraction of power consumption comparing to other methods.Comment: 5 pages, 2 figures, 2 table
Random Sketching, Clustering, and Short-Term Memory in Spiking Neural Networks
We study input compression in a biologically inspired model of neural computation. We demonstrate that a network consisting of a random projection step (implemented via random synaptic connectivity) followed by a sparsification step (implemented via winner-take-all competition) can reduce well-separated high-dimensional input vectors to well-separated low-dimensional vectors. By augmenting our network with a third module, we can efficiently map each input (along with any small perturbations of the input) to a unique representative neuron, solving a neural clustering problem.
Both the size of our network and its processing time, i.e., the time it takes the network to compute the compressed output given a presented input, are independent of the (potentially large) dimension of the input patterns and depend only on the number of distinct inputs that the network must encode and the pairwise relative Hamming distance between these inputs. The first two steps of our construction mirror known biological networks, for example, in the fruit fly olfactory system [Caron et al., 2013; Lin et al., 2014; Dasgupta et al., 2017]. Our analysis helps provide a theoretical understanding of these networks and lay a foundation for how random compression and input memorization may be implemented in biological neural networks.
Technically, a contribution in our network design is the implementation of a short-term memory. Our network can be given a desired memory time t_m as an input parameter and satisfies the following with high probability: any pattern presented several times within a time window of t_m rounds will be mapped to a single representative output neuron. However, a pattern not presented for c?t_m rounds for some constant c>1 will be "forgotten", and its representative output neuron will be released, to accommodate newly introduced patterns
- …