1,982 research outputs found
Morphological Network: How Far Can We Go with Morphological Neurons?
In recent years, the idea of using morphological operations as networks has
received much attention. Mathematical morphology provides very efficient and
useful image processing and image analysis tools based on basic operators like
dilation and erosion, defined in terms of kernels. Many other morphological
operations are built up using the dilation and erosion operations. Although the
learning of structuring elements such as dilation or erosion using the
backpropagation algorithm is not new, the order and the way these morphological
operations are used is not standard. In this paper, we have theoretically
analyzed the use of morphological operations for processing 1D feature vectors
and shown that this gets extended to the 2D case in a simple manner. Our
theoretical results show that a morphological block represents a sum of hinge
functions. Hinge functions are used in many places for classification and
regression tasks (Breiman (1993)). We have also proved a universal
approximation theorem -- a stack of two morphological blocks can approximate
any continuous function over arbitrary compact sets. To experimentally validate
the efficacy of this network in real-life applications, we have evaluated its
performance on satellite image classification datasets since morphological
operations are very sensitive to geometrical shapes and structures. We have
also shown results on a few tasks like segmentation of blood vessels from
fundus images, segmentation of lungs from chest x-ray and image dehazing. The
results are encouraging and further establishes the potential of morphological
networks.Comment: 35 pages, 19 figures, 7 table
An Online Unsupervised Structural Plasticity Algorithm for Spiking Neural Networks
In this article, we propose a novel Winner-Take-All (WTA) architecture
employing neurons with nonlinear dendrites and an online unsupervised
structural plasticity rule for training it. Further, to aid hardware
implementations, our network employs only binary synapses. The proposed
learning rule is inspired by spike time dependent plasticity (STDP) but differs
for each dendrite based on its activation level. It trains the WTA network
through formation and elimination of connections between inputs and synapses.
To demonstrate the performance of the proposed network and learning rule, we
employ it to solve two, four and six class classification of random Poisson
spike time inputs. The results indicate that by proper tuning of the inhibitory
time constant of the WTA, a trade-off between specificity and sensitivity of
the network can be achieved. We use the inhibitory time constant to set the
number of subpatterns per pattern we want to detect. We show that while the
percentage of successful trials are 92%, 88% and 82% for two, four and six
class classification when no pattern subdivisions are made, it increases to
100% when each pattern is subdivided into 5 or 10 subpatterns. However, the
former scenario of no pattern subdivision is more jitter resilient than the
later ones.Comment: 11 pages, 10 figures, journa
Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites
This paper presents a spike-based model which employs neurons with
functionally distinct dendritic compartments for classifying high dimensional
binary patterns. The synaptic inputs arriving on each dendritic subunit are
nonlinearly processed before being linearly integrated at the soma, giving the
neuron a capacity to perform a large number of input-output mappings. The model
utilizes sparse synaptic connectivity; where each synapse takes a binary value.
The optimal connection pattern of a neuron is learned by using a simple
hardware-friendly, margin enhancing learning algorithm inspired by the
mechanism of structural plasticity in biological neurons. The learning
algorithm groups correlated synaptic inputs on the same dendritic branch. Since
the learning results in modified connection patterns, it can be incorporated
into current event-based neuromorphic systems with little overhead. This work
also presents a branch-specific spike-based version of this structural
plasticity rule. The proposed model is evaluated on benchmark binary
classification problems and its performance is compared against that achieved
using Support Vector Machine (SVM) and Extreme Learning Machine (ELM)
techniques. Our proposed method attains comparable performance while utilizing
10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio
Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations
In this paper, we describe a new neuro-inspired, hardware-friendly readout
stage for the liquid state machine (LSM), a popular model for reservoir
computing. Compared to the parallel perceptron architecture trained by the
p-delta algorithm, which is the state of the art in terms of performance of
readout stages, our readout architecture and learning algorithm can attain
better performance with significantly less synaptic resources making it
attractive for VLSI implementation. Inspired by the nonlinear properties of
dendrites in biological neurons, our readout stage incorporates neurons having
multiple dendrites with a lumped nonlinearity. The number of synaptic
connections on each branch is significantly lower than the total number of
connections from the liquid neurons and the learning algorithm tries to find
the best 'combination' of input connections on each branch to reduce the error.
Hence, the learning involves network rewiring (NRW) of the readout network
similar to structural plasticity observed in its biological counterparts. We
show that compared to a single perceptron using analog weights, this
architecture for the readout can attain, even by using the same number of
binary valued synapses, up to 3.3 times less error for a two-class spike train
classification problem and 2.4 times less error for an input rate approximation
task. Even with 60 times larger synapses, a group of 60 parallel perceptrons
cannot attain the performance of the proposed dendritically enhanced readout.
An additional advantage of this method for hardware implementations is that the
'choice' of connectivity can be easily implemented exploiting address event
representation (AER) protocols commonly used in current neuromorphic systems
where the connection matrix is stored in memory. Also, due to the use of binary
synapses, our proposed method is more robust against statistical variations.Comment: 14 pages, 19 figures, Journa
- …