5 research outputs found
Blind nonnegative source separation using biological neural networks
Blind source separation, i.e. extraction of independent sources from a
mixture, is an important problem for both artificial and natural signal
processing. Here, we address a special case of this problem when sources (but
not the mixing matrix) are known to be nonnegative, for example, due to the
physical nature of the sources. We search for the solution to this problem that
can be implemented using biologically plausible neural networks. Specifically,
we consider the online setting where the dataset is streamed to a neural
network. The novelty of our approach is that we formulate blind nonnegative
source separation as a similarity matching problem and derive neural networks
from the similarity matching objective. Importantly, synaptic weights in our
networks are updated according to biologically plausible local learning rules.Comment: Accepted for publication in Neural Computatio
A Spiking Neural Network with Local Learning Rules Derived From Nonnegative Similarity Matching
The design and analysis of spiking neural network algorithms will be
accelerated by the advent of new theoretical approaches. In an attempt at such
approach, we provide a principled derivation of a spiking algorithm for
unsupervised learning, starting from the nonnegative similarity matching cost
function. The resulting network consists of integrate-and-fire units and
exhibits local learning rules, making it biologically plausible and also
suitable for neuromorphic hardware. We show in simulations that the algorithm
can perform sparse feature extraction and manifold learning, two tasks which
can be formulated as nonnegative similarity matching problems.Comment: ICASSP 201
On the achievability of blind source separation for high-dimensional nonlinear source mixtures
For many years, a combination of principal component analysis (PCA) and
independent component analysis (ICA) has been used for blind source separation
(BSS). However, it remains unclear why these linear methods work well with
real-world data that involve nonlinear source mixtures. This work theoretically
validates that a cascade of linear PCA and ICA can solve a nonlinear BSS
problem accurately---when the sensory inputs are generated from hidden sources
via the nonlinear mapping with sufficient dimensionality. Our proposed theorem,
termed the asymptotic linearization theorem, theoretically guarantees that
applying linear PCA to the inputs can reliably extract a subspace spanned by
the linear projections from every hidden source as the major components---and
thus projecting the inputs onto their major eigenspace can effectively recover
a linear transformation of the hidden sources. Then, subsequent application of
linear ICA can separate all the true independent hidden sources accurately.
Zero-element-wise-error nonlinear BSS is asymptotically attained when the
source dimensionality is large and the input dimensionality is larger than the
source dimensionality. Our proposed theorem is validated analytically and
numerically. Moreover, the same computation can be performed by using
Hebbian-like plasticity rules, implying the biological plausibility of this
nonlinear BSS strategy. Our results highlight the utility of linear PCA and ICA
for accurately and reliably recovering nonlinearly mixed sources---and further
suggest the importance of employing sensors with sufficient dimensionality to
identify true hidden sources of real-world data
Neuroscience-inspired online unsupervised learning algorithms
Although the currently popular deep learning networks achieve unprecedented
performance on some tasks, the human brain still has a monopoly on general
intelligence. Motivated by this and biological implausibility of deep learning
networks, we developed a family of biologically plausible artificial neural
networks (NNs) for unsupervised learning. Our approach is based on optimizing
principled objective functions containing a term that matches the pairwise
similarity of outputs to the similarity of inputs, hence the name -
similarity-based. Gradient-based online optimization of such similarity-based
objective functions can be implemented by NNs with biologically plausible local
learning rules. Similarity-based cost functions and associated NNs solve
unsupervised learning tasks such as linear dimensionality reduction, sparse
and/or nonnegative feature extraction, blind nonnegative source separation,
clustering and manifold learning.Comment: Accepted for publication in IEEE Signal Processing Magazin
Contrastive Similarity Matching for Supervised Learning
We propose a novel biologically-plausible solution to the credit assignment
problem motivated by observations in the ventral visual pathway and trained
deep neural networks. In both, representations of objects in the same category
become progressively more similar, while objects belonging to different
categories become less similar. We use this observation to motivate a
layer-specific learning goal in a deep network: each layer aims to learn a
representational similarity matrix that interpolates between previous and later
layers. We formulate this idea using a contrastive similarity matching
objective function and derive from it deep neural networks with feedforward,
lateral, and feedback connections, and neurons that exhibit
biologically-plausible Hebbian and anti-Hebbian plasticity. Contrastive
similarity matching can be interpreted as an energy-based learning algorithm,
but with significant differences from others in how a contrastive function is
constructed