2 research outputs found
Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks
Synaptic plasticity is widely accepted to be the mechanism behind learning in
the brain's neural networks. A central question is how synapses, with access to
only local information about the network, can still organize collectively and
perform circuit-wide learning in an efficient manner. In single-layered and
all-to-all connected neural networks, local plasticity has been shown to
implement gradient-based learning on a class of cost functions that contain a
term that aligns the similarity of outputs to the similarity of inputs. Whether
such cost functions exist for networks with other architectures is not known.
In this paper, we introduce structured and deep similarity matching cost
functions, and show how they can be optimized in a gradient-based manner by
neural networks with local learning rules. These networks extend F\"oldiak's
Hebbian/Anti-Hebbian network to deep architectures and structured feedforward,
lateral and feedback connections. Credit assignment problem is solved elegantly
by a factorization of the dual learning objective to synapse specific local
objectives. Simulations show that our networks learn meaningful features.Comment: Accepted for publication in NeurIPS 2019; Minor typos fixe
A neural network walks into a lab: towards using deep nets as models for human behavior
What might sound like the beginning of a joke has become an attractive
prospect for many cognitive scientists: the use of deep neural network models
(DNNs) as models of human behavior in perceptual and cognitive tasks. Although
DNNs have taken over machine learning, attempts to use them as models of human
behavior are still in the early stages. Can they become a versatile model class
in the cognitive scientist's toolbox? We first argue why DNNs have the
potential to be interesting models of human behavior. We then discuss how that
potential can be more fully realized. On the one hand, we argue that the cycle
of training, testing, and revising DNNs needs to be revisited through the lens
of the cognitive scientist's goals. Specifically, we argue that methods for
assessing the goodness of fit between DNN models and human behavior have to
date been impoverished. On the other hand, cognitive science might have to
start using more complex tasks (including richer stimulus spaces), but doing so
might be beneficial for DNN-independent reasons as well. Finally, we highlight
avenues where traditional cognitive process models and DNNs may show productive
synergy