5,930 research outputs found
Nonnormal amplification in random balanced neuronal networks
In dynamical models of cortical networks, the recurrent connectivity can
amplify the input given to the network in two distinct ways. One is induced by
the presence of near-critical eigenvalues in the connectivity matrix W,
producing large but slow activity fluctuations along the corresponding
eigenvectors (dynamical slowing). The other relies on W being nonnormal, which
allows the network activity to make large but fast excursions along specific
directions. Here we investigate the tradeoff between nonnormal amplification
and dynamical slowing in the spontaneous activity of large random neuronal
networks composed of excitatory and inhibitory neurons. We use a Schur
decomposition of W to separate the two amplification mechanisms. Assuming
linear stochastic dynamics, we derive an exact expression for the expected
amount of purely nonnormal amplification. We find that amplification is very
limited if dynamical slowing must be kept weak. We conclude that, to achieve
strong transient amplification with little slowing, the connectivity must be
structured. We show that unidirectional connections between neurons of the same
type together with reciprocal connections between neurons of different types,
allow for amplification already in the fast dynamical regime. Finally, our
results also shed light on the differences between balanced networks in which
inhibition exactly cancels excitation, and those where inhibition dominates.Comment: 13 pages, 7 figure
Towards Interpretable Deep Learning Models for Knowledge Tracing
As an important technique for modeling the knowledge states of learners, the
traditional knowledge tracing (KT) models have been widely used to support
intelligent tutoring systems and MOOC platforms. Driven by the fast
advancements of deep learning techniques, deep neural network has been recently
adopted to design new KT models for achieving better prediction performance.
However, the lack of interpretability of these models has painfully impeded
their practical applications, as their outputs and working mechanisms suffer
from the intransparent decision process and complex inner structures. We thus
propose to adopt the post-hoc method to tackle the interpretability issue for
deep learning based knowledge tracing (DLKT) models. Specifically, we focus on
applying the layer-wise relevance propagation (LRP) method to interpret
RNN-based DLKT model by backpropagating the relevance from the model's output
layer to its input layer. The experiment results show the feasibility using the
LRP method for interpreting the DLKT model's predictions, and partially
validate the computed relevance scores from both question level and concept
level. We believe it can be a solid step towards fully interpreting the DLKT
models and promote their practical applications in the education domain
Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
Change detection is one of the central problems in earth observation and was
extensively investigated over recent decades. In this paper, we propose a novel
recurrent convolutional neural network (ReCNN) architecture, which is trained
to learn a joint spectral-spatial-temporal feature representation in a unified
framework for change detection in multispectral images. To this end, we bring
together a convolutional neural network (CNN) and a recurrent neural network
(RNN) into one end-to-end network. The former is able to generate rich
spectral-spatial feature representations, while the latter effectively analyzes
temporal dependency in bi-temporal images. In comparison with previous
approaches to change detection, the proposed network architecture possesses
three distinctive properties: 1) It is end-to-end trainable, in contrast to
most existing methods whose components are separately trained or computed; 2)
it naturally harnesses spatial information that has been proven to be
beneficial to change detection task; 3) it is capable of adaptively learning
the temporal dependency between multitemporal images, unlike most of algorithms
that use fairly simple operation like image differencing or stacking. As far as
we know, this is the first time that a recurrent convolutional network
architecture has been proposed for multitemporal remote sensing image analysis.
The proposed network is validated on real multispectral data sets. Both visual
and quantitative analysis of experimental results demonstrates competitive
performance in the proposed mode
- …