79,732 research outputs found
How much should you ask? On the question structure in QA systems
Datasets that boosted state-of-the-art solutions for Question Answering (QA)
systems prove that it is possible to ask questions in natural language manner.
However, users are still used to query-like systems where they type in keywords
to search for answer. In this study we validate which parts of questions are
essential for obtaining valid answer. In order to conclude that, we take
advantage of LIME - a framework that explains prediction by local
approximation. We find that grammar and natural language is disregarded by QA.
State-of-the-art model can answer properly even if 'asked' only with a few
words with high coefficients calculated with LIME. According to our knowledge,
it is the first time that QA model is being explained by LIME.Comment: Accepted to Analyzing and interpreting neural networks for NLP
workshop at EMNLP 201
An instruction systolic array architecture for multiple neural network types
Modern electronic systems, especially sensor and imaging systems, are beginning to
incorporate their own neural network subsystems. In order for these neural systems to learn in
real-time they must be implemented using VLSI technology, with as much of the learning
processes incorporated on-chip as is possible. The majority of current VLSI implementations
literally implement a series of neural processing cells, which can be connected together in an
arbitrary fashion. Many do not perform the entire neural learning process on-chip, instead
relying on other external systems to carry out part of the computation requirements of the
algorithm.
The work presented here utilises two dimensional instruction systolic arrays in an attempt to
define a general neural architecture which is closer to the biological basis of neural networks - it
is the synapses themselves, rather than the neurons, that have dedicated processing units. A
unified architecture is described which can be programmed at the microcode level in order to
facilitate the processing of multiple neural network types.
An essential part of neural network processing is the neuron activation function, which can
range from a sequential algorithm to a discrete mathematical expression. The architecture
presented can easily carry out the sequential functions, and introduces a fast method of
mathematical approximation for the more complex functions. This can be evaluated on-chip,
thus implementing the entire neural process within a single system.
VHDL circuit descriptions for the chip have been generated, and the systolic processing
algorithms and associated microcode instruction set for three different neural paradigms have
been designed. A software simulator of the architecture has been written, giving results for
several common applications in the field
A Neural Network model with Bidirectional Whitening
We present here a new model and algorithm which performs an efficient Natural
gradient descent for Multilayer Perceptrons. Natural gradient descent was
originally proposed from a point of view of information geometry, and it
performs the steepest descent updates on manifolds in a Riemannian space. In
particular, we extend an approach taken by the "Whitened neural networks"
model. We make the whitening process not only in feed-forward direction as in
the original model, but also in the back-propagation phase. Its efficacy is
shown by an application of this "Bidirectional whitened neural networks" model
to a handwritten character recognition data (MNIST data).Comment: 16page
- …