22,566 research outputs found
A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications
This survey samples from the ever-growing family of adaptive resonance theory
(ART) neural network models used to perform the three primary machine learning
modalities, namely, unsupervised, supervised and reinforcement learning. It
comprises a representative list from classic to modern ART models, thereby
painting a general picture of the architectures developed by researchers over
the past 30 years. The learning dynamics of these ART models are briefly
described, and their distinctive characteristics such as code representation,
long-term memory and corresponding geometric interpretation are discussed.
Useful engineering properties of ART (speed, configurability, explainability,
parallelization and hardware implementation) are examined along with current
challenges. Finally, a compilation of online software libraries is provided. It
is expected that this overview will be helpful to new and seasoned ART
researchers
Uncertainty-Aware Attention for Reliable Interpretation and Prediction
Department of Computer Science and EngineeringAttention mechanism is effective in both focusing the deep learning models on relevant features and
interpreting them. However, attentions may be unreliable since the networks that generate them are
often trained in a weakly-supervised manner. To overcome this limitation, we introduce the notion of
input-dependent uncertainty to the attention mechanism, such that it generates attention for each
feature with varying degrees of noise based on the given input, to learn larger variance on instances it
is uncertain about. We learn this Uncertainty-aware Attention (UA) mechanism using variational
inference, and validate it on various risk prediction tasks from electronic health records on which our
model significantly outperforms existing attention models. The analysis of the learned attentions
shows that our model generates attentions that comply with clinicians' interpretation, and provide
richer interpretation via learned variance. Further evaluation of both the accuracy of the uncertainty
calibration and the prediction performance with "I don't know'' decision show that UA yields networks
with high reliability as well.ope
Applying Winnow to Context-Sensitive Spelling Correction
Multiplicative weight-updating algorithms such as Winnow have been studied
extensively in the COLT literature, but only recently have people started to
use them in applications. In this paper, we apply a Winnow-based algorithm to a
task in natural language: context-sensitive spelling correction. This is the
task of fixing spelling errors that happen to result in valid words, such as
substituting {\it to\/} for {\it too}, {\it casual\/} for {\it causal}, and so
on. Previous approaches to this problem have been statistics-based; we compare
Winnow to one of the more successful such approaches, which uses Bayesian
classifiers. We find that: (1)~When the standard (heavily-pruned) set of
features is used to describe problem instances, Winnow performs comparably to
the Bayesian method; (2)~When the full (unpruned) set of features is used,
Winnow is able to exploit the new features and convincingly outperform Bayes;
and (3)~When a test set is encountered that is dissimilar to the training set,
Winnow is better than Bayes at adapting to the unfamiliar test set, using a
strategy we will present for combining learning on the training set with
unsupervised learning on the (noisy) test set.Comment: 9 page
- …