175,602 research outputs found
Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents
User interaction with voice-powered agents generates large amounts of
unlabeled utterances. In this paper, we explore techniques to efficiently
transfer the knowledge from these unlabeled utterances to improve model
performance on Spoken Language Understanding (SLU) tasks. We use Embeddings
from Language Model (ELMo) to take advantage of unlabeled data by learning
contextualized word representations. Additionally, we propose ELMo-Light
(ELMoL), a faster and simpler unsupervised pre-training method for SLU. Our
findings suggest unsupervised pre-training on a large corpora of unlabeled
utterances leads to significantly better SLU performance compared to training
from scratch and it can even outperform conventional supervised transfer.
Additionally, we show that the gains from unsupervised transfer techniques can
be further improved by supervised transfer. The improvements are more
pronounced in low resource settings and when using only 1000 labeled in-domain
samples, our techniques match the performance of training from scratch on
10-15x more labeled in-domain data.Comment: To appear at AAAI 201
Eliminating Redundant Training Data Using Unsupervised Clustering Techniques
Training data for supervised learning neural networks can be clustered such that the input/output pairs in each cluster are redundant. Redundant training data can adversely affect training time. In this paper we apply two clustering algorithms, ART2 -A and the Generalized Equality Classifier, to identify training data clusters and thus reduce the training data and training time. The approach is demonstrated for a high dimensional nonlinear continuous time mapping. The demonstration shows six-fold decrease in training time at little or no loss of accuracy in the handling of evaluation data
Analysis of Spectrum Occupancy Using Machine Learning Algorithms
In this paper, we analyze the spectrum occupancy using different machine
learning techniques. Both supervised techniques (naive Bayesian classifier
(NBC), decision trees (DT), support vector machine (SVM), linear regression
(LR)) and unsupervised algorithm (hidden markov model (HMM)) are studied to
find the best technique with the highest classification accuracy (CA). A
detailed comparison of the supervised and unsupervised algorithms in terms of
the computational time and classification accuracy is performed. The classified
occupancy status is further utilized to evaluate the probability of secondary
user outage for the future time slots, which can be used by system designers to
define spectrum allocation and spectrum sharing policies. Numerical results
show that SVM is the best algorithm among all the supervised and unsupervised
classifiers. Based on this, we proposed a new SVM algorithm by combining it
with fire fly algorithm (FFA), which is shown to outperform all other
algorithms.Comment: 21 pages, 6 figure
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization
We study unsupervised multi-document summarization evaluation metrics, which
require neither human-written reference summaries nor human annotations (e.g.
preferences, ratings, etc.). We propose SUPERT, which rates the quality of a
summary by measuring its semantic similarity with a pseudo reference summary,
i.e. selected salient sentences from the source documents, using contextualized
embeddings and soft token alignment techniques. Compared to the
state-of-the-art unsupervised evaluation metrics, SUPERT correlates better with
human ratings by 18-39%. Furthermore, we use SUPERT as rewards to guide a
neural-based reinforcement learning summarizer, yielding favorable performance
compared to the state-of-the-art unsupervised summarizers. All source code is
available at https://github.com/yg211/acl20-ref-free-eval.Comment: ACL 202
- …