338 research outputs found
An Attention-Based Model for Predicting Contextual Informativeness and Curriculum Learning Applications
Both humans and machines learn the meaning of unknown words through
contextual information in a sentence, but not all contexts are equally helpful
for learning. We introduce an effective method for capturing the level of
contextual informativeness with respect to a given target word. Our study makes
three main contributions. First, we develop models for estimating contextual
informativeness, focusing on the instructional aspect of sentences. Our
attention-based approach using pre-trained embeddings demonstrates
state-of-the-art performance on our single-context dataset and an existing
multi-sentence context dataset. Second, we show how our model identifies key
contextual elements in a sentence that are likely to contribute most to a
reader's understanding of the target word. Third, we examine how our contextual
informativeness model, originally developed for vocabulary learning
applications for students, can be used for developing better training curricula
for word embedding models in batch learning and few-shot machine learning
settings. We believe our results open new possibilities for applications that
support language learning for both human and machine learner
Visual Question Answering: A Survey of Methods and Datasets
Visual Question Answering (VQA) is a challenging task that has received
increasing attention from both the computer vision and the natural language
processing communities. Given an image and a question in natural language, it
requires reasoning over visual elements of the image and general knowledge to
infer the correct answer. In the first part of this survey, we examine the
state of the art by comparing modern approaches to the problem. We classify
methods by their mechanism to connect the visual and textual modalities. In
particular, we examine the common approach of combining convolutional and
recurrent neural networks to map images and questions to a common feature
space. We also discuss memory-augmented and modular architectures that
interface with structured knowledge bases. In the second part of this survey,
we review the datasets available for training and evaluating VQA systems. The
various datatsets contain questions at different levels of complexity, which
require different capabilities and types of reasoning. We examine in depth the
question/answer pairs from the Visual Genome project, and evaluate the
relevance of the structured annotations of images with scene graphs for VQA.
Finally, we discuss promising future directions for the field, in particular
the connection to structured knowledge bases and the use of natural language
processing models.Comment: 25 page
CEQE: Contextualized Embeddings for Query Expansion
In this work we leverage recent advances in context-sensitive language models to improve the task of query expansion. Contextualized word representation models, such as ELMo and BERT, are rapidly replacing static embedding models. We propose a new model, Contextualized Embeddings for Query Expansion (CEQE), that utilizes query-focused contextualized embedding vectors. We study the behavior of contextual representations generated for query expansion in ad-hoc document retrieval. We conduct our experiments on probabilistic retrieval models as well as in combination with neural ranking models. We evaluate CEQE on two standard TREC collections: Robust and Deep Learning. We find that CEQE outperforms static embedding-based expansion methods on multiple collections (by up to 18% on Robust and 31% on Deep Learning on average precision) and also improves over proven probabilistic pseudo-relevance feedback (PRF) models. We further find that multiple passes of expansion and reranking result in continued gains in effectiveness with CEQE-based approaches outperforming other approaches. The final model incorporating neural and CEQE-based expansion score achieves gains of up to 5% in P@20 and 2% in AP on Robust over the state-of-the-art transformer-based re-ranking model, Birch
Decoding EEG brain activity for multi-modal natural language processing
Until recently, human behavioral data from reading has mainly been of
interest to researchers to understand human cognition. However, these human
language processing signals can also be beneficial in machine learning-based
natural language processing tasks. Using EEG brain activity to this purpose is
largely unexplored as of yet. In this paper, we present the first large-scale
study of systematically analyzing the potential of EEG brain activity data for
improving natural language processing tasks, with a special focus on which
features of the signal are most beneficial. We present a multi-modal machine
learning architecture that learns jointly from textual input as well as from
EEG features. We find that filtering the EEG signals into frequency bands is
more beneficial than using the broadband signal. Moreover, for a range of word
embedding types, EEG data improves binary and ternary sentiment classification
and outperforms multiple baselines. For more complex tasks such as relation
detection, further research is needed. Finally, EEG data shows to be
particularly promising when limited training data is available
- …