2,911 research outputs found
Supervised and Unsupervised Transfer Learning for Question Answering
Although transfer learning has been shown to be successful for tasks like
object and speech recognition, its applicability to question answering (QA) has
yet to be well-studied. In this paper, we conduct extensive experiments to
investigate the transferability of knowledge learned from a source QA dataset
to a target dataset using two QA models. The performance of both models on a
TOEFL listening comprehension test (Tseng et al., 2016) and MCTest (Richardson
et al., 2013) is significantly improved via a simple transfer learning
technique from MovieQA (Tapaswi et al., 2016). In particular, one of the models
achieves the state-of-the-art on all target datasets; for the TOEFL listening
comprehension test, it outperforms the previous best model by 7%. Finally, we
show that transfer learning is helpful even in unsupervised scenarios when
correct answers for target QA dataset examples are not available.Comment: To appear in NAACL HLT 2018 (long paper
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns
visual concepts, words, and semantic parsing of sentences without explicit
supervision on any of them; instead, our model learns by simply looking at
images and reading paired questions and answers. Our model builds an
object-based scene representation and translates sentences into executable,
symbolic programs. To bridge the learning of two modules, we use a
neuro-symbolic reasoning module that executes these programs on the latent
scene representation. Analogical to human concept learning, the perception
module learns visual concepts based on the language description of the object
being referred to. Meanwhile, the learned visual concepts facilitate learning
new words and parsing new sentences. We use curriculum learning to guide the
searching over the large compositional space of images and language. Extensive
experiments demonstrate the accuracy and efficiency of our model on learning
visual concepts, word representations, and semantic parsing of sentences.
Further, our method allows easy generalization to new object attributes,
compositions, language concepts, scenes and questions, and even new program
domains. It also empowers applications including visual question answering and
bidirectional image-text retrieval.Comment: ICLR 2019 (Oral). Project page: http://nscl.csail.mit.edu
A Method for Creating Structural Models of Text Documents Using Neural Networks.
The article describes modern neural network BERT-based models and considers their application for Natural Language Processing tasks such as question answering and named entity recognition. The article presents a method for solving the problem of automatically creating structural models of text documents. The proposed method is hybrid and is based on jointly utilizing several NLP models. The method builds a structural model of a document by extracting sentences that correspond to various aspects of the document. Information extraction is performed by using the BERT Question Answering model with questions that are prepared separately for each aspect. The answers are filtered via the BERT Named Entity Recognition model and used to generate the contents of each field of the structural model. The article proposes two algorithms for field content generation: Exclusive answer choosing algorithm and Generalizing answer forming algorithm, that are used for short and voluminous fields respectively. The article also describes the software implementation of the proposed method and discusses the results of experiments conducted to evaluate the quality of the method.The article describes modern neural network BERT-based models and considers their application for Natural Language Processing tasks such as question answering and named entity recognition. The article presents a method for solving the problem of automatically creating structural models of text documents. The proposed method is hybrid and is based on jointly utilizing several NLP models. The method builds a structural model of a document by extracting sentences that correspond to various aspects of the document. Information extraction is performed by using the BERT Question Answering model with questions that are prepared separately for each aspect. The answers are filtered via the BERT Named Entity Recognition model and used to generate the contents of each field of the structural model. The article proposes two algorithms for field content generation: Exclusive answer choosing algorithm and Generalizing answer forming algorithm, that are used for short and voluminous fields respectively. The article also describes the software implementation of the proposed method and discusses the results of experiments conducted to evaluate the quality of the method
Chain of Thought Prompt Tuning in Vision Language Models
Language-Image Pre-training has demonstrated promising results on zero-shot
and few-shot downstream tasks by prompting visual models with natural language
prompts. However, most recent studies only use a single prompt for tuning,
neglecting the inherent step-to-step cognitive reasoning process that humans
conduct in complex task settings, for example, when processing images from
unfamiliar domains. Chain of Thought is a simple and effective approximation to
human reasoning process and has been proven useful for natural language
processing (NLP) tasks. Based on this cognitive intuition, we believe that
conducting effective reasoning is also an important problem in visual tasks,
and a chain of thought could be a solution to this problem. In this work, we
propose a novel chain of thought prompt tuning for vision-language modeling.
Extensive experiments show that our method not only generalizes better in image
classification tasks, has greater transferability beyond a single dataset, and
has stronger domain generalization performance, but also performs much better
in imagetext retrieval and visual question answering, which require more
reasoning capabilities. We are the first to successfully adapt chain-of-thought
prompting that combines visual and textual embeddings. We will release our
code
Benchmarking Long-tail Generalization with Likelihood Splits
In order to reliably process natural language, NLP systems must generalize to
the long tail of rare utterances. We propose a method to create challenging
benchmarks that require generalizing to the tail of the distribution by
re-splitting existing datasets. We create 'Likelihood Splits' where examples
that are assigned lower likelihood by a pre-trained language model (LM) are
placed in the test set, and more likely examples are in the training set. This
simple approach can be customized to construct meaningful train-test splits for
a wide range of tasks. Likelihood Splits surface more challenges than random
splits: relative error rates of state-of-the-art models increase by 59% for
semantic parsing on Spider, 93% for natural language inference on SNLI, and 33%
for yes/no question answering on BoolQ, on our splits compared with the
corresponding random splits. Moreover, Likelihood Splits create fairer
benchmarks than adversarial filtering; when the LM used to create the splits is
also employed as the task model, our splits do not unfairly penalize the LM.Comment: Updated final Findings of EACL versio
- …