15,167 research outputs found
Contextualized Word Representations for Reading Comprehension
Reading a document and extracting an answer to a question about its content
has attracted substantial attention recently. While most work has focused on
the interaction between the question and the document, in this work we evaluate
the importance of context when the question and document are processed
independently. We take a standard neural architecture for this task, and show
that by providing rich contextualized word representations from a large
pre-trained language model as well as allowing the model to choose between
context-dependent and context-independent word representations, we can obtain
dramatic improvements and reach performance comparable to state-of-the-art on
the competitive SQuAD dataset.Comment: 6 pages, 1 figure, NAACL 201
Stochastic Answer Networks for Machine Reading Comprehension
We propose a simple yet robust stochastic answer network (SAN) that simulates
multi-step reasoning in machine reading comprehension. Compared to previous
work such as ReasoNet which used reinforcement learning to determine the number
of steps, the unique feature is the use of a kind of stochastic prediction
dropout on the answer module (final layer) of the neural network during the
training. We show that this simple trick improves robustness and achieves
results competitive to the state-of-the-art on the Stanford Question Answering
Dataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading
COmprehension Dataset (MS MARCO).Comment: 11 pages, 5 figures, Accepted to ACL 201
The brain is a prediction machine that cares about good and bad - Any implications for neuropragmatics?
Experimental pragmatics asks how people construct contextualized meaning in communication. So what does it mean for this field to add neuroas a prefix to its name? After analyzing the options for any subfield of cognitive science, I argue that neuropragmatics can and occasionally should go beyond the instrumental use of EEG or fMRI and beyond mapping classic theoretical distinctions onto Brodmann areas. In particular, if experimental pragmatics ‘goes neuro’, it should take into account that the brain evolved as a control system that helps its bearer negotiate a highly complex, rapidly changing and often not so friendly environment. In this context, the ability to predict current unknowns, and to rapidly tell good from bad, are essential ingredients of processing. Using insights from non-linguistic areas of cognitive neuroscience as well as from EEG research on utterance comprehension, I argue that for a balanced development of experimental pragmatics, these two characteristics of the brain cannot be ignored
- …