Question Dependent Recurrent Entity Network for Question Answering

Question Answering is a task which requires building models capable of providing answers to questions expressed in human language. Full question answering involves some form of reasoning ability. We introduce a neural network architecture for this task, which is a form of $Memory\ Network$, that recognizes entities and their relations to answers through a focus attention mechanism. Our model is named $Question\ Dependent\ Recurrent\ Entity\ Network$ and extends $Recurrent\ Entity\ Network$ by exploiting aspects of the question during the memorization process. We validate the model on both synthetic and real datasets: the $bAbI$ question answering dataset and the $CNN\ \&\ Daily\ News$$reading\ comprehension$ dataset. In our experiments, the models achieved a State-of-The-Art in the former and competitive results in the latter.Comment: 14 page

Question Dependent Recurrent Entity Network for Question Answering

Question Answering is a task which requires building models capable of providing answers to questions expressed in human language. Full question answering involves some form of reasoning ability. We introduce a neural network architecture for this task, which is a form of Memory Network, that recognizes entities and their relations to answers through a focus attention mechanism. Our model is named Question Dependent Recurrent Entity Network and extends the Recurrent Entity Network by exploiting aspects of the question during the memorization process. We validate the model on both synthetic and real datasets: the bAbI question answering dataset and the CNN & Daily News reading comprehension dataset. In our experiments, our models improved the existing Recurrent Entity Network and achieved competitive results in both dataset

Modeling Task Effects in Human Reading with Neural Attention

Humans read by making a sequence of fixations and saccades. They often skip words, without apparent detriment to understanding. We offer a novel explanation for skipping: readers optimize a tradeoff between performing a language-related task and fixating as few words as possible. We propose a neural architecture that combines an attention module (deciding whether to skip words) and a task module (memorizing the input). We show that our model predicts human skipping behavior, while also modeling reading times well, even though it skips 40% of the input. A key prediction of our model is that different reading tasks should result in different skipping behaviors. We confirm this prediction in an eye-tracking experiment in which participants answers questions about a text. We are able to capture these experimental results using the our model, replacing the memorization module with a task module that performs neural question answering