12 research outputs found
Generating Distractors for Reading Comprehension Questions from Real Examinations
We investigate the task of distractor generation for multiple choice reading
comprehension questions from examinations. In contrast to all previous works,
we do not aim at preparing words or short phrases distractors, instead, we
endeavor to generate longer and semantic-rich distractors which are closer to
distractors in real reading comprehension from examinations. Taking a reading
comprehension article, a pair of question and its correct option as input, our
goal is to generate several distractors which are somehow related to the
answer, consistent with the semantic context of the question and have some
trace in the article. We propose a hierarchical encoder-decoder framework with
static and dynamic attention mechanisms to tackle this task. Specifically, the
dynamic attention can combine sentence-level and word-level attention varying
at each recurrent time step to generate a more readable sequence. The static
attention is to modulate the dynamic attention not to focus on question
irrelevant sentences or sentences which contribute to the correct option. Our
proposed framework outperforms several strong baselines on the first prepared
distractor generation dataset of real reading comprehension questions. For
human evaluation, compared with those distractors generated by baselines, our
generated distractors are more functional to confuse the annotators.Comment: AAAI201
Automatic Distractor Generation for Multiple Choice Questions in Standard Tests
To assess the knowledge proficiency of a learner, multiple choice question is
an efficient and widespread form in standard tests. However, the composition of
the multiple choice question, especially the construction of distractors is
quite challenging. The distractors are required to both incorrect and plausible
enough to confuse the learners who did not master the knowledge. Currently, the
distractors are generated by domain experts which are both expensive and
time-consuming. This urges the emergence of automatic distractor generation,
which can benefit various standard tests in a wide range of domains. In this
paper, we propose a question and answer guided distractor generation (EDGE)
framework to automate distractor generation. EDGE consists of three major
modules: (1) the Reforming Question Module and the Reforming Passage Module
apply gate layers to guarantee the inherent incorrectness of the generated
distractors; (2) the Distractor Generator Module applies attention mechanism to
control the level of plausibility. Experimental results on a large-scale public
dataset demonstrate that our model significantly outperforms existing models
and achieves a new state-of-the-art.Comment: accepted by COLING202
Knowledge-Driven Distractor Generation for Cloze-style Multiple Choice Questions
In this paper, we propose a novel configurable framework to automatically
generate distractive choices for open-domain cloze-style multiple-choice
questions, which incorporates a general-purpose knowledge base to effectively
create a small distractor candidate set, and a feature-rich learning-to-rank
model to select distractors that are both plausible and reliable. Experimental
results on datasets across four domains show that our framework yields
distractors that are more plausible and reliable than previous methods. This
dataset can also be used as a benchmark for distractor generation in the
future.Comment: To appear at AAAI 202
Knowledge Questions from Knowledge Graphs
We address the novel problem of automatically generating quiz-style knowledge questions from a knowledge graph such as DBpedia. Questions of this kind have ample applications, for instance, to educate users about or to evaluate their knowledge in a specific domain. To solve the problem, we propose an end-to-end approach. The approach first selects a named entity from the knowledge graph as an answer. It then generates a structured triple-pattern query, which yields the answer as its sole result. If a multiple-choice question is desired, the approach selects alternative answer options. Finally, our approach uses a template-based method to verbalize the structured query and yield a natural language question. A key challenge is estimating how difficult the generated question is to human users. To do this, we make use of historical data from the Jeopardy! quiz show and a semantically annotated Web-scale document collection, engineer suitable features, and train a logistic regression classifier to predict question difficulty. Experiments demonstrate the viability of our overall approach