1 research outputs found
GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level
Scenario-based question answering (SQA) has attracted increasing research
attention. It typically requires retrieving and integrating knowledge from
multiple sources, and applying general knowledge to a specific case described
by a scenario. SQA widely exists in the medical, geography, and legal
domains---both in practice and in the exams. In this paper, we introduce the
GeoSQA dataset. It consists of 1,981 scenarios and 4,110 multiple-choice
questions in the geography domain at high school level, where diagrams (e.g.,
maps, charts) have been manually annotated with natural language descriptions
to benefit NLP research. Benchmark results on a variety of state-of-the-art
methods for question answering, textual entailment, and reading comprehension
demonstrate the unique challenges presented by SQA for future research.Comment: 6 pages, to appear at the 2019 Conference on Empirical Methods in
Natural Language Processing and 9th International Joint Conference on Natural
Language Processing (EMNLP-IJCNLP 2019