655 research outputs found
Multi-grained Evidence Inference for Multi-choice Reading Comprehension
Multi-choice Machine Reading Comprehension (MRC) is a major and challenging
task for machines to answer questions according to provided options. Answers in
multi-choice MRC cannot be directly extracted in the given passages, and
essentially require machines capable of reasoning from accurate extracted
evidence. However, the critical evidence may be as simple as just one word or
phrase, while it is hidden in the given redundant, noisy passage with multiple
linguistic hierarchies from phrase, fragment, sentence until the entire
passage. We thus propose a novel general-purpose model enhancement which
integrates multi-grained evidence comprehensively, named Multi-grained evidence
inferencer (Mugen), to make up for the inability. Mugen extracts three
different granularities of evidence: coarse-, middle- and fine-grained
evidence, and integrates evidence with the original passages, achieving
significant and consistent performance improvement on four multi-choice MRC
benchmarks.Comment: Accepted by TASLP 2023, vol. 31, pp. 3896-390
Rationalization for Explainable NLP: A Survey
Recent advances in deep learning have improved the performance of many
Natural Language Processing (NLP) tasks such as translation,
question-answering, and text classification. However, this improvement comes at
the expense of model explainability. Black-box models make it difficult to
understand the internals of a system and the process it takes to arrive at an
output. Numerical (LIME, Shapley) and visualization (saliency heatmap)
explainability techniques are helpful; however, they are insufficient because
they require specialized knowledge. These factors led rationalization to emerge
as a more accessible explainable technique in NLP. Rationalization justifies a
model's output by providing a natural language explanation (rationale). Recent
improvements in natural language generation have made rationalization an
attractive technique because it is intuitive, human-comprehensible, and
accessible to non-technical users. Since rationalization is a relatively new
field, it is disorganized. As the first survey, rationalization literature in
NLP from 2007-2022 is analyzed. This survey presents available methods,
explainable evaluations, code, and datasets used across various NLP tasks that
use rationalization. Further, a new subfield in Explainable AI (XAI), namely,
Rational AI (RAI), is introduced to advance the current state of
rationalization. A discussion on observed insights, challenges, and future
directions is provided to point to promising research opportunities
Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering
Recent developments in pre-trained neural language modeling have led to leaps
in accuracy on commonsense question-answering benchmarks. However, there is
increasing concern that models overfit to specific tasks, without learning to
utilize external knowledge or perform general semantic reasoning. In contrast,
zero-shot evaluations have shown promise as a more robust measure of a model's
general reasoning abilities. In this paper, we propose a novel neuro-symbolic
framework for zero-shot question answering across commonsense tasks. Guided by
a set of hypotheses, the framework studies how to transform various
pre-existing knowledge resources into a form that is most effective for
pre-training models. We vary the set of language models, training regimes,
knowledge sources, and data generation strategies, and measure their impact
across tasks. Extending on prior work, we devise and compare four constrained
distractor-sampling strategies. We provide empirical results across five
commonsense question-answering tasks with data generated from five external
knowledge resources. We show that, while an individual knowledge graph is
better suited for specific tasks, a global knowledge graph brings consistent
gains across different tasks. In addition, both preserving the structure of the
task as well as generating fair and informative questions help language models
learn more effectively.Comment: AAAI 202
- …