1,086 research outputs found
Context Aware Textual Entailment
In conversations, stories, news reporting, and other forms of natural language, understanding requires participants to make assumptions (hypothesis) based on background knowledge, a process called entailment. These assumptions may then be supported, contradicted, or refined as a conversation or story progresses and additional facts become known and context changes. It is often the case that we do not know an aspect of the story with certainty but rather believe it to be the case; i.e., what we know is associated with uncertainty or ambiguity. In this research a method has been developed to identify different contexts of the input raw text along with specific features of the contexts such as time, location, and objects. The method includes a two-phase SVM classifier along with a voting mechanism in the second phase to identify the contexts. Rule-based algorithms were utilized to extract the context elements. This research also develops a new contextË—aware text representation. This representation maintains semantic aspects of sentences, as well as textual contexts and context elements. The method can offer both graph representation and First-Order-Logic representation of the text. This research also extracts a First-Order Logic (FOL) and XML representation of a text or series of texts. The method includes entailment using background knowledge from sources (VerbOcean and WordNet), with resolution of conflicts between extracted clauses, and handling the role of context in resolving uncertain truth
Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment
Designing systems that can reason across cultures requires that they are
grounded in the norms of the contexts in which they operate. However, current
research on developing computational models of social norms has primarily
focused on American society. Here, we propose a novel approach to discover and
compare descriptive social norms across Chinese and American cultures. We
demonstrate our approach by leveraging discussions on a Chinese Q&A platform
(Zhihu) and the existing SocialChemistry dataset as proxies for contrasting
cultural axes, align social situations cross-culturally, and extract social
norms from texts using in-context learning. Embedding Chain-of-Thought
prompting in a human-AI collaborative framework, we build a high-quality
dataset of 3,069 social norms aligned with social situations across Chinese and
American cultures alongside corresponding free-text explanations. To test the
ability of models to reason about social norms across cultures, we introduce
the task of explainable social norm entailment, showing that existing models
under 3B parameters have significant room for improvement in both automatic and
human evaluation. Further analysis of cross-cultural norm differences based on
our dataset shows empirical alignment with the social orientations framework,
revealing several situational and descriptive nuances in norms across these
cultures.Comment: EMNLP 2023 Main Conference (Long Paper
Fact Checking in Community Forums
Community Question Answering (cQA) forums are very popular nowadays, as they
represent effective means for communities around particular topics to share
information. Unfortunately, this information is not always factual. Thus, here
we explore a new dimension in the context of cQA, which has been ignored so
far: checking the veracity of answers to particular questions in cQA forums. As
this is a new problem, we create a specialized dataset for it. We further
propose a novel multi-faceted model, which captures information from the answer
content (what is said and how), from the author profile (who says it), from the
rest of the community forum (where it is said), and from external authoritative
sources of information (external support). Evaluation results show a MAP value
of 86.54, which is 21 points absolute above the baseline.Comment: AAAI-2018; Fact-Checking; Veracity; Community-Question Answering;
Neural Networks; Distributed Representation
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
Natural language inference (NLI) is an increasingly important task for
natural language understanding, which requires one to infer whether a sentence
entails another. However, the ability of NLI models to make pragmatic
inferences remains understudied. We create an IMPlicature and PRESupposition
diagnostic dataset (IMPPRES), consisting of >25k semiautomatically generated
sentence pairs illustrating well-studied pragmatic inference types. We use
IMPPRES to evaluate whether BERT, InferSent, and BOW NLI models trained on
MultiNLI (Williams et al., 2018) learn to make pragmatic inferences. Although
MultiNLI appears to contain very few pairs illustrating these inference types,
we find that BERT learns to draw pragmatic inferences. It reliably treats
scalar implicatures triggered by "some" as entailments. For some presupposition
triggers like "only", BERT reliably recognizes the presupposition as an
entailment, even when the trigger is embedded under an entailment canceling
operator like negation. BOW and InferSent show weaker evidence of pragmatic
reasoning. We conclude that NLI training encourages models to learn some, but
not all, pragmatic inferences.Comment: to appear in Proceedings of ACL 202
- …