28 research outputs found
Recommended from our members
Verbal analogy problem sets: An inventory of testing materials.
Analogical reasoning is an active topic of investigation across education, artificial intelligence (AI), cognitive psychology, and related fields. In all fields of inquiry, explicit analogy problems provide useful tools for investigating the mechanisms underlying analogical reasoning. Such sets have been developed by researchers working in the fields of educational testing, AI, and cognitive psychology. However, these analogy tests have not been systematically made accessible across all the relevant fields. The present paper aims to remedy this situation by presenting a working inventory of verbal analogy problem sets, intended to capture and organize sets from diverse sources
Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns
Attention, specifically scaled dot-product attention, has proven effective
for natural language, but it does not have a mechanism for handling
hierarchical patterns of arbitrary nesting depth, which limits its ability to
recognize certain syntactic structures. To address this shortcoming, we propose
stack attention: an attention operator that incorporates stacks, inspired by
their theoretical connections to context-free languages (CFLs). We show that
stack attention is analogous to standard attention, but with a latent model of
syntax that requires no syntactic supervision. We propose two variants: one
related to deterministic pushdown automata (PDAs) and one based on
nondeterministic PDAs, which allows transformers to recognize arbitrary CFLs.
We show that transformers with stack attention are very effective at learning
CFLs that standard transformers struggle on, achieving strong results on a CFL
with theoretically maximal parsing difficulty. We also show that stack
attention is more effective at natural language modeling under a constrained
parameter budget, and we include results on machine translation.Comment: 20 pages, 4 figures. Published as a spotlight paper at ICLR 202
Multimodal Analogical Reasoning over Knowledge Graphs
Analogical reasoning is fundamental to human cognition and holds an important
place in various fields. However, previous studies mainly focus on single-modal
analogical reasoning and ignore taking advantage of structure knowledge.
Notably, the research in cognitive psychology has demonstrated that information
from multimodal sources always brings more powerful cognitive transfer than
single modality sources. To this end, we introduce the new task of multimodal
analogical reasoning over knowledge graphs, which requires multimodal reasoning
ability with the help of background knowledge. Specifically, we construct a
Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph
MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained
Transformer baselines, illustrating the potential challenges of the proposed
task. We further propose a novel model-agnostic Multimodal analogical reasoning
framework with Transformer (MarT) motivated by the structure mapping theory,
which can obtain better performance. Code and datasets are available in
https://github.com/zjunlp/MKG_Analogy.Comment: Accepted by ICLR 202
Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challenges
The growing prevalence and rapid evolution of offensive language in social
media amplify the complexities of detection, particularly highlighting the
challenges in identifying such content across diverse languages. This survey
presents a systematic and comprehensive exploration of Cross-Lingual Transfer
Learning (CLTL) techniques in offensive language detection in social media. Our
study stands as the first holistic overview to focus exclusively on the
cross-lingual scenario in this domain. We analyse 67 relevant papers and
categorise these studies across various dimensions, including the
characteristics of multilingual datasets used, the cross-lingual resources
employed, and the specific CLTL strategies implemented. According to "what to
transfer", we also summarise three main CLTL transfer approaches: instance,
feature, and parameter transfer. Additionally, we shed light on the current
challenges and future research opportunities in this field. Furthermore, we
have made our survey resources available online, including two comprehensive
tables that provide accessible references to the multilingual datasets and CLTL
methods used in the reviewed literature.Comment: 35 pages, 7 figure
Analogies minus analogy test: measuring regularities in word embeddings
Vector space models of words have long been claimed to capture linguistic
regularities as simple vector translations, but problems have been raised with
this claim. We decompose and empirically analyze the classic arithmetic word
analogy test, to motivate two new metrics that address the issues with the
standard test, and which distinguish between class-wise offset concentration
(similar directions between pairs of words drawn from different broad classes,
such as France--London, China--Ottawa, ...) and pairing consistency (the
existence of a regular transformation between correctly-matched pairs such as
France:Paris::China:Beijing). We show that, while the standard analogy test is
flawed, several popular word embeddings do nevertheless encode linguistic
regularities