3,546 research outputs found
Similarity of Semantic Relations
There are at least two kinds of similarity. Relational similarity is
correspondence between relations, in contrast with attributional similarity,
which is correspondence between attributes. When two words have a high
degree of attributional similarity, we call them synonyms. When two pairs
of words have a high degree of relational similarity, we say that their
relations are analogous. For example, the word pair mason:stone is analogous
to the pair carpenter:wood. This paper introduces Latent Relational Analysis (LRA),
a method for measuring relational similarity. LRA has potential applications in many
areas, including information extraction, word sense disambiguation,
and information retrieval. Recently the Vector Space Model (VSM) of information
retrieval has been adapted to measuring relational similarity,
achieving a score of 47% on a collection of 374 college-level multiple-choice
word analogy questions. In the VSM approach, the relation between a pair of words is
characterized by a vector of frequencies of predefined patterns in a large corpus.
LRA extends the VSM approach in three ways: (1) the patterns are derived automatically
from the corpus, (2) the Singular Value Decomposition (SVD) is used to smooth the frequency
data, and (3) automatically generated synonyms are used to explore variations of the
word pairs. LRA achieves 56% on the 374 analogy questions, statistically equivalent to the
average human score of 57%. On the related problem of classifying semantic relations, LRA
achieves similar gains over the VSM
Im-Promptu: In-Context Composition from Image Prompts
Large language models are few-shot learners that can solve diverse tasks from
a handful of demonstrations. This implicit understanding of tasks suggests that
the attention mechanisms over word tokens may play a role in analogical
reasoning. In this work, we investigate whether analogical reasoning can enable
in-context composition over composable elements of visual stimuli. First, we
introduce a suite of three benchmarks to test the generalization properties of
a visual in-context learner. We formalize the notion of an analogy-based
in-context learner and use it to design a meta-learning framework called
Im-Promptu. Whereas the requisite token granularity for language is well
established, the appropriate compositional granularity for enabling in-context
generalization in visual stimuli is usually unspecified. To this end, we use
Im-Promptu to train multiple agents with different levels of compositionality,
including vector representations, patch representations, and object slots. Our
experiments reveal tradeoffs between extrapolation abilities and the degree of
compositionality, with non-compositional representations extending learned
composition rules to unseen domains but performing poorly on combinatorial
tasks. Patch-based representations require patches to contain entire objects
for robust extrapolation. At the same time, object-centric tokenizers coupled
with a cross-attention module generate consistent and high-fidelity solutions,
with these inductive biases being particularly crucial for compositional
generalization. Lastly, we demonstrate a use case of Im-Promptu as an intuitive
programming interface for image generation
Through the Lens of Core Competency: Survey on Evaluation of Large Language Models
From pre-trained language model (PLM) to large language model (LLM), the
field of natural language processing (NLP) has witnessed steep performance
gains and wide practical uses. The evaluation of a research field guides its
direction of improvement. However, LLMs are extremely hard to thoroughly
evaluate for two reasons. First of all, traditional NLP tasks become inadequate
due to the excellent performance of LLM. Secondly, existing evaluation tasks
are difficult to keep up with the wide range of applications in real-world
scenarios. To tackle these problems, existing works proposed various benchmarks
to better evaluate LLMs. To clarify the numerous evaluation tasks in both
academia and industry, we investigate multiple papers concerning LLM
evaluations. We summarize 4 core competencies of LLM, including reasoning,
knowledge, reliability, and safety. For every competency, we introduce its
definition, corresponding benchmarks, and metrics. Under this competency
architecture, similar tasks are combined to reflect corresponding ability,
while new tasks can also be easily added into the system. Finally, we give our
suggestions on the future direction of LLM's evaluation
Data-Driven Design-by-Analogy: State of the Art and Future Directions
Design-by-Analogy (DbA) is a design methodology wherein new solutions,
opportunities or designs are generated in a target domain based on inspiration
drawn from a source domain; it can benefit designers in mitigating design
fixation and improving design ideation outcomes. Recently, the increasingly
available design databases and rapidly advancing data science and artificial
intelligence technologies have presented new opportunities for developing
data-driven methods and tools for DbA support. In this study, we survey
existing data-driven DbA studies and categorize individual studies according to
the data, methods, and applications in four categories, namely, analogy
encoding, retrieval, mapping, and evaluation. Based on both nuanced organic
review and structured analysis, this paper elucidates the state of the art of
data-driven DbA research to date and benchmarks it with the frontier of data
science and AI research to identify promising research opportunities and
directions for the field. Finally, we propose a future conceptual data-driven
DbA system that integrates all propositions.Comment: A Preprint Versio
Analogy-Forming Transformers for Few-Shot 3D Parsing
We present Analogical Networks, a model that encodes domain knowledge
explicitly, in a collection of structured labelled 3D scenes, in addition to
implicitly, as model parameters, and segments 3D object scenes with analogical
reasoning: instead of mapping a scene to part segments directly, our model
first retrieves related scenes from memory and their corresponding part
structures, and then predicts analogous part structures for the input scene,
via an end-to-end learnable modulation mechanism. By conditioning on more than
one retrieved memories, compositions of structures are predicted, that mix and
match parts across the retrieved memories. One-shot, few-shot or many-shot
learning are treated uniformly in Analogical Networks, by conditioning on the
appropriate set of memories, whether taken from a single, few or many memory
exemplars, and inferring analogous parses. We show Analogical Networks are
competitive with state-of-the-art 3D segmentation transformers in many-shot
settings, and outperform them, as well as existing paradigms of meta-learning
and few-shot learning, in few-shot settings. Analogical Networks successfully
segment instances of novel object categories simply by expanding their memory,
without any weight updates. Our code and models are publicly available in the
project webpage: http://analogicalnets.github.io/.Comment: ICLR 202
Neural Analogical Matching
Analogy is core to human cognition. It allows us to solve problems based on
prior experience, it governs the way we conceptualize new information, and it
even influences our visual perception. The importance of analogy to humans has
made it an active area of research in the broader field of artificial
intelligence, resulting in data-efficient models that learn and reason in
human-like ways. While cognitive perspectives of analogy and deep learning have
generally been studied independently of one another, the integration of the two
lines of research is a promising step towards more robust and efficient
learning techniques. As part of a growing body of research on such an
integration, we introduce the Analogical Matching Network: a neural
architecture that learns to produce analogies between structured, symbolic
representations that are largely consistent with the principles of
Structure-Mapping Theory.Comment: AAAI versio
- …