69,990 research outputs found
AMC: Attention guided Multi-modal Correlation Learning for Image Search
Given a user's query, traditional image search systems rank images according
to its relevance to a single modality (e.g., image content or surrounding
text). Nowadays, an increasing number of images on the Internet are available
with associated meta data in rich modalities (e.g., titles, keywords, tags,
etc.), which can be exploited for better similarity measure with queries. In
this paper, we leverage visual and textual modalities for image search by
learning their correlation with input query. According to the intent of query,
attention mechanism can be introduced to adaptively balance the importance of
different modalities. We propose a novel Attention guided Multi-modal
Correlation (AMC) learning method which consists of a jointly learned hierarchy
of intra and inter-attention networks. Conditioned on query's intent,
intra-attention networks (i.e., visual intra-attention network and language
intra-attention network) attend on informative parts within each modality; a
multi-modal inter-attention network promotes the importance of the most
query-relevant modalities. In experiments, we evaluate AMC models on the search
logs from two real world image search engines and show a significant boost on
the ranking of user-clicked images in search results. Additionally, we extend
AMC models to caption ranking task on COCO dataset and achieve competitive
results compared with recent state-of-the-arts.Comment: CVPR 201
Cognitive Personalized Search Integrating Large Language Models with an Efficient Memory Mechanism
Traditional search engines usually provide identical search results for all
users, overlooking individual preferences. To counter this limitation,
personalized search has been developed to re-rank results based on user
preferences derived from query logs. Deep learning-based personalized search
methods have shown promise, but they rely heavily on abundant training data,
making them susceptible to data sparsity challenges. This paper proposes a
Cognitive Personalized Search (CoPS) model, which integrates Large Language
Models (LLMs) with a cognitive memory mechanism inspired by human cognition.
CoPS employs LLMs to enhance user modeling and user search experience. The
cognitive memory mechanism comprises sensory memory for quick sensory
responses, working memory for sophisticated cognitive responses, and long-term
memory for storing historical interactions. CoPS handles new queries using a
three-step approach: identifying re-finding behaviors, constructing user
profiles with relevant historical information, and ranking documents based on
personalized query intent. Experiments show that CoPS outperforms baseline
models in zero-shot scenarios.Comment: Accepted by WWW 202
Intent Models for Contextualising and Diversifying Query Suggestions
The query suggestion or auto-completion mechanisms help users to type less
while interacting with a search engine. A basic approach that ranks suggestions
according to their frequency in the query logs is suboptimal. Firstly, many
candidate queries with the same prefix can be removed as redundant. Secondly,
the suggestions can also be personalised based on the user's context. These two
directions to improve the aforementioned mechanisms' quality can be in
opposition: while the latter aims to promote suggestions that address search
intents that a user is likely to have, the former aims to diversify the
suggestions to cover as many intents as possible. We introduce a
contextualisation framework that utilises a short-term context using the user's
behaviour within the current search session, such as the previous query, the
documents examined, and the candidate query suggestions that the user has
discarded. This short-term context is used to contextualise and diversify the
ranking of query suggestions, by modelling the user's information need as a
mixture of intent-specific user models. The evaluation is performed offline on
a set of approximately 1.0M test user sessions. Our results suggest that the
proposed approach significantly improves query suggestions compared to the
baseline approach.Comment: A short version of this paper was presented at CIKM 201
- …