10,653 research outputs found
Pose-Guided Multi-Granularity Attention Network for Text-Based Person Search
Text-based person search aims to retrieve the corresponding person images in
an image database by virtue of a describing sentence about the person, which
poses great potential for various applications such as video surveillance.
Extracting visual contents corresponding to the human description is the key to
this cross-modal matching problem. Moreover, correlated images and descriptions
involve different granularities of semantic relevance, which is usually ignored
in previous methods. To exploit the multilevel corresponding visual contents,
we propose a pose-guided multi-granularity attention network (PMA). Firstly, we
propose a coarse alignment network (CA) to select the related image regions to
the global description by a similarity-based attention. To further capture the
phrase-related visual body part, a fine-grained alignment network (FA) is
proposed, which employs pose information to learn latent semantic alignment
between visual body part and textual noun phrase. To verify the effectiveness
of our model, we perform extensive experiments on the CUHK Person Description
Dataset (CUHK-PEDES) which is currently the only available dataset for
text-based person search. Experimental results show that our approach
outperforms the state-of-the-art methods by 15 \% in terms of the top-1 metric.Comment: published in AAAI2020(oral
SMAN : Stacked Multi-Modal Attention Network for cross-modal image-text retrieval
This article focuses on tackling the task of the cross-modal image-text retrieval which has been an interdisciplinary topic in both computer vision and natural language processing communities. Existing global representation alignment-based methods fail to pinpoint the semantically meaningful portion of images and texts, while the local representation alignment schemes suffer from the huge computational burden for aggregating the similarity of visual fragments and textual words exhaustively. In this article, we propose a stacked multimodal attention network (SMAN) that makes use of the stacked multimodal attention mechanism to exploit the fine-grained interdependencies between image and text, thereby mapping the aggregation of attentive fragments into a common space for measuring cross-modal similarity. Specifically, we sequentially employ intramodal information and multimodal information as guidance to perform multiple-step attention reasoning so that the fine-grained correlation between image and text can be modeled. As a consequence, we are capable of discovering the semantically meaningful visual regions or words in a sentence which contributes to measuring the cross-modal similarity in a more precise manner. Moreover, we present a novel bidirectional ranking loss that enforces the distance among pairwise multimodal instances to be closer. Doing so allows us to make full use of pairwise supervised information to preserve the manifold structure of heterogeneous pairwise data. Extensive experiments on two benchmark datasets demonstrate that our SMAN consistently yields competitive performance compared to state-of-the-art methods
Attentive Aspect Modeling for Review-aware Recommendation
In recent years, many studies extract aspects from user reviews and integrate
them with ratings for improving the recommendation performance. The common
aspects mentioned in a user's reviews and a product's reviews indicate indirect
connections between the user and product. However, these aspect-based methods
suffer from two problems. First, the common aspects are usually very sparse,
which is caused by the sparsity of user-product interactions and the diversity
of individual users' vocabularies. Second, a user's interests on aspects could
be different with respect to different products, which are usually assumed to
be static in existing methods. In this paper, we propose an Attentive
Aspect-based Recommendation Model (AARM) to tackle these challenges. For the
first problem, to enrich the aspect connections between user and product,
besides common aspects, AARM also models the interactions between synonymous
and similar aspects. For the second problem, a neural attention network which
simultaneously considers user, product and aspect information is constructed to
capture a user's attention towards aspects when examining different products.
Extensive quantitative and qualitative experiments show that AARM can
effectively alleviate the two aforementioned problems and significantly
outperforms several state-of-the-art recommendation methods on top-N
recommendation task.Comment: Camera-ready manuscript for TOI
- …