190 research outputs found
Efficient Image Gallery Representations at Scale Through Multi-Task Learning
Image galleries provide a rich source of diverse information about a product
which can be leveraged across many recommendation and retrieval applications.
We study the problem of building a universal image gallery encoder through
multi-task learning (MTL) approach and demonstrate that it is indeed a
practical way to achieve generalizability of learned representations to new
downstream tasks. Additionally, we analyze the relative predictive performance
of MTL-trained solutions against optimal and substantially more expensive
solutions, and find signals that MTL can be a useful mechanism to address
sparsity in low-resource binary tasks.Comment: Proceedings of the 43rd International ACM SIGIR Conference on
Research and Development in Information Retrieva
Context-Aware Embeddings for Automatic Art Analysis
Automatic art analysis aims to classify and retrieve artistic representations
from a collection of images by using computer vision and machine learning
techniques. In this work, we propose to enhance visual representations from
neural networks with contextual artistic information. Whereas visual
representations are able to capture information about the content and the style
of an artwork, our proposed context-aware embeddings additionally encode
relationships between different artistic attributes, such as author, school, or
historical period. We design two different approaches for using context in
automatic art analysis. In the first one, contextual data is obtained through a
multi-task learning model, in which several attributes are trained together to
find visual relationships between elements. In the second approach, context is
obtained through an art-specific knowledge graph, which encodes relationships
between artistic attributes. An exhaustive evaluation of both of our models in
several art analysis problems, such as author identification, type
classification, or cross-modal retrieval, show that performance is improved by
up to 7.3% in art classification and 37.24% in retrieval when context-aware
embeddings are used
- …