2,850 research outputs found
Visual Landmark Recognition from Internet Photo Collections: A Large-Scale Evaluation
The task of a visual landmark recognition system is to identify photographed
buildings or objects in query photos and to provide the user with relevant
information on them. With their increasing coverage of the world's landmark
buildings and objects, Internet photo collections are now being used as a
source for building such systems in a fully automatic fashion. This process
typically consists of three steps: clustering large amounts of images by the
objects they depict; determining object names from user-provided tags; and
building a robust, compact, and efficient recognition index. To this date,
however, there is little empirical information on how well current approaches
for those steps perform in a large-scale open-set mining and recognition task.
Furthermore, there is little empirical information on how recognition
performance varies for different types of landmark objects and where there is
still potential for improvement. With this paper, we intend to fill these gaps.
Using a dataset of 500k images from Paris, we analyze each component of the
landmark recognition pipeline in order to answer the following questions: How
many and what kinds of objects can be discovered automatically? How can we best
use the resulting image clusters to recognize the object in a query? How can
the object be efficiently represented in memory for recognition? How reliably
can semantic information be extracted? And finally: What are the limiting
factors in the resulting pipeline from query to semantics? We evaluate how
different choices of methods and parameters for the individual pipeline steps
affect overall system performance and examine their effects for different query
categories such as buildings, paintings or sculptures
Visual link retrieval and knowledge discovery in painting datasets
Visual arts have invaluable importance for the cultural, historic and
economic growth of our societies. One of the building blocks of most analysis
in visual arts is to find similarities among paintings of different artists and
painting schools. To help art historians better understand visual arts, the
present paper presents a framework for visual link retrieval and knowledge
discovery in digital painting datasets. The proposed framework is based on a
deep convolutional neural network to perform feature extraction and on a fully
unsupervised nearest neighbor approach to retrieve visual links among digitized
paintings. The fully unsupervised strategy makes attractive the proposed method
especially in those cases where metadata are either scarce or unavailable or
difficult to collect. In addition, the proposed framework includes a graph
analysis that makes it possible to study influences among artists, thus
providing historical knowledge discovery.Comment: submitted to Multimedia Tools and Application
Deep learning approaches to pattern extraction and recognition in paintings and drawings: an overview
This paper provides an overview of some of the most relevant deep learning approaches to pattern extraction and recognition in visual arts, particularly painting and drawing. Recent advances in deep learning and computer vision, coupled with the growing availability of large digitized visual art collections, have opened new opportunities for computer science researchers to assist the art community with automatic tools to analyse and further understand visual arts. Among other benefits, a deeper understanding of visual arts has the potential to make them more accessible to a wider population, ultimately supporting the spread of culture
Visual link retrieval and knowledge discovery in painting datasets
Visual arts are of inestimable importance for the cultural, historic and economic growth of our society. One of the building blocks of most analysis in visual arts is to find similarity relationships among paintings of different artists and painting schools. To help art historians better understand visual arts, this paper presents a framework for visual link retrieval and knowledge discovery in digital painting datasets. Visual link retrieval is accomplished by using a deep convolutional neural network to perform feature extraction and a fully unsupervised nearest neighbor mechanism to retrieve links among digitized paintings. Historical knowledge discovery is achieved by performing a graph analysis that makes it possible to study influences among artists. An experimental evaluation on a database collecting paintings by very popular artists shows the effectiveness of the method. The unsupervised strategy makes the method interesting especially in cases where metadata are scarce, unavailable or difficult to collect
Deep convolutional embedding for digitized painting clustering
Clustering artworks is difficult because of several reasons. On one hand,
recognizing meaningful patterns in accordance with domain knowledge and visual
perception is extremely hard. On the other hand, the application of traditional
clustering and feature reduction techniques to the highly dimensional pixel
space can be ineffective. To address these issues, we propose a deep
convolutional embedding model for clustering digital paintings, in which the
task of mapping the input raw data to an abstract, latent space is optimized
jointly with the task of finding a set of cluster centroids in this latent
feature space. Quantitative and qualitative experimental results show the
effectiveness of the proposed method. The model is also able to outperform
other state-of-the-art deep clustering approaches to the same problem. The
proposed method may be beneficial to several art-related tasks, particularly
visual link retrieval and historical knowledge discovery in painting datasets
Linking Art through Human Poses
We address the discovery of composition transfer in artworks based on their
visual content. Automated analysis of large art collections, which are growing
as a result of art digitization among museums and galleries, is an important
tool for art history and assists cultural heritage preservation. Modern image
retrieval systems offer good performance on visually similar artworks, but fail
in the cases of more abstract composition transfer. The proposed approach links
artworks through a pose similarity of human figures depicted in images. Human
figures are the subject of a large fraction of visual art from middle ages to
modernity and their distinctive poses were often a source of inspiration among
artists. The method consists of two steps -- fast pose matching and robust
spatial verification. We experimentally show that explicit human pose matching
is superior to standard content-based image retrieval methods on a manually
annotated art composition transfer dataset
Large-scale interactive retrieval in art collections using multi-style feature aggregation
Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are inadequate to handle this amount of data, and it requires appropriate computer-based methods to analyze them. This article presents a visual search algorithm and user interface to support art historians to find objects and motifs in extensive datasets. Artistic image collections are subject to significant domain shifts induced by large variations in styles, artistic media, and materials. This poses new challenges to most computer vision models which are trained on photographs. To alleviate this problem, we introduce a multi-style feature aggregation that projects images into the same distribution, leading to more accurate and style-invariant search results. Our retrieval system is based on a voting procedure combined with fast nearest-neighbor search and enables finding and localizing motifs within an extensive image collection in seconds. The presented approach significantly improves the state-of-the-art in terms of accuracy and search time on various datasets and applies to large and inhomogeneous collections. In addition to the search algorithm, we introduce a user interface that allows art historians to apply our algorithm in practice. The interface enables users to search for single regions, multiple regions regarding different connection types and holds an interactive feedback system to improve retrieval results further. With our methodological contribution and easy-to-use user interface, this work manifests further progress towards a computer-based analysis of visual art
Semantic enrichment for enhancing LAM data and supporting digital humanities. Review article
With the rapid development of the digital humanities (DH) field, demands for historical and cultural heritage data have generated deep interest in the data provided by libraries, archives, and museums (LAMs). In order to enhance LAM data’s quality and discoverability while enabling a self-sustaining ecosystem, “semantic enrichment” becomes a strategy increasingly used by LAMs during recent years. This article introduces a number of semantic enrichment methods and efforts that can be applied to LAM data at various levels, aiming to support deeper and wider exploration and use of LAM data in DH research. The real cases, research projects, experiments, and pilot studies shared in this article demonstrate endless potential for LAM data, whether they are structured, semi-structured, or unstructured, regardless of what types of original artifacts carry the data. Following their roadmaps would encourage more effective initiatives and strengthen this effort to maximize LAM data’s discoverability, use- and reuse-ability, and their value in the mainstream of DH and Semantic Web
- …