35 research outputs found
Multimodal news article analysis
The intersection of Computer Vision and Natural Language Processing has been a hot topic of research in recent years, with results that were unthinkable only a few years ago. In view of this progress, we want to highlight online news articles as a potential next step for this area of research. The rich interrelations of text, tags, images or videos, as well as a vast corpus of general knowledge are an exciting benchmark for high-capacity models such as the deep neural networks. In this paper we present a series of tasks and baseline approaches to leverage corpus such as the BreakingNews dataset.Peer ReviewedPostprint (author's final draft
Attribute-Graph: A Graph based approach to Image Ranking
We propose a novel image representation, termed Attribute-Graph, to rank
images by their semantic similarity to a given query image. An Attribute-Graph
is an undirected fully connected graph, incorporating both local and global
image characteristics. The graph nodes characterise objects as well as the
overall scene context using mid-level semantic attributes, while the edges
capture the object topology. We demonstrate the effectiveness of
Attribute-Graphs by applying them to the problem of image ranking. We benchmark
the performance of our algorithm on the 'rPascal' and 'rImageNet' datasets,
which we have created in order to evaluate the ranking performance on complex
queries containing multiple objects. Our experimental evaluation shows that
modelling images as Attribute-Graphs results in improved ranking performance
over existing techniques.Comment: In IEEE International Conference on Computer Vision (ICCV) 201
Deep Fishing: Gradient Features from Deep Nets
Convolutional Networks (ConvNets) have recently improved image recognition
performance thanks to end-to-end learning of deep feed-forward models from raw
pixels. Deep learning is a marked departure from the previous state of the art,
the Fisher Vector (FV), which relied on gradient-based encoding of local
hand-crafted features. In this paper, we discuss a novel connection between
these two approaches. First, we show that one can derive gradient
representations from ConvNets in a similar fashion to the FV. Second, we show
that this gradient representation actually corresponds to a structured matrix
that allows for efficient similarity computation. We experimentally study the
benefits of transferring this representation over the outputs of ConvNet
layers, and find consistent improvements on the Pascal VOC 2007 and 2012
datasets.Comment: To appear at BMVC 201
Exploiting Local Features from Deep Networks for Image Retrieval
Deep convolutional neural networks have been successfully applied to image
classification tasks. When these same networks have been applied to image
retrieval, the assumption has been made that the last layers would give the
best performance, as they do in classification. We show that for instance-level
image retrieval, lower layers often perform better than the last layers in
convolutional neural networks. We present an approach for extracting
convolutional features from different layers of the networks, and adopt VLAD
encoding to encode features into a single vector for each image. We investigate
the effect of different layers and scales of input images on the performance of
convolutional features using the recent deep networks OxfordNet and GoogLeNet.
Experiments demonstrate that intermediate layers or higher layers with finer
scales produce better results for image retrieval, compared to the last layer.
When using compressed 128-D VLAD descriptors, our method obtains
state-of-the-art results and outperforms other VLAD and CNN based approaches on
two out of three test datasets. Our work provides guidance for transferring
deep networks trained on image classification to image retrieval tasks.Comment: CVPR DeepVision Workshop 201
Evaluation of Output Embeddings for Fine-Grained Image Classification
Image classification has advanced significantly in recent years with the
availability of large-scale image sets. However, fine-grained classification
remains a major challenge due to the annotation cost of large numbers of
fine-grained categories. This project shows that compelling classification
performance can be achieved on such categories even without labeled training
data. Given image and class embeddings, we learn a compatibility function such
that matching embeddings are assigned a higher score than mismatching ones;
zero-shot classification of an image proceeds by finding the label yielding the
highest joint compatibility score. We use state-of-the-art image features and
focus on different supervised attributes and unsupervised output embeddings
either derived from hierarchies or learned from unlabeled text corpora. We
establish a substantially improved state-of-the-art on the Animals with
Attributes and Caltech-UCSD Birds datasets. Most encouragingly, we demonstrate
that purely unsupervised output embeddings (learned from Wikipedia and improved
with fine-grained text) achieve compelling results, even outperforming the
previous supervised state-of-the-art. By combining different output embeddings,
we further improve results.Comment: @inproceedings {ARWLS15, title = {Evaluation of Output Embeddings for
Fine-Grained Image Classification}, booktitle = {IEEE Computer Vision and
Pattern Recognition}, year = {2015}, author = {Zeynep Akata and Scott Reed
and Daniel Walter and Honglak Lee and Bernt Schiele}
An ensemble learning approach for the classification of remote sensing scenes based on covariance pooling of CNN features
International audienceThis paper aims at presenting a novel ensemble learning approach based on the concept of covariance pooling of CNN features issued from a pretrained model. Starting from a supervised classification algorithm, named multilayer stacked covariance pooling (MSCP), which exploits simultaneously second order statistics and deep learning features, we propose an alternative strategy which employs an ensemble learning approach among the stacked convolutional feature maps. The aggregation of multiple learning algorithm decisions, produced by different stacked subsets, permits to obtain a better predictive classification performance. An application for the classification of large scale remote sensing images is next proposed. The experimental results, conducted on two challenging datasets, namely UC Merced and AID datasets, improve the classification accuracy while maintaining a low computation time. This confirms, besides the interest of exploiting second order statistics, the benefit of adopting an ensemble learning approach
Large- Scale Content Based Face Image Retrieval using Attribute Enhanced Sparse Codewords.
Content based image retrieval (CBIR) have turn into majority dynamic exploration regions within previous couple of existence. Numerous index strategies be in light of worldwide component circulations. Be that as it may, these worldwide circulations have restricted segregating force since they are not able to catch nearby picture data. Photographs with individuals are the foremost attention of users. Consequently with exponentially increasing pictures, huge size contented base features representation recovery is a facilitating knowledge in favor of various developing applications. The main objective is to apply automatically spotted human characteristics that comprise semantic cue of facade pictures toward increase gratified base facade recovery through creating semantic codeword pro effectual huge size countenance recovery. With leveraging person characteristics into scalable as well as methodical structure, suggest and offer two orthogonal systems named attribute improved meager code and attribute entrenched upturned index toward develop facade recovery. We compare proposed method with other three methods namely LBP, ATTR and SC methods. The results illustrate that the proposed methods can attain qualified enhancement in Mean Average Precision (MAP) associated to the existing methods.
DOI: 10.17762/ijritcc2321-8169.15084