19,361 research outputs found
Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classification
This paper addresses the task of zero-shot image classification. The key
contribution of the proposed approach is to control the semantic embedding of
images -- one of the main ingredients of zero-shot learning -- by formulating
it as a metric learning problem. The optimized empirical criterion associates
two types of sub-task constraints: metric discriminating capacity and accurate
attribute prediction. This results in a novel expression of zero-shot learning
not requiring the notion of class in the training phase: only pairs of
image/attributes, augmented with a consistency indicator, are given as ground
truth. At test time, the learned model can predict the consistency of a test
image with a given set of attributes , allowing flexible ways to produce
recognition inferences. Despite its simplicity, the proposed approach gives
state-of-the-art results on four challenging datasets used for zero-shot
recognition evaluation.Comment: in ECCV 2016, Oct 2016, amsterdam, Netherlands. 201
Learning Cross-Modal Deep Embeddings for Multi-Object Image Retrieval using Text and Sketch
In this work we introduce a cross modal image retrieval system that allows
both text and sketch as input modalities for the query. A cross-modal deep
network architecture is formulated to jointly model the sketch and text input
modalities as well as the the image output modality, learning a common
embedding between text and images and between sketches and images. In addition,
an attention model is used to selectively focus the attention on the different
objects of the image, allowing for retrieval with multiple objects in the
query. Experiments show that the proposed method performs the best in both
single and multiple object image retrieval in standard datasets.Comment: Accepted at ICPR 201
Correcting Knowledge Base Assertions
The usefulness and usability of knowledge bases (KBs) is often limited by quality issues. One common issue is the presence of erroneous assertions, often caused by lexical or semantic confusion. We study the problem of correcting such assertions, and present a general correction framework which combines lexical matching, semantic embedding, soft constraint mining and semantic consistency checking. The framework is evaluated using DBpedia and an enterprise medical KB
Zero Shot Recognition with Unreliable Attributes
In principle, zero-shot learning makes it possible to train a recognition
model simply by specifying the category's attributes. For example, with
classifiers for generic attributes like \emph{striped} and \emph{four-legged},
one can construct a classifier for the zebra category by enumerating which
properties it possesses---even without providing zebra training images. In
practice, however, the standard zero-shot paradigm suffers because attribute
predictions in novel images are hard to get right. We propose a novel random
forest approach to train zero-shot models that explicitly accounts for the
unreliability of attribute predictions. By leveraging statistics about each
attribute's error tendencies, our method obtains more robust discriminative
models for the unseen classes. We further devise extensions to handle the
few-shot scenario and unreliable attribute descriptions. On three datasets, we
demonstrate the benefit for visual category learning with zero or few training
examples, a critical domain for rare categories or categories defined on the
fly.Comment: NIPS 201
- …