93 research outputs found
Stochastic Non-convex Ordinal Embedding with Stabilized Barzilai-Borwein Step Size
Learning representation from relative similarity comparisons, often called
ordinal embedding, gains rising attention in recent years. Most of the existing
methods are batch methods designed mainly based on the convex optimization,
say, the projected gradient descent method. However, they are generally
time-consuming due to that the singular value decomposition (SVD) is commonly
adopted during the update, especially when the data size is very large. To
overcome this challenge, we propose a stochastic algorithm called SVRG-SBB,
which has the following features: (a) SVD-free via dropping convexity, with
good scalability by the use of stochastic algorithm, i.e., stochastic variance
reduced gradient (SVRG), and (b) adaptive step size choice via introducing a
new stabilized Barzilai-Borwein (SBB) method as the original version for convex
problems might fail for the considered stochastic \textit{non-convex}
optimization problem. Moreover, we show that the proposed algorithm converges
to a stationary point at a rate in our setting,
where is the number of total iterations. Numerous simulations and
real-world data experiments are conducted to show the effectiveness of the
proposed algorithm via comparing with the state-of-the-art methods,
particularly, much lower computational cost with good prediction performance.Comment: 11 pages, 3 figures, 2 tables, accepted by AAAI201
Recommended from our members
Hyperbolic Ordinal Embedding
Given ordinal relations such as the object i is more similar to j than k is to l, ordinal embedding is to embed these objects into a low-dimensional space with all ordinal constraints
preserved. Although existing approaches have preserved ordinal relations in Euclidean
space, whether Euclidean space is compatible with true data structure is largely ignored,
although it is essential to effective embedding. Since real data often exhibit hierarchical
structure, it is hard for Euclidean space approaches to achieve effective embeddings in low
dimensionality, which incurs high computational complexity or overfitting. In this paper we
propose a novel hyperbolic ordinal embedding (HOE) method to embed objects in hyperbolic space. Due to the hierarchy-friendly property of hyperbolic space, HOE can effectively
capture the hierarchy to achieve embeddings in an extremely low-dimensional space. We
have not only theoretically proved the superiority of hyperbolic space and the limitations
of Euclidean space for embedding hierarchical data, but also experimentally demonstrated
that HOE significantly outperforms Euclidean-based methods
A Similarity Measure for Material Appearance
We present a model to measure the similarity in appearance between different
materials, which correlates with human similarity judgments. We first create a
database of 9,000 rendered images depicting objects with varying materials,
shape and illumination. We then gather data on perceived similarity from
crowdsourced experiments; our analysis of over 114,840 answers suggests that
indeed a shared perception of appearance similarity exists. We feed this data
to a deep learning architecture with a novel loss function, which learns a
feature space for materials that correlates with such perceived appearance
similarity. Our evaluation shows that our model outperforms existing metrics.
Last, we demonstrate several applications enabled by our metric, including
appearance-based search for material suggestions, database visualization,
clustering and summarization, and gamut mapping.Comment: 12 pages, 17 figure
Relative Comparison Kernel Learning with Auxiliary Kernels
In this work we consider the problem of learning a positive semidefinite
kernel matrix from relative comparisons of the form: "object A is more similar
to object B than it is to C", where comparisons are given by humans. Existing
solutions to this problem assume many comparisons are provided to learn a high
quality kernel. However, this can be considered unrealistic for many real-world
tasks since relative assessments require human input, which is often costly or
difficult to obtain. Because of this, only a limited number of these
comparisons may be provided. In this work, we explore methods for aiding the
process of learning a kernel with the help of auxiliary kernels built from more
easily extractable information regarding the relationships among objects. We
propose a new kernel learning approach in which the target kernel is defined as
a conic combination of auxiliary kernels and a kernel whose elements are
learned directly. We formulate a convex optimization to solve for this target
kernel that adds only minor overhead to methods that use no auxiliary
information. Empirical results show that in the presence of few training
relative comparisons, our method can learn kernels that generalize to more
out-of-sample comparisons than methods that do not utilize auxiliary
information, as well as similar methods that learn metrics over objects
- …