2 research outputs found
Pattern Search Multidimensional Scaling
We present a novel view of nonlinear manifold learning using derivative-free
optimization techniques. Specifically, we propose an extension of the classical
multi-dimensional scaling (MDS) method, where instead of performing gradient
descent, we sample and evaluate possible "moves" in a sphere of fixed radius
for each point in the embedded space. A fixed-point convergence guarantee can
be shown by formulating the proposed algorithm as an instance of General
Pattern Search (GPS) framework. Evaluation on both clean and noisy synthetic
datasets shows that pattern search MDS can accurately infer the intrinsic
geometry of manifolds embedded in high-dimensional spaces. Additionally,
experiments on real data, even under noisy conditions, demonstrate that the
proposed pattern search MDS yields state-of-the-art results.Comment: 36 pages, Under review for JML
Bootstrapped Coordinate Search for Multidimensional Scaling
In this work, a unified framework for gradient-free Multidimensional Scaling
(MDS) based on Coordinate Search (CS) is proposed. This family of algorithms is
an instance of General Pattern Search (GPS) methods which avoid the explicit
computation of derivatives but instead evaluate the objective function while
searching on coordinate steps of the embedding space. The backbone element of
CSMDS framework is the corresponding probability matrix that correspond to how
likely is each corresponding coordinate to be evaluated. We propose a
Bootstrapped instance of CSMDS (BS CSMDS) which enhances the probability of the
direction that decreases the most the objective function while also reducing
the corresponding probability of all the other coordinates. BS CSMDS manages to
avoid unnecessary function evaluations and result to significant speedup over
other CSMDS alternatives while also obtaining the same error rate. Experiments
on both synthetic and real data reveal that BS CSMDS performs consistently
better than other CSMDS alternatives under various experimental setups