2 research outputs found
On Kernelization of Supervised Mahalanobis Distance Learners
This paper focuses on the problem of kernelizing an existing supervised
Mahalanobis distance learner. The following features are included in the paper.
Firstly, three popular learners, namely, "neighborhood component analysis",
"large margin nearest neighbors" and "discriminant neighborhood embedding",
which do not have kernel versions are kernelized in order to improve their
classification performances. Secondly, an alternative kernelization framework
called "KPCA trick" is presented. Implementing a learner in the new framework
gains several advantages over the standard framework, e.g. no mathematical
formulas and no reprogramming are required for a kernel implementation, the
framework avoids troublesome problems such as singularity, etc. Thirdly, while
the truths of representer theorems are just assumptions in previous papers
related to ours, here, representer theorems are formally proven. The proofs
validate both the kernel trick and the KPCA trick in the context of Mahalanobis
distance learning. Fourthly, unlike previous works which always apply brute
force methods to select a kernel, we investigate two approaches which can be
efficiently adopted to construct an appropriate kernel for a given dataset.
Finally, numerical results on various real-world datasets are presented.Comment: 23 pages, 5 figures. There is a seriously wrong formula in derivation
of a gradient formula of the "kernel NCA" in the two previous versions. In
this new version, a new theoretical result is provided to properly account
kernel NC