36 research outputs found
A Unifying View of Multiple Kernel Learning
Recent research on multiple kernel learning has lead to a number of
approaches for combining kernels in regularized risk minimization. The proposed
approaches include different formulations of objectives and varying
regularization strategies. In this paper we present a unifying general
optimization criterion for multiple kernel learning and show how existing
formulations are subsumed as special cases. We also derive the criterion's dual
representation, which is suitable for general smooth optimization algorithms.
Finally, we evaluate multiple kernel learning in this framework analytically
using a Rademacher complexity bound on the generalization error and empirically
in a set of experiments
The Feature Importance Ranking Measure
Most accurate predictions are typically obtained by learning machines with
complex feature spaces (as e.g. induced by kernels). Unfortunately, such
decision rules are hardly accessible to humans and cannot easily be used to
gain insights about the application domain. Therefore, one often resorts to
linear models in combination with variable selection, thereby sacrificing some
predictive power for presumptive interpretability. Here, we introduce the
Feature Importance Ranking Measure (FIRM), which by retrospective analysis of
arbitrary learning machines allows to achieve both excellent predictive
performance and superior interpretation. In contrast to standard raw feature
weighting, FIRM takes the underlying correlation structure of the features into
account. Thereby, it is able to discover the most relevant features, even if
their appearance in the training data is entirely prevented by noise. The
desirable properties of FIRM are investigated analytically and illustrated in
simulations.Comment: 15 pages, 3 figures. to appear in the Proceedings of the European
Conference on Machine Learning and Principles and Practice of Knowledge
Discovery in Databases (ECML/PKDD), 200
Flexible nonparametric kernel learning with different loss functions
Side information is highly useful in the learning of a nonparametric kernel matrix. However, this often leads to an expensive semidefinite program (SDP). In recent years, a number of dedicated solvers have been proposed. Though much better than off-the-shelf SDP solvers, they still cannot scale to large data sets. In this paper, we propose a novel solver based on the alternating direction method of multipliers (ADMM). The key idea is to use a low-rank decomposition of the kernel matrix Z = XTY , with the constraint that X = Y . The resultant optimization problem, though non-convex, has favorable convergence properties and can be efficiently solved without requiring eigen-decomposition in each iteration. Experimental results on a number of real-world data sets demonstrate that the proposed method is as accurate as directly solving the SDP, but can be one to two orders of magnitude faster. © Springer-Verlag 2013
Semantic and spatial content fusion for scene recognition
In the field of scene recognition, it is usually insufficient to use only one visual feature regardless of how discriminative the feature is. Therefore, the spatial location and semantic relationships of local features need to be captured together with the scene contextual information. In this paper we proposed a novel framework to project image contextual feature space with semantic space of local features into a map function. This embedding is performed based on a subset of training images denoted as an exemplar-set. This exemplar-set is composed of images that describe better the scene category\u27s attributes than the other images. The proposed framework learns a weighted combination of local semantic topics as well as global and spatial information, where the weights represent the features\u27 contributions in each scene category. An empirical study was performed on two of the most challenging scene datasets 15-Scene categories and 67-Indoor Scenes and the promising results of 89.47 and 45.0 were achieved respectively
Learning General Gaussian Kernel Hyperparameters for SVR
International audienceWe propose a new method for general gaussian kernel hyperparameters optimization for support vector regression. The hyperparameters are constrained to lie on a differentiable manifold. The proposed optimization technique is based on a gradient-like descent algorithm adapted to the geometrical structure of the manifold of symmetric positive-definite matrices. We compare the performance of our approach with the classical support vector regression on real world data sets. Experiments demonstrate that the optimization improves prediction accuracy and reduces the number of support vectors