100,247 research outputs found
Fast Bounded Online Gradient Descent Algorithms for Scalable Kernel-Based Online Learning
Kernel-based online learning has often shown state-of-the-art performance for
many online learning tasks. It, however, suffers from a major shortcoming, that
is, the unbounded number of support vectors, making it non-scalable and
unsuitable for applications with large-scale datasets. In this work, we study
the problem of bounded kernel-based online learning that aims to constrain the
number of support vectors by a predefined budget. Although several algorithms
have been proposed in literature, they are neither computationally efficient
due to their intensive budget maintenance strategy nor effective due to the use
of simple Perceptron algorithm. To overcome these limitations, we propose a
framework for bounded kernel-based online learning based on an online gradient
descent approach. We propose two efficient algorithms of bounded online
gradient descent (BOGD) for scalable kernel-based online learning: (i) BOGD by
maintaining support vectors using uniform sampling, and (ii) BOGD++ by
maintaining support vectors using non-uniform sampling. We present theoretical
analysis of regret bound for both algorithms, and found promising empirical
performance in terms of both efficacy and efficiency by comparing them to
several well-known algorithms for bounded kernel-based online learning on
large-scale datasets.Comment: ICML201
Semi-supervised Online Multiple Kernel Learning Algorithm for Big Data
In order to improve the performance of machine learning in big data, online multiple kernel learning algorithms are proposed in this paper. First, a supervised online multiple kernel learning algorithm for big data (SOMK_bd) is proposed to reduce the computational workload during kernel modification. In SOMK_bd, the traditional kernel learning algorithm is improved and kernel integration is only carried out in the constructed kernel subset. Next, an unsupervised online multiple kernel learning algorithm for big data (UOMK_bd) is proposed. In UOMK_bd, the traditional kernel learning algorithm is improved to adapt to the online environment and data replacement strategy is used to modify the kernel function in unsupervised manner. Then, a semi-supervised online multiple kernel learning algorithm for big data (SSOMK_bd) is proposed. Based on incremental learning, SSOMK_bd makes full use of the abundant information of large scale incomplete labeled data, and uses SOMK_bd and UOMK_bd to update the current reading data. Finally, experiments are conducted on UCI data set and the results show that the proposed algorithms are effective
- …