74,748 research outputs found
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
We propose an efficient nonparametric strategy for learning a message
operator in expectation propagation (EP), which takes as input the set of
incoming messages to a factor node, and produces an outgoing message as output.
This learned operator replaces the multivariate integral required in classical
EP, which may not have an analytic expression. We use kernel-based regression,
which is trained on a set of probability distributions representing the
incoming messages, and the associated outgoing messages. The kernel approach
has two main advantages: first, it is fast, as it is implemented using a novel
two-layer random feature representation of the input message distributions;
second, it has principled uncertainty estimates, and can be cheaply updated
online, meaning it can request and incorporate new training data when it
encounters inputs on which it is uncertain. In experiments, our approach is
able to solve learning problems where a single message operator is required for
multiple, substantially different data sets (logistic regression for a variety
of classification problems), where it is essential to accurately assess
uncertainty and to efficiently and robustly update the message operator.Comment: accepted to UAI 2015. Correct typos. Add more content to the
appendix. Main results unchange
Online Learning with Multiple Operator-valued Kernels
We consider the problem of learning a vector-valued function f in an online
learning setting. The function f is assumed to lie in a reproducing Hilbert
space of operator-valued kernels. We describe two online algorithms for
learning f while taking into account the output structure. A first contribution
is an algorithm, ONORMA, that extends the standard kernel-based online learning
algorithm NORMA from scalar-valued to operator-valued setting. We report a
cumulative error bound that holds both for classification and regression. We
then define a second algorithm, MONORMA, which addresses the limitation of
pre-defining the output structure in ONORMA by learning sequentially a linear
combination of operator-valued kernels. Our experiments show that the proposed
algorithms achieve good performance results with low computational cost
Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics
Kernel-based methods exhibit well-documented performance in various nonlinear
learning tasks. Most of them rely on a preselected kernel, whose prudent choice
presumes task-specific prior information. Especially when the latter is not
available, multi-kernel learning has gained popularity thanks to its
flexibility in choosing kernels from a prescribed kernel dictionary. Leveraging
the random feature approximation and its recent orthogonality-promoting
variant, the present contribution develops a scalable multi-kernel learning
scheme (termed Raker) to obtain the sought nonlinear learning function `on the
fly,' first for static environments. To further boost performance in dynamic
environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is
developed. AdaRaker accounts not only for data-driven learning of kernel
combination, but also for the unknown dynamics. Performance is analyzed in
terms of both static and dynamic regrets. AdaRaker is uniquely capable of
tracking nonlinear learning functions in environments with unknown dynamics,
and with with analytic performance guarantees. Tests with synthetic and real
datasets are carried out to showcase the effectiveness of the novel algorithms.Comment: 36 page
Large scale online multiple kernel regression with application to time-series prediction
National Research Foundation (NRF) Singapor
Support Vector Regression Based S-transform for Prediction of Single and Multiple Power Quality Disturbances
This paper presents a novel approach using Support Vector Regression (SVR) based
S-transform to predict the classes of single and multiple power quality disturbances in a
three-phase industrial power system. Most of the power quality disturbances recorded in an
industrial power system are non-stationary and comprise of multiple power quality
disturbances that coexist together for only a short duration in time due to the contribution
of the network impedances and types of customers’ connected loads. The ability to detect
and predict all the types of power quality disturbances encrypted in a voltage signal is vital
in the analyses on the causes of the power quality disturbances and in the identification of
incipient fault in the networks. In this paper, the performances of two types of SVR based
S-transform, the non-linear radial basis function (RBF) SVR based S-transform and the
multilayer perceptron (MLP) SVR based S-transform, were compared for their abilities in
making prediction for the classes of single and multiple power quality disturbances. The
results for the analyses of 651 numbers of single and multiple voltage disturbances gave
prediction accuracies of 86.1% (MLP SVR) and 93.9% (RBF SVR) respectively.
Keywords: Power Quality, Power Quality Prediction, S-transform, SVM, SV
- …