12 research outputs found
Online Learning with Multiple Operator-valued Kernels
We consider the problem of learning a vector-valued function f in an online
learning setting. The function f is assumed to lie in a reproducing Hilbert
space of operator-valued kernels. We describe two online algorithms for
learning f while taking into account the output structure. A first contribution
is an algorithm, ONORMA, that extends the standard kernel-based online learning
algorithm NORMA from scalar-valued to operator-valued setting. We report a
cumulative error bound that holds both for classification and regression. We
then define a second algorithm, MONORMA, which addresses the limitation of
pre-defining the output structure in ONORMA by learning sequentially a linear
combination of operator-valued kernels. Our experiments show that the proposed
algorithms achieve good performance results with low computational cost
Learning Output Kernels for Multi-Task Problems
Simultaneously solving multiple related learning tasks is beneficial under a
variety of circumstances, but the prior knowledge necessary to correctly model
task relationships is rarely available in practice. In this paper, we develop a
novel kernel-based multi-task learning technique that automatically reveals
structural inter-task relationships. Building over the framework of output
kernel learning (OKL), we introduce a method that jointly learns multiple
functions and a low-rank multi-task kernel by solving a non-convex
regularization problem. Optimization is carried out via a block coordinate
descent strategy, where each subproblem is solved using suitable conjugate
gradient (CG) type iterative methods for linear operator equations. The
effectiveness of the proposed approach is demonstrated on pharmacological and
collaborative filtering data
Gaussian process regression for forecasting battery state of health
Accurately predicting the future capacity and remaining useful life of
batteries is necessary to ensure reliable system operation and to minimise
maintenance costs. The complex nature of battery degradation has meant that
mechanistic modelling of capacity fade has thus far remained intractable;
however, with the advent of cloud-connected devices, data from cells in various
applications is becoming increasingly available, and the feasibility of
data-driven methods for battery prognostics is increasing. Here we propose
Gaussian process (GP) regression for forecasting battery state of health, and
highlight various advantages of GPs over other data-driven and mechanistic
approaches. GPs are a type of Bayesian non-parametric method, and hence can
model complex systems whilst handling uncertainty in a principled manner. Prior
information can be exploited by GPs in a variety of ways: explicit mean
functions can be used if the functional form of the underlying degradation
model is available, and multiple-output GPs can effectively exploit
correlations between data from different cells. We demonstrate the predictive
capability of GPs for short-term and long-term (remaining useful life)
forecasting on a selection of capacity vs. cycle datasets from lithium-ion
cells.Comment: 13 pages, 7 figures, published in the Journal of Power Sources, 201
Learning in the Wild with Incremental Skeptical Gaussian Processes
The ability to learn from human supervision is fundamental for personal
assistants and other interactive applications of AI. Two central challenges for
deploying interactive learners in the wild are the unreliable nature of the
supervision and the varying complexity of the prediction task. We address a
simple but representative setting, incremental classification in the wild,
where the supervision is noisy and the number of classes grows over time. In
order to tackle this task, we propose a redesign of skeptical learning centered
around Gaussian Processes (GPs). Skeptical learning is a recent interactive
strategy in which, if the machine is sufficiently confident that an example is
mislabeled, it asks the annotator to reconsider her feedback. In many cases,
this is often enough to obtain clean supervision. Our redesign, dubbed ISGP,
leverages the uncertainty estimates supplied by GPs to better allocate labeling
and contradiction queries, especially in the presence of noise. Our experiments
on synthetic and real-world data show that, as a result, while the original
formulation of skeptical learning produces over-confident models that can fail
completely in the wild, ISGP works well at varying levels of noise and as new
classes are observed.Comment: 7 pages, 3 figures, code:
https://gitlab.com/abonte/incremental-skeptical-g
A multivariate timeseries modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data
The ability to determine patient acuity (or severity of illness) has immediate practical use for clinicians. We evaluate the use of multivariate timeseries modeling with the multi-task Gaussian process (GP) models using noisy, incomplete, sparse, heterogeneous and unevenly-sampled clinical data, including both physiological signals and clinical notes. The learned multi-task GP (MTGP) hyperparameters are then used to assess and forecast patient acuity. Experiments were conducted with two real clinical data sets acquired from ICU patients: firstly, estimating cerebrovascular pressure reactivity, an important indicator of secondary damage for traumatic brain injury patients, by learning the interactions between intracranial pressure and mean arterial blood pressure signals, and secondly, mortality prediction using clinical progress notes. In both cases, MTGPs provided improved results: an MTGP model provided better results than single-task GP models for signal interpolation and forecasting (0.91 vs 0.69 RMSE), and the use of MTGP hyperparameters obtained improved results when used as additional classification features (0.812 vs 0.788 AUC).Intel Science and Technology Center for Big DataNational Institutes of Health. (U.S.). National Library of Medicine (Biomedical Informatics Research Training Grant NIH/NLM 2T15 LM007092-22)National Institute of Biomedical Imaging and Bioengineering (U.S.) (R01 Grant EB001659)Singapore. Agency for Science, Technology and Research (Graduate Scholarship
Nonparametric Bayesian Mixed-effect Model: a Sparse Gaussian Process Approach
Multi-task learning models using Gaussian processes (GP) have been developed
and successfully applied in various applications. The main difficulty with this
approach is the computational cost of inference using the union of examples
from all tasks. Therefore sparse solutions, that avoid using the entire data
directly and instead use a set of informative "representatives" are desirable.
The paper investigates this problem for the grouped mixed-effect GP model where
each individual response is given by a fixed-effect, taken from one of a set of
unknown groups, plus a random individual effect function that captures
variations among individuals. Such models have been widely used in previous
work but no sparse solutions have been developed. The paper presents the first
sparse solution for such problems, showing how the sparse approximation can be
obtained by maximizing a variational lower bound on the marginal likelihood,
generalizing ideas from single-task Gaussian processes to handle the
mixed-effect model as well as grouping. Experiments using artificial and real
data validate the approach showing that it can recover the performance of
inference with the full sample, that it outperforms baseline methods, and that
it outperforms state of the art sparse solutions for other multi-task GP
formulations.Comment: Preliminary version appeared in ECML201
A multivariate timeseries modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data
The ability to determine patient acuity (or severity of illness) has immediate practical use for clinicians. We evaluate the use of multivariate timeseries modeling with the multi-task Gaussian process (GP) models using noisy, incomplete, sparse, heterogeneous and unevenly-sampled clinical data, including both physiological signals and clinical notes. The learned multi-task GP (MTGP) hyperparameters are then used to assess and forecast patient acuity. Experiments were conducted with two real clinical data sets acquired from ICU patients: firstly, estimating cerebrovascular pressure reactivity, an important indicator of secondary damage for traumatic brain injury patients, by learning the interactions between intracranial pressure and mean arterial blood pressure signals, and secondly, mortality prediction using clinical progress notes. In both cases, MTGPs provided improved results: an MTGP model provided better results than single-task GP models for signal interpolation and forecasting (0.91 vs 0.69 RMSE), and the use of MTGP hyperparameters obtained improved results when used as additional classification features (0.812 vs 0.788 AUC).Intel Science and Technology Center for Big DataNational Institutes of Health. (U.S.). National Library of Medicine (Biomedical Informatics Research Training Grant NIH/NLM 2T15 LM007092-22)National Institute of Biomedical Imaging and Bioengineering (U.S.) (R01 Grant EB001659)Singapore. Agency for Science, Technology and Research (Graduate Scholarship