142 research outputs found
Multitask Online Mirror Descent
We introduce and analyze MT-OMD, a multitask generalization of Online Mirror
Descent (OMD) which operates by sharing updates between tasks. We prove that
the regret of MT-OMD is of order , where
is the task variance according to the geometry induced by the
regularizer, is the number of tasks, and is the time horizon. Whenever
tasks are similar, that is , our method improves upon the
bound obtained by running independent OMDs on each task. We further
provide a matching lower bound, and show that our multitask extensions of
Online Gradient Descent and Exponentiated Gradient, two major instances of OMD,
enjoy closed-form updates, making them easy to use in practice. Finally, we
present experiments on both synthetic and real-world datasets supporting our
findings
Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
We propose a simple yet powerful test statistic to quantify the discrepancy
between two conditional distributions. The new statistic avoids the explicit
estimation of the underlying distributions in highdimensional space and it
operates on the cone of symmetric positive semidefinite (SPS) matrix using the
Bregman matrix divergence. Moreover, it inherits the merits of the correntropy
function to explicitly incorporate high-order statistics in the data. We
present the properties of our new statistic and illustrate its connections to
prior art. We finally show the applications of our new statistic on three
different machine learning problems, namely the multi-task learning over
graphs, the concept drift detection, and the information-theoretic feature
selection, to demonstrate its utility and advantage. Code of our statistic is
available at https://bit.ly/BregmanCorrentropy.Comment: manuscript accepted at IJCAI 20; added additional notes on
computational complexity and auto-differentiable property; code is available
at https://github.com/SJYuCNEL/Bregman-Correntropy-Conditional-Divergenc
Lifelong Spectral Clustering
In the past decades, spectral clustering (SC) has become one of the most
effective clustering algorithms. However, most previous studies focus on
spectral clustering tasks with a fixed task set, which cannot incorporate with
a new spectral clustering task without accessing to previously learned tasks.
In this paper, we aim to explore the problem of spectral clustering in a
lifelong machine learning framework, i.e., Lifelong Spectral Clustering (L2SC).
Its goal is to efficiently learn a model for a new spectral clustering task by
selectively transferring previously accumulated experience from knowledge
library. Specifically, the knowledge library of L2SC contains two components:
1) orthogonal basis library: capturing latent cluster centers among the
clusters in each pair of tasks; 2) feature embedding library: embedding the
feature manifold information shared among multiple related tasks. As a new
spectral clustering task arrives, L2SC firstly transfers knowledge from both
basis library and feature library to obtain encoding matrix, and further
redefines the library base over time to maximize performance across all the
clustering tasks. Meanwhile, a general online update formulation is derived to
alternatively update the basis library and feature library. Finally, the
empirical experiments on several real-world benchmark datasets demonstrate that
our L2SC model can effectively improve the clustering performance when
comparing with other state-of-the-art spectral clustering algorithms.Comment: 9 pages,7 figure
- …