93,368 research outputs found
Fast Online Node Labeling for Very Large Graphs
This paper studies the online node classification problem under a
transductive learning setting. Current methods either invert a graph kernel
matrix with runtime and space complexity
or sample a large volume of random spanning trees, thus are difficult to scale
to large graphs. In this work, we propose an improvement based on the
\textit{online relaxation} technique introduced by a series of works (Rakhlin
et al.,2012; Rakhlin and Sridharan, 2015; 2017). We first prove an effective
regret when suitable parameterized graph
kernels are chosen, then propose an approximate algorithm FastONL enjoying
regret based on this relaxation. The key of
FastONL is a \textit{generalized local push} method that effectively
approximates inverse matrix columns and applies to a series of popular kernels.
Furthermore, the per-prediction cost is
locally dependent on
the graph with linear memory cost. Experiments show that our scalable method
enjoys a better tradeoff between local and global consistency.Comment: 40 pages,17 figures, ICML 202
Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics
Kernel-based methods exhibit well-documented performance in various nonlinear
learning tasks. Most of them rely on a preselected kernel, whose prudent choice
presumes task-specific prior information. Especially when the latter is not
available, multi-kernel learning has gained popularity thanks to its
flexibility in choosing kernels from a prescribed kernel dictionary. Leveraging
the random feature approximation and its recent orthogonality-promoting
variant, the present contribution develops a scalable multi-kernel learning
scheme (termed Raker) to obtain the sought nonlinear learning function `on the
fly,' first for static environments. To further boost performance in dynamic
environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is
developed. AdaRaker accounts not only for data-driven learning of kernel
combination, but also for the unknown dynamics. Performance is analyzed in
terms of both static and dynamic regrets. AdaRaker is uniquely capable of
tracking nonlinear learning functions in environments with unknown dynamics,
and with with analytic performance guarantees. Tests with synthetic and real
datasets are carried out to showcase the effectiveness of the novel algorithms.Comment: 36 page
- …