143 research outputs found
Distributed Adaptive Learning with Multiple Kernels in Diffusion Networks
We propose an adaptive scheme for distributed learning of nonlinear functions
by a network of nodes. The proposed algorithm consists of a local adaptation
stage utilizing multiple kernels with projections onto hyperslabs and a
diffusion stage to achieve consensus on the estimates over the whole network.
Multiple kernels are incorporated to enhance the approximation of functions
with several high and low frequency components common in practical scenarios.
We provide a thorough convergence analysis of the proposed scheme based on the
metric of the Cartesian product of multiple reproducing kernel Hilbert spaces.
To this end, we introduce a modified consensus matrix considering this specific
metric and prove its equivalence to the ordinary consensus matrix. Besides, the
use of hyperslabs enables a significant reduction of the computational demand
with only a minor loss in the performance. Numerical evaluations with synthetic
and real data are conducted showing the efficacy of the proposed algorithm
compared to the state of the art schemes.Comment: Double-column 15 pages, 10 figures, submitted to IEEE Trans. Signal
Processin
Analyzing sparse dictionaries for online learning with kernels
Many signal processing and machine learning methods share essentially the
same linear-in-the-parameter model, with as many parameters as available
samples as in kernel-based machines. Sparse approximation is essential in many
disciplines, with new challenges emerging in online learning with kernels. To
this end, several sparsity measures have been proposed in the literature to
quantify sparse dictionaries and constructing relevant ones, the most prolific
ones being the distance, the approximation, the coherence and the Babel
measures. In this paper, we analyze sparse dictionaries based on these
measures. By conducting an eigenvalue analysis, we show that these sparsity
measures share many properties, including the linear independence condition and
inducing a well-posed optimization problem. Furthermore, we prove that there
exists a quasi-isometry between the parameter (i.e., dual) space and the
dictionary's induced feature space.Comment: 10 page
Recovering Latent Signals from a Mixture of Measurements using a Gaussian Process Prior
In sensing applications, sensors cannot always measure the latent quantity of
interest at the required resolution, sometimes they can only acquire a blurred
version of it due the sensor's transfer function. To recover latent signals
when only noisy mixed measurements of the signal are available, we propose the
Gaussian process mixture of measurements (GPMM), which models the latent signal
as a Gaussian process (GP) and allows us to perform Bayesian inference on such
signal conditional to a set of noisy mixture of measurements. We describe how
to train GPMM, that is, to find the hyperparameters of the GP and the mixing
weights, and how to perform inference on the latent signal under GPMM;
additionally, we identify the solution to the underdetermined linear system
resulting from a sensing application as a particular case of GPMM. The proposed
model is validated in the recovery of three signals: a smooth synthetic signal,
a real-world heart-rate time series and a step function, where GPMM
outperformed the standard GP in terms of estimation error, uncertainty
representation and recovery of the spectral content of the latent signal.Comment: Published on IEEE Signal Processing Letters on Dec. 201
Recursive multikernel filters exploiting nonlinear temporal structure
In kernel methods, temporal information on the data is commonly included by using time-delayed embeddings as inputs. Recently, an alternative formulation was proposed by defining a γ-filter explicitly in a reproducing kernel Hilbert space, giving rise to a complex model where multiple kernels operate on different temporal combinations of the input signal. In the original formulation, the kernels are then simply combined to obtain a single kernel matrix (for instance by averaging), which provides computational benefits but discards important information on the temporal structure of the signal. Inspired by works on multiple kernel learning, we overcome this drawback by considering the different kernels separately. We propose an efficient strategy to adaptively combine and select these kernels during the training phase. The resulting batch and online algorithms automatically learn to process highly nonlinear temporal information extracted from the input signal, which is implicitly encoded in the kernel values. We evaluate our proposal on several artificial and real tasks, showing that it can outperform classical approaches both in batch and online settings.S. Van Vaerenbergh is supported by the Spanish Ministry of Economy and Competitiveness (under project TEC2014-57402-JIN). S. Scardapane is supported in part by Italian MIUR, “Progetti di Ricerca di Rilevante Interesse Nazionale”, GAUChO project, under Grant 2015YPXH4W-004
A stochastic behavior analysis of stochastic restricted-gradient descent algorithm in reproducing kernel Hilbert spaces
This paper presents a stochastic behavior analysis of a kernel-based
stochastic restricted-gradient descent method. The restricted gradient gives a
steepest ascent direction within the so-called dictionary subspace. The
analysis provides the transient and steady state performance in the mean
squared error criterion. It also includes stability conditions in the mean and
mean-square sense. The present study is based on the analysis of the kernel
normalized least mean square (KNLMS) algorithm initially proposed by Chen et
al. Simulation results validate the analysis
- …