11,270 research outputs found
Stochastic Behavior of the Nonnegative Least Mean Fourth Algorithm for Stationary Gaussian Inputs and Slow Learning
Some system identification problems impose nonnegativity constraints on the
parameters to estimate due to inherent physical characteristics of the unknown
system. The nonnegative least-mean-square (NNLMS) algorithm and its variants
allow to address this problem in an online manner. A nonnegative least mean
fourth (NNLMF) algorithm has been recently proposed to improve the performance
of these algorithms in cases where the measurement noise is not Gaussian. This
paper provides a first theoretical analysis of the stochastic behavior of the
NNLMF algorithm for stationary Gaussian inputs and slow learning. Simulation
results illustrate the accuracy of the proposed analysis.Comment: 11 pages, 8 figures, submitted for publicatio
Efficient State-Space Inference of Periodic Latent Force Models
Latent force models (LFM) are principled approaches to incorporating
solutions to differential equations within non-parametric inference methods.
Unfortunately, the development and application of LFMs can be inhibited by
their computational cost, especially when closed-form solutions for the LFM are
unavailable, as is the case in many real world problems where these latent
forces exhibit periodic behaviour. Given this, we develop a new sparse
representation of LFMs which considerably improves their computational
efficiency, as well as broadening their applicability, in a principled way, to
domains with periodic or near periodic latent forces. Our approach uses a
linear basis model to approximate one generative model for each periodic force.
We assume that the latent forces are generated from Gaussian process priors and
develop a linear basis model which fully expresses these priors. We apply our
approach to model the thermal dynamics of domestic buildings and show that it
is effective at predicting day-ahead temperatures within the homes. We also
apply our approach within queueing theory in which quasi-periodic arrival rates
are modelled as latent forces. In both cases, we demonstrate that our approach
can be implemented efficiently using state-space methods which encode the
linear dynamic systems via LFMs. Further, we show that state estimates obtained
using periodic latent force models can reduce the root mean squared error to
17% of that from non-periodic models and 27% of the nearest rival approach
which is the resonator model.Comment: 61 pages, 13 figures, accepted for publication in JMLR. Updates from
earlier version occur throughout article in response to JMLR review
Stochastic Behavior Analysis of the Gaussian Kernel Least-Mean-Square Algorithm
The kernel least-mean-square (KLMS) algorithm is a popular algorithm in nonlinear adaptive filtering due to its
simplicity and robustness. In kernel adaptive filters, the statistics of the input to the linear filter depends on the parameters of the kernel employed. Moreover, practical implementations require a finite nonlinearity model order. A Gaussian KLMS has two design parameters, the step size and the Gaussian kernel bandwidth. Thus, its design requires analytical models for the algorithm behavior as a function of these two parameters. This paper studies the steady-state behavior and the transient behavior of the
Gaussian KLMS algorithm for Gaussian inputs and a finite order nonlinearity model. In particular, we derive recursive expressions for the mean-weight-error vector and the mean-square-error. The model predictions show excellent agreement with Monte Carlo simulations in transient and steady state. This allows the explicit analytical determination of stability limits, and gives opportunity
to choose the algorithm parameters a priori in order to achieve prescribed convergence speed and quality of the estimate. Design examples are presented which validate the theoretical analysis and illustrates its application
A Novel Family of Adaptive Filtering Algorithms Based on The Logarithmic Cost
We introduce a novel family of adaptive filtering algorithms based on a
relative logarithmic cost. The new family intrinsically combines the higher and
lower order measures of the error into a single continuous update based on the
error amount. We introduce important members of this family of algorithms such
as the least mean logarithmic square (LMLS) and least logarithmic absolute
difference (LLAD) algorithms that improve the convergence performance of the
conventional algorithms. However, our approach and analysis are generic such
that they cover other well-known cost functions as described in the paper. The
LMLS algorithm achieves comparable convergence performance with the least mean
fourth (LMF) algorithm and extends the stability bound on the step size. The
LLAD and least mean square (LMS) algorithms demonstrate similar convergence
performance in impulse-free noise environments while the LLAD algorithm is
robust against impulsive interferences and outperforms the sign algorithm (SA).
We analyze the transient, steady state and tracking performance of the
introduced algorithms and demonstrate the match of the theoretical analyzes and
simulation results. We show the extended stability bound of the LMLS algorithm
and analyze the robustness of the LLAD algorithm against impulsive
interferences. Finally, we demonstrate the performance of our algorithms in
different scenarios through numerical examples.Comment: Submitted to IEEE Transactions on Signal Processin
A constructive mean field analysis of multi population neural networks with random synaptic weights and stochastic inputs
We deal with the problem of bridging the gap between two scales in neuronal
modeling. At the first (microscopic) scale, neurons are considered individually
and their behavior described by stochastic differential equations that govern
the time variations of their membrane potentials. They are coupled by synaptic
connections acting on their resulting activity, a nonlinear function of their
membrane potential. At the second (mesoscopic) scale, interacting populations
of neurons are described individually by similar equations. The equations
describing the dynamical and the stationary mean field behaviors are considered
as functional equations on a set of stochastic processes. Using this new point
of view allows us to prove that these equations are well-posed on any finite
time interval and to provide a constructive method for effectively computing
their unique solution. This method is proved to converge to the unique solution
and we characterize its complexity and convergence rate. We also provide
partial results for the stationary problem on infinite time intervals. These
results shed some new light on such neural mass models as the one of Jansen and
Rit \cite{jansen-rit:95}: their dynamics appears as a coarse approximation of
the much richer dynamics that emerges from our analysis. Our numerical
experiments confirm that the framework we propose and the numerical methods we
derive from it provide a new and powerful tool for the exploration of neural
behaviors at different scales.Comment: 55 pages, 4 figures, to appear in "Frontiers in Neuroscience
- âŠ