13 research outputs found
On-Line Learning of Linear Dynamical Systems: Exponential Forgetting in Kalman Filters
Kalman filter is a key tool for time-series forecasting and analysis. We show
that the dependence of a prediction of Kalman filter on the past is decaying
exponentially, whenever the process noise is non-degenerate. Therefore, Kalman
filter may be approximated by regression on a few recent observations.
Surprisingly, we also show that having some process noise is essential for the
exponential decay. With no process noise, it may happen that the forecast
depends on all of the past uniformly, which makes forecasting more difficult.
Based on this insight, we devise an on-line algorithm for improper learning
of a linear dynamical system (LDS), which considers only a few most recent
observations. We use our decay results to provide the first regret bounds
w.r.t. to Kalman filters within learning an LDS. That is, we compare the
results of our algorithm to the best, in hindsight, Kalman filter for a given
signal. Also, the algorithm is practical: its per-update run-time is linear in
the regression depth
Variance Estimation For Dynamic Regression via Spectrum Thresholding
We consider the dynamic linear regression problem, where the predictor vector
may vary with time. This problem can be modeled as a linear dynamical system,
where the parameters that need to be learned are the variance of both the
process noise and the observation noise. While variance estimation for dynamic
regression is a natural problem, with a variety of applications, existing
approaches to this problem either lack guarantees or only have asymptotic
guarantees without explicit rates. In addition, all existing approaches rely
strongly on Guassianity of the noises. In this paper we study the global system
operator: the operator that maps the noise vectors to the output. In
particular, we obtain estimates on its spectrum, and as a result derive the
first known variance estimators with finite sample complexity guarantees.
Moreover, our results hold for arbitrary sub Gaussian distributions of noise
terms. We evaluate the approach on synthetic and real-world benchmarks
Community Detection via Measure Space Embedding
Abstract We present a new algorithm for community detection. The algorithm uses random walks to embed the graph in a space of measures, after which a modification of k-means in that space is applied. The algorithm is therefore fast and easily parallelizable. We evaluate the algorithm on standard random graph benchmarks, including some overlapping community benchmarks, and find its performance to be better or at least as good as previously known algorithms. We also prove a linear time (in number of edges) guarantee for the algorithm on a p, q-stochastic block model with where p ≥ c ·