56,684 research outputs found
Universal Sequential Outlier Hypothesis Testing
Universal outlier hypothesis testing is studied in a sequential setting.
Multiple observation sequences are collected, a small subset of which are
outliers. A sequence is considered an outlier if the observations in that
sequence are generated by an "outlier" distribution, distinct from a common
"typical" distribution governing the majority of the sequences. Apart from
being distinct, the outlier and typical distributions can be arbitrarily close.
The goal is to design a universal test to best discern all the outlier
sequences. A universal test with the flavor of the repeated significance test
is proposed and its asymptotic performance is characterized under various
universal settings. The proposed test is shown to be universally consistent.
For the model with identical outliers, the test is shown to be asymptotically
optimal universally when the number of outliers is the largest possible and
with the typical distribution being known, and its asymptotic performance
otherwise is also characterized. An extension of the findings to the model with
multiple distinct outliers is also discussed. In all cases, it is shown that
the asymptotic performance guarantees for the proposed test when neither the
outlier nor typical distribution is known converge to those when the typical
distribution is known.Comment: Proc. of the Asilomar Conference on Signals, Systems, and Computers,
2014. To appea
Sequential Gaussian Processes for Online Learning of Nonstationary Functions
Many machine learning problems can be framed in the context of estimating
functions, and often these are time-dependent functions that are estimated in
real-time as observations arrive. Gaussian processes (GPs) are an attractive
choice for modeling real-valued nonlinear functions due to their flexibility
and uncertainty quantification. However, the typical GP regression model
suffers from several drawbacks: i) Conventional GP inference scales
with respect to the number of observations; ii) updating a GP model
sequentially is not trivial; and iii) covariance kernels often enforce
stationarity constraints on the function, while GPs with non-stationary
covariance kernels are often intractable to use in practice. To overcome these
issues, we propose an online sequential Monte Carlo algorithm to fit mixtures
of GPs that capture non-stationary behavior while allowing for fast,
distributed inference. By formulating hyperparameter optimization as a
multi-armed bandit problem, we accelerate mixing for real time inference. Our
approach empirically improves performance over state-of-the-art methods for
online GP estimation in the context of prediction for simulated non-stationary
data and hospital time series data
Lifelong Generative Modeling
Lifelong learning is the problem of learning multiple consecutive tasks in a
sequential manner, where knowledge gained from previous tasks is retained and
used to aid future learning over the lifetime of the learner. It is essential
towards the development of intelligent machines that can adapt to their
surroundings. In this work we focus on a lifelong learning approach to
unsupervised generative modeling, where we continuously incorporate newly
observed distributions into a learned model. We do so through a student-teacher
Variational Autoencoder architecture which allows us to learn and preserve all
the distributions seen so far, without the need to retain the past data nor the
past models. Through the introduction of a novel cross-model regularizer,
inspired by a Bayesian update rule, the student model leverages the information
learned by the teacher, which acts as a probabilistic knowledge store. The
regularizer reduces the effect of catastrophic interference that appears when
we learn over sequences of distributions. We validate our model's performance
on sequential variants of MNIST, FashionMNIST, PermutedMNIST, SVHN and Celeb-A
and demonstrate that our model mitigates the effects of catastrophic
interference faced by neural networks in sequential learning scenarios.Comment: 32 page
- …