1,423 research outputs found
Non-parametric Estimation of Stochastic Differential Equations with Sparse Gaussian Processes
The application of Stochastic Differential Equations (SDEs) to the analysis
of temporal data has attracted increasing attention, due to their ability to
describe complex dynamics with physically interpretable equations. In this
paper, we introduce a non-parametric method for estimating the drift and
diffusion terms of SDEs from a densely observed discrete time series. The use
of Gaussian processes as priors permits working directly in a function-space
view and thus the inference takes place directly in this space. To cope with
the computational complexity that requires the use of Gaussian processes, a
sparse Gaussian process approximation is provided. This approximation permits
the efficient computation of predictions for the drift and diffusion terms by
using a distribution over a small subset of pseudo-samples. The proposed method
has been validated using both simulated data and real data from economy and
paleoclimatology. The application of the method to real data demonstrates its
ability to capture the behaviour of complex systems
Preconditioning Kernel Matrices
The computational and storage complexity of kernel machines presents the
primary barrier to their scaling to large, modern, datasets. A common way to
tackle the scalability issue is to use the conjugate gradient algorithm, which
relieves the constraints on both storage (the kernel matrix need not be stored)
and computation (both stochastic gradients and parallelization can be used).
Even so, conjugate gradient is not without its own issues: the conditioning of
kernel matrices is often such that conjugate gradients will have poor
convergence in practice. Preconditioning is a common approach to alleviating
this issue. Here we propose preconditioned conjugate gradients for kernel
machines, and develop a broad range of preconditioners particularly useful for
kernel matrices. We describe a scalable approach to both solving kernel
machines and learning their hyperparameters. We show this approach is exact in
the limit of iterations and outperforms state-of-the-art approximations for a
given computational budget
A Fusion Framework for Camouflaged Moving Foreground Detection in the Wavelet Domain
Detecting camouflaged moving foreground objects has been known to be
difficult due to the similarity between the foreground objects and the
background. Conventional methods cannot distinguish the foreground from
background due to the small differences between them and thus suffer from
under-detection of the camouflaged foreground objects. In this paper, we
present a fusion framework to address this problem in the wavelet domain. We
first show that the small differences in the image domain can be highlighted in
certain wavelet bands. Then the likelihood of each wavelet coefficient being
foreground is estimated by formulating foreground and background models for
each wavelet band. The proposed framework effectively aggregates the
likelihoods from different wavelet bands based on the characteristics of the
wavelet transform. Experimental results demonstrated that the proposed method
significantly outperformed existing methods in detecting camouflaged foreground
objects. Specifically, the average F-measure for the proposed algorithm was
0.87, compared to 0.71 to 0.8 for the other state-of-the-art methods.Comment: 13 pages, accepted by IEEE TI
Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases
For big data analysis, high computational cost for Bayesian methods often
limits their applications in practice. In recent years, there have been many
attempts to improve computational efficiency of Bayesian inference. Here we
propose an efficient and scalable computational technique for a
state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian
Monte Carlo (HMC). The key idea is to explore and exploit the structure and
regularity in parameter space for the underlying probabilistic model to
construct an effective approximation of its geometric properties. To this end,
we build a surrogate function to approximate the target distribution using
properly chosen random bases and an efficient optimization process. The
resulting method provides a flexible, scalable, and efficient sampling
algorithm, which converges to the correct target distribution. We show that by
choosing the basis functions and optimization process differently, our method
can be related to other approaches for the construction of surrogate functions
such as generalized additive models or Gaussian process models. Experiments
based on simulated and real data show that our approach leads to substantially
more efficient sampling algorithms compared to existing state-of-the art
methods
Security Evaluation of Support Vector Machines in Adversarial Environments
Support Vector Machines (SVMs) are among the most popular classification
techniques adopted in security applications like malware detection, intrusion
detection, and spam filtering. However, if SVMs are to be incorporated in
real-world security systems, they must be able to cope with attack patterns
that can either mislead the learning algorithm (poisoning), evade detection
(evasion), or gain information about their internal parameters (privacy
breaches). The main contributions of this chapter are twofold. First, we
introduce a formal general framework for the empirical evaluation of the
security of machine-learning systems. Second, according to our framework, we
demonstrate the feasibility of evasion, poisoning and privacy attacks against
SVMs in real-world security problems. For each attack technique, we evaluate
its impact and discuss whether (and how) it can be countered through an
adversary-aware design of SVMs. Our experiments are easily reproducible thanks
to open-source code that we have made available, together with all the employed
datasets, on a public repository.Comment: 47 pages, 9 figures; chapter accepted into book 'Support Vector
Machine Applications
Double Normalizing Flows: Flexible Bayesian Gaussian Process ODEs Learning
Recently, Gaussian processes have been utilized to model the vector field of
continuous dynamical systems. Bayesian inference for such models
\cite{hegde2022variational} has been extensively studied and has been applied
in tasks such as time series prediction, providing uncertain estimates.
However, previous Gaussian Process Ordinary Differential Equation (ODE) models
may underperform on datasets with non-Gaussian process priors, as their
constrained priors and mean-field posteriors may lack flexibility. To address
this limitation, we incorporate normalizing flows to reparameterize the vector
field of ODEs, resulting in a more flexible and expressive prior distribution.
Additionally, due to the analytically tractable probability density functions
of normalizing flows, we apply them to the posterior inference of GP ODEs,
generating a non-Gaussian posterior. Through these dual applications of
normalizing flows, our model improves accuracy and uncertainty estimates for
Bayesian Gaussian Process ODEs. The effectiveness of our approach is
demonstrated on simulated dynamical systems and real-world human motion data,
including tasks such as time series prediction and missing data recovery.
Experimental results indicate that our proposed method effectively captures
model uncertainty while improving accuracy
Bayesian Quantile and Expectile Optimisation
Bayesian optimisation is widely used to optimise stochastic black box
functions. While most strategies are focused on optimising conditional
expectations, a large variety of applications require risk-averse decisions and
alternative criteria accounting for the distribution tails need to be
considered. In this paper, we propose new variational models for Bayesian
quantile and expectile regression that are well-suited for heteroscedastic
settings. Our models consist of two latent Gaussian processes accounting
respectively for the conditional quantile (or expectile) and variance that are
chained through asymmetric likelihood functions. Furthermore, we propose two
Bayesian optimisation strategies, either derived from a GP-UCB or Thompson
sampling, that are tailored to such models and that can accommodate large
batches of points. As illustrated in the experimental section, the proposed
approach clearly outperforms the state of the art
- …