2,950 research outputs found
Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods
We formulate the problem of neural network optimization as Bayesian
filtering, where the observations are the backpropagated gradients. While
neural network optimization has previously been studied using natural gradient
methods which are closely related to Bayesian inference, they were unable to
recover standard optimizers such as Adam and RMSprop with a root-mean-square
gradient normalizer, instead getting a mean-square normalizer. To recover the
root-mean-square normalizer, we find it necessary to account for the temporal
dynamics of all the other parameters as they are geing optimized. The resulting
optimizer, AdaBayes, adaptively transitions between SGD-like and Adam-like
behaviour, automatically recovers AdamW, a state of the art variant of Adam
with decoupled weight decay, and has generalisation performance competitive
with SGD
Dynamic Bayesian Combination of Multiple Imperfect Classifiers
Classifier combination methods need to make best use of the outputs of
multiple, imperfect classifiers to enable higher accuracy classifications. In
many situations, such as when human decisions need to be combined, the base
decisions can vary enormously in reliability. A Bayesian approach to such
uncertain combination allows us to infer the differences in performance between
individuals and to incorporate any available prior knowledge about their
abilities when training data is sparse. In this paper we explore Bayesian
classifier combination, using the computationally efficient framework of
variational Bayesian inference. We apply the approach to real data from a large
citizen science project, Galaxy Zoo Supernovae, and show that our method far
outperforms other established approaches to imperfect decision combination. We
go on to analyse the putative community structure of the decision makers, based
on their inferred decision making strategies, and show that natural groupings
are formed. Finally we present a dynamic Bayesian classifier combination
approach and investigate the changes in base classifier performance over time.Comment: 35 pages, 12 figure
Enhanced particle PHD filtering for multiple human tracking
PhD ThesisVideo-based single human tracking has found wide application but multiple
human tracking is more challenging and enhanced processing techniques are
required to estimate the positions and number of targets in each frame. In
this thesis, the particle probability hypothesis density (PHD) lter is therefore
the focus due to its ability to estimate both localization and cardinality
information related to multiple human targets. To improve the tracking performance
of the particle PHD lter, a number of enhancements are proposed.
The Student's-t distribution is employed within the state and measurement
models of the PHD lter to replace the Gaussian distribution because
of its heavier tails, and thereby better predict particles with larger amplitudes.
Moreover, the variational Bayesian approach is utilized to estimate
the relationship between the measurement noise covariance matrix and the
state model, and a joint multi-dimensioned Student's-t distribution is exploited.
In order to obtain more observable measurements, a backward retrodiction
step is employed to increase the measurement set, building upon the
concept of a smoothing algorithm. To make further improvement, an adaptive
step is used to combine the forward ltering and backward retrodiction
ltering operations through the similarities of measurements achieved over
discrete time. As such, the errors in the delayed measurements generated by
false alarms and environment noise are avoided.
In the nal work, information describing human behaviour is employed
iv
Abstract v
to aid particle sampling in the prediction step of the particle PHD lter,
which is captured in a social force model. A novel social force model is
proposed based on the exponential function. Furthermore, a Markov Chain
Monte Carlo (MCMC) step is utilized to resample the predicted particles,
and the acceptance ratio is calculated by the results from the social force
model to achieve more robust prediction. Then, a one class support vector
machine (OCSVM) is applied in the measurement model of the PHD lter,
trained on human features, to mitigate noise from the environment and to
achieve better tracking performance.
The proposed improvements of the particle PHD lters are evaluated
with benchmark datasets such as the CAVIAR, PETS2009 and TUD datasets
and assessed with quantitative and global evaluation measures, and are compared
with state-of-the-art techniques to con rm the improvement of multiple
human tracking performance
- …