46 research outputs found
Computing the Kullback-Leibler Divergence between two Weibull Distributions
We derive a closed form solution for the Kullback-Leibler divergence between
two Weibull distributions. These notes are meant as reference material and
intended to provide a guided tour towards a result that is often mentioned but
seldom made explicit in the literature
Retinal metric: a stimulus distance measure derived from population neural responses
The ability of the organism to distinguish between various stimuli is limited
by the structure and noise in the population code of its sensory neurons. Here
we infer a distance measure on the stimulus space directly from the recorded
activity of 100 neurons in the salamander retina. In contrast to previously
used measures of stimulus similarity, this "neural metric" tells us how
distinguishable a pair of stimulus clips is to the retina, given the noise in
the neural population response. We show that the retinal distance strongly
deviates from Euclidean, or any static metric, yet has a simple structure: we
identify the stimulus features that the neural population is jointly sensitive
to, and show the SVM-like kernel function relating the stimulus and neural
response spaces. We show that the non-Euclidean nature of the retinal distance
has important consequences for neural decoding.Comment: 5 pages, 4 figures, to appear in Phys Rev Let
Direct Ensemble Estimation of Density Functionals
Estimating density functionals of analog sources is an important problem in
statistical signal processing and information theory. Traditionally, estimating
these quantities requires either making parametric assumptions about the
underlying distributions or using non-parametric density estimation followed by
integration. In this paper we introduce a direct nonparametric approach which
bypasses the need for density estimation by using the error rates of k-NN
classifiers asdata-driven basis functions that can be combined to estimate a
range of density functionals. However, this method is subject to a non-trivial
bias that dramatically slows the rate of convergence in higher dimensions. To
overcome this limitation, we develop an ensemble method for estimating the
value of the basis function which, under some minor constraints on the
smoothness of the underlying distributions, achieves the parametric rate of
convergence regardless of data dimension.Comment: 5 page
An on-line speaker adaptation method for HMM-based speech recognizers
In the past few years numerous techniques have been proposed to improve the efficiency of basic adaptation methods like MLLR and MAP. These adaptation methods have a common aim, which is to increase the likelihood of the phoneme models for a particular speaker. During their operation, these speaker adaptation methods need precise phonetic segmentation information of the actual utterance, but these data samples are often faulty. To improve the overall performance, only those frames from the spoken sentence which are well segmented should be retained, while the incorrectly segmented data should not be used during adaptation. Several heuristic algorithms have been proposed in the literature for the selection of the reliably segmented data blocks, and here we would like to suggest some new heuristics that discriminate between faulty and well-segmented data. The effect of these methods on the efficiency of speech recognition using speaker adaptation is examined, and conclusions for each will be drawn. Besided post-filtering the set of the segmented adaptation examples, another way of improving the efficiency of the adaptation method might be to create a more precise segmentation, which should then reduce the chance of faulty data samples being included. We suggest a method like this here as well which is based on a scoring procedure for the N-best lists, taking into account phoneme duration
Beyond Gauss: Image-Set Matching on the Riemannian Manifold of PDFs
State-of-the-art image-set matching techniques typically implicitly model
each image-set with a Gaussian distribution. Here, we propose to go beyond
these representations and model image-sets as probability distribution
functions (PDFs) using kernel density estimators. To compare and match
image-sets, we exploit Csiszar f-divergences, which bear strong connections to
the geodesic distance defined on the space of PDFs, i.e., the statistical
manifold. Furthermore, we introduce valid positive definite kernels on the
statistical manifolds, which let us make use of more powerful classification
schemes to match image-sets. Finally, we introduce a supervised dimensionality
reduction technique that learns a latent space where f-divergences reflect the
class labels of the data. Our experiments on diverse problems, such as
video-based face recognition and dynamic texture classification, evidence the
benefits of our approach over the state-of-the-art image-set matching methods