746 research outputs found
Stochastic Discriminative EM
Stochastic discriminative EM (sdEM) is an online-EM-type algorithm for
discriminative training of probabilistic generative models belonging to the
exponential family. In this work, we introduce and justify this algorithm as a
stochastic natural gradient descent method, i.e. a method which accounts for
the information geometry in the parameter space of the statistical model. We
show how this learning algorithm can be used to train probabilistic generative
models by minimizing different discriminative loss functions, such as the
negative conditional log-likelihood and the Hinge loss. The resulting models
trained by sdEM are always generative (i.e. they define a joint probability
distribution) and, in consequence, allows to deal with missing data and latent
variables in a principled way either when being learned or when making
predictions. The performance of this method is illustrated by several text
classification problems for which a multinomial naive Bayes and a latent
Dirichlet allocation based classifier are learned using different
discriminative loss functions.Comment: UAI 2014 paper + Supplementary Material. In Proceedings of the
Thirtieth Conference on Uncertainty in Artificial Intelligence (UAI 2014),
edited by Nevin L. Zhang and Jian Tian. AUAI Pres
Shuffled Multi-Channel Sparse Signal Recovery
Mismatches between samples and their respective channel or target commonly
arise in several real-world applications. For instance, whole-brain calcium
imaging of freely moving organisms, multiple-target tracking or multi-person
contactless vital sign monitoring may be severely affected by mismatched
sample-channel assignments. To systematically address this fundamental problem,
we pose it as a signal reconstruction problem where we have lost
correspondences between the samples and their respective channels. Assuming
that we have a sensing matrix for the underlying signals, we show that the
problem is equivalent to a structured unlabeled sensing problem, and establish
sufficient conditions for unique recovery. To the best of our knowledge, a
sampling result for the reconstruction of shuffled multi-channel signals has
not been considered in the literature and existing methods for unlabeled
sensing cannot be directly applied. We extend our results to the case where the
signals admit a sparse representation in an overcomplete dictionary (i.e., the
sensing matrix is not precisely known), and derive sufficient conditions for
the reconstruction of shuffled sparse signals. We propose a robust
reconstruction method that combines sparse signal recovery with robust linear
regression for the two-channel case. The performance and robustness of the
proposed approach is illustrated in an application related to whole-brain
calcium imaging. The proposed methodology can be generalized to sparse signal
representations other than the ones considered in this work to be applied in a
variety of real-world problems with imprecise measurement or channel
assignment.Comment: Submitted to TS
- …