764 research outputs found
Distributed Parameter Estimation via Pseudo-likelihood
Estimating statistical models within sensor networks requires distributed
algorithms, in which both data and computation are distributed across the nodes
of the network. We propose a general approach for distributed learning based on
combining local estimators defined by pseudo-likelihood components,
encompassing a number of combination methods, and provide both theoretical and
experimental analysis. We show that simple linear combination or max-voting
methods, when combined with second-order information, are statistically
competitive with more advanced and costly joint optimization. Our algorithms
have many attractive properties including low communication and computational
cost and "any-time" behavior.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
A Low Density Lattice Decoder via Non-Parametric Belief Propagation
The recent work of Sommer, Feder and Shalvi presented a new family of codes
called low density lattice codes (LDLC) that can be decoded efficiently and
approach the capacity of the AWGN channel. A linear time iterative decoding
scheme which is based on a message-passing formulation on a factor graph is
given.
In the current work we report our theoretical findings regarding the relation
between the LDLC decoder and belief propagation. We show that the LDLC decoder
is an instance of non-parametric belief propagation and further connect it to
the Gaussian belief propagation algorithm. Our new results enable borrowing
knowledge from the non-parametric and Gaussian belief propagation domains into
the LDLC domain. Specifically, we give more general convergence conditions for
convergence of the LDLC decoder (under the same assumptions of the original
LDLC convergence analysis). We discuss how to extend the LDLC decoder from
Latin square to full rank, non-square matrices. We propose an efficient
construction of sparse generator matrix and its matching decoder. We report
preliminary experimental results which show our decoder has comparable symbol
to error rate compared to the original LDLC decoder.%Comment: Submitted for publicatio
Gibbs Sampling for (Coupled) Infinite Mixture Models in the Stick Breaking Representation
Nonparametric Bayesian approaches to clustering, information retrieval,
language modeling and object recognition have recently shown great promise as a
new paradigm for unsupervised data analysis. Most contributions have focused on
the Dirichlet process mixture models or extensions thereof for which efficient
Gibbs samplers exist. In this paper we explore Gibbs samplers for infinite
complexity mixture models in the stick breaking representation. The advantage
of this representation is improved modeling flexibility. For instance, one can
design the prior distribution over cluster sizes or couple multiple infinite
mixture models (e.g. over time) at the level of their parameters (i.e. the
dependent Dirichlet process model). However, Gibbs samplers for infinite
mixture models (as recently introduced in the statistics literature) seem to
mix poorly over cluster labels. Among others issues, this can have the adverse
effect that labels for the same cluster in coupled mixture models are mixed up.
We introduce additional moves in these samplers to improve mixing over cluster
labels and to bring clusters into correspondence. An application to modeling of
storm trajectories is used to illustrate these ideas.Comment: Appears in Proceedings of the Twenty-Second Conference on Uncertainty
in Artificial Intelligence (UAI2006
AUC margin loss for limited, imbalanced and noisy medical image diagnosis - A case study on CheXpert5000
The AUC margin loss is a valuable loss function for medical image classification as it addresses the problems of imbalanced and noisy labels. It is used by the current winner of the CheXpert competition. The CheXpert dataset is a large dataset (200k+ images), however datasets in the range of 1k-10k medical datasets are much more common. This raises the question if optimizing AUC margin loss also is effective in scenarios with limited data.We compare AUC margin loss optimization to binary cross-entropy on limited, imbalanced and noisy CheXpert5000, a subset of CheXpert dataset. We show that AUC margin loss is beneficial for limited data and considerably improves accuracy in the presence of label noise. It also improves out-of-box calibration
- …