45,186 research outputs found
Exact Dimensionality Selection for Bayesian PCA
We present a Bayesian model selection approach to estimate the intrinsic
dimensionality of a high-dimensional dataset. To this end, we introduce a novel
formulation of the probabilisitic principal component analysis model based on a
normal-gamma prior distribution. In this context, we exhibit a closed-form
expression of the marginal likelihood which allows to infer an optimal number
of components. We also propose a heuristic based on the expected shape of the
marginal likelihood curve in order to choose the hyperparameters. In
non-asymptotic frameworks, we show on simulated data that this exact
dimensionality selection approach is competitive with both Bayesian and
frequentist state-of-the-art methods
Cluster Variation Method in Statistical Physics and Probabilistic Graphical Models
The cluster variation method (CVM) is a hierarchy of approximate variational
techniques for discrete (Ising--like) models in equilibrium statistical
mechanics, improving on the mean--field approximation and the Bethe--Peierls
approximation, which can be regarded as the lowest level of the CVM. In recent
years it has been applied both in statistical physics and to inference and
optimization problems formulated in terms of probabilistic graphical models.
The foundations of the CVM are briefly reviewed, and the relations with
similar techniques are discussed. The main properties of the method are
considered, with emphasis on its exactness for particular models and on its
asymptotic properties.
The problem of the minimization of the variational free energy, which arises
in the CVM, is also addressed, and recent results about both provably
convergent and message-passing algorithms are discussed.Comment: 36 pages, 17 figure
Approximate reasoning for real-time probabilistic processes
We develop a pseudo-metric analogue of bisimulation for generalized
semi-Markov processes. The kernel of this pseudo-metric corresponds to
bisimulation; thus we have extended bisimulation for continuous-time
probabilistic processes to a much broader class of distributions than
exponential distributions. This pseudo-metric gives a useful handle on
approximate reasoning in the presence of numerical information -- such as
probabilities and time -- in the model. We give a fixed point characterization
of the pseudo-metric. This makes available coinductive reasoning principles for
reasoning about distances. We demonstrate that our approach is insensitive to
potentially ad hoc articulations of distance by showing that it is intrinsic to
an underlying uniformity. We provide a logical characterization of this
uniformity using a real-valued modal logic. We show that several quantitative
properties of interest are continuous with respect to the pseudo-metric. Thus,
if two processes are metrically close, then observable quantitative properties
of interest are indeed close.Comment: Preliminary version appeared in QEST 0
- …