115,331 research outputs found
Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures
Probabilistic graphical models are a central tool in AI; however, they are
generally not as expressive as deep neural models, and inference is notoriously
hard and slow. In contrast, deep probabilistic models such as sum-product
networks (SPNs) capture joint distributions in a tractable fashion, but still
lack the expressive power of intractable models based on deep neural networks.
Therefore, we introduce conditional SPNs (CSPNs), conditional density
estimators for multivariate and potentially hybrid domains which allow
harnessing the expressive power of neural networks while still maintaining
tractability guarantees. One way to implement CSPNs is to use an existing SPN
structure and condition its parameters on the input, e.g., via a deep neural
network. This approach, however, might misrepresent the conditional
independence structure present in data. Consequently, we also develop a
structure-learning approach that derives both the structure and parameters of
CSPNs from data. Our experimental evidence demonstrates that CSPNs are
competitive with other probabilistic models and yield superior performance on
multilabel image classification compared to mean field and mixture density
networks. Furthermore, they can successfully be employed as building blocks for
structured probabilistic models, such as autoregressive image models.Comment: 13 pages, 6 figure
Model-based kernel sum rule: kernel Bayesian inference with probabilistic model
Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned from data in a nonparametric manner. Various algorithms of kernel Bayesian inference have been developed by combining kernelized basic probabilistic operations such as the kernel sum rule and kernel Bayes’ rule. However, the current framework is fully nonparametric, and it does not allow a user to flexibly combine nonparametric and model-based inferences. This is inefficient when there are good probabilistic models (or simulation models) available for some parts of a graphical model; this is in particular true in scientific fields where “models” are the central topic of study. Our contribution in this paper is to introduce a novel approach, termed the model-based kernel sum rule (Mb-KSR), to combine a probabilistic model and kernel Bayesian inference. By combining the Mb-KSR with the existing kernelized probabilistic rules, one can develop various algorithms for hybrid (i.e., nonparametric and model-based) inferences. As an illustrative example, we consider Bayesian filtering in a state space model, where typically there exists an accurate probabilistic model for the state transition process. We propose a novel filtering method that combines model-based inference for the state transition process and data-driven, nonparametric inference for the observation generating process. We empirically validate our approach with synthetic and real-data experiments, the latter being the problem of vision-based mobile robot localization in robotics, which illustrates the effectiveness of the proposed hybrid approach
Recommended from our members
True-false lumen segmentation of aortic dissection using multi-scale wavelet analysis and generative-discriminative model matching
Computer aided diagnosis in the medical image domain requires sophisticated probabilistic models to formulate quantitative behavior in image space. In the diagnostic process detailed knowledge of model performance with respect to accuracy, variability, and uncertainty is crucial. This challenge has lead to the fusion of two successful learning schools namely generative and discriminative learning. In this paper, we propose a generative-discriminative learning approach to predict object boundaries in medical image datasets. In our approach, we perform probabilistic model matching of both modeling domains to fuse into the prediction step appearance and structural information of the object of interest while exploiting the strength of both learning paradigms. In particular, we apply our method to the task of true-false lumen segmentation of aortic dissections an acute disease that requires automated quantification for assisted medical diagnosis. We report empirical results for true-false lumen discrimination of aortic dissection segmentation showing superior behavior of the hybrid generative-discriminative approach over their non hybrid generative counterpart
Hybrid Models for Human Motion Recognition
Probabilistic models have been previously shown to be
efficient and effective for modeling and recognition of human motion. In particular we focus on methods which represent the human motion model as a triangulated graph.
Previous approaches learned models based just on positions
and velocities of the body parts while ignoring their
appearance. Moreover, a heuristic approach was commonly
used to obtain translation invariance.
In this paper we suggest an improved approach for
learning such models and using them for human motion
recognition. The suggested approach combines multiple
cues, i.e., positions, velocities and appearance into both
the learning and detection phases. Furthermore, we introduce
global variables in the model, which can represent
global properties such as translation, scale or view-point.
The model is learned in an unsupervised manner from unlabelled data. We show that the suggested hybrid probabilistic model (which combines global variables, like translation, with local variables, like relative positions and appearances of body parts), leads to: (i) faster convergence of learning phase, (ii) robustness to occlusions, and, (iii) higher recognition rate
Continuous Mixtures of Tractable Probabilistic Models
Probabilistic models based on continuous latent spaces, such as variational
autoencoders, can be understood as uncountable mixture models where components
depend continuously on the latent code. They have proven expressive tools for
generative and probabilistic modelling, but are at odds with tractable
probabilistic inference, that is, computing marginals and conditionals of the
represented probability distribution. Meanwhile, tractable probabilistic models
such as probabilistic circuits (PCs) can be understood as hierarchical discrete
mixture models, which allows them to perform exact inference, but often they
show subpar performance in comparison to continuous latent-space models. In
this paper, we investigate a hybrid approach, namely continuous mixtures of
tractable models with a small latent dimension. While these models are
analytically intractable, they are well amenable to numerical integration
schemes based on a finite set of integration points. With a large enough number
of integration points the approximation becomes de-facto exact. Moreover, using
a finite set of integration points, the approximation method can be compiled
into a PC performing `exact inference in an approximate model'. In experiments,
we show that this simple scheme proves remarkably effective, as PCs learned
this way set new state-of-the-art for tractable models on many standard density
estimation benchmarks
- …