451 research outputs found
Copula-type Estimators for Flexible Multivariate Density Modeling using Mixtures
Copulas are popular as models for multivariate dependence because they allow
the marginal densities and the joint dependence to be modeled separately.
However, they usually require that the transformation from uniform marginals to
the marginals of the joint dependence structure is known. This can only be done
for a restricted set of copulas, e.g. a normal copula. Our article introduces
copula-type estimators for flexible multivariate density estimation which also
allow the marginal densities to be modeled separately from the joint
dependence, as in copula modeling, but overcomes the lack of flexibility of
most popular copula estimators. An iterative scheme is proposed for estimating
copula-type estimators and its usefulness is demonstrated through simulation
and real examples. The joint dependence is is modeled by mixture of normals and
mixture of normals factor analyzers models, and mixture of t and mixture of t
factor analyzers models. We develop efficient Variational Bayes algorithms for
fitting these in which model selection is performed automatically. Based on
these mixture models, we construct four classes of copula-type densities which
are far more flexible than current popular copula densities, and outperform
them in simulation and several real data sets.Comment: 27 pages, 3 figure
Cumulative Distribution Functions As The Foundation For Probabilistic Models
This thesis discusses applications of probabilistic and connectionist models for
constructing and training cumulative distribution functions (CDFs). First, it is shown
how existing tools from the copula literature can be combined to build probabilistic
models. It is found that this simple construction leads to numerical and scalability
issues that make training and inference challenging.
Next, several innovative ideas, combining neural networks, automatic differentiation
and copula functions, introduce how to assemble black-box probabilistic
models. The basic building block is a cumulative distribution function that is straightforward
to construct, composed of arithmetic operations and nonlinear functions.
There is no need to assume any specific parametric probability density function
(PDF), making the model flexible and normalisation unnecessary. The only requirement
is to design a computational graph that parameterises monotonically
non-decreasing functions with a constrained range. Training can be then performed
using standard tools from any neural network software library.
Finally, factorial hidden Markov models (FHMMs) for sequential data are
presented. It is shown how to leverage cumulative distribution functions in the
form of the Gaussian copula and amortised stochastic variational method to encode
hidden Markov chains coherently. This approach enables efficient learning and
inference to model long sequences of high-dimensional data with long-range dependencies.
Tackling such complex problems was impossible with the established
FHMM approximate inference algorithm.
It is empirically verified on several problems that some of the estimators introduced
in this work can perform comparably or better than the currently popular
models. Especially for tasks requiring tail-area or marginal probabilities that can be
read directly from a cumulative distribution function
- …