102 research outputs found
Estimating Mixture Entropy with Pairwise Distances
Mixture distributions arise in many parametric and non-parametric settings --
for example, in Gaussian mixture models and in non-parametric estimation. It is
often necessary to compute the entropy of a mixture, but, in most cases, this
quantity has no closed-form expression, making some form of approximation
necessary. We propose a family of estimators based on a pairwise distance
function between mixture components, and show that this estimator class has
many attractive properties. For many distributions of interest, the proposed
estimators are efficient to compute, differentiable in the mixture parameters,
and become exact when the mixture components are clustered. We prove this
family includes lower and upper bounds on the mixture entropy. The Chernoff
-divergence gives a lower bound when chosen as the distance function,
with the Bhattacharyya distance providing the tightest lower bound for
components that are symmetric and members of a location family. The
Kullback-Leibler divergence gives an upper bound when used as the distance
function. We provide closed-form expressions of these bounds for mixtures of
Gaussians, and discuss their applications to the estimation of mutual
information. We then demonstrate that our bounds are significantly tighter than
well-known existing bounds using numeric simulations. This estimator class is
very useful in optimization problems involving maximization/minimization of
entropy and mutual information, such as MaxEnt and rate distortion problems.Comment: Corrects several errata in published version, in particular in
Section V (bounds on mutual information
The Global Joint Distribution of Income and Health
We investigate the evolution of global welfare in two dimensions: income per capita and life expectancy. First, we estimate the marginal distributions of income and life expectancy separately. More importantly, in contrast to previous univariate approaches, we consider income and life expectancy jointly and estimate their bivariate global distribution for 137 countries during 1970 - 2000. We reach several conclusions: the global joint distribution has evolved from a bimodal into a unimodal one, the evolution of the health distribution has preceded that of income, global inequality and poverty has decreased over time and the evolution of the global distribution has been welfare improving. Our decomposition of overall welfare indicates that global inequality would be underestimated if within-country inequality is not taken into account. Moreover, global inequality and poverty would be substantially underestimated if the dependence between the income and health distributions is ignored.income, health, global distribution, inequality, poverty
The quantification of SIMS depth profiles by Maximum Entropy reconstruction
The quantification procedures applied to raw SIMS data were devised
on the basis of a simple model for the sputtering and ionisation that occur
during measurement. The model and the associated quantification
procedures have long been known to be inaccurate. If SIMS is to remain a
useful analysis tool in the future, the quantification procedures must be
adjusted such that current features of interest are accurately measured.
This thesis describes the development of a more accurate (though
empirical) model for the effects of the analysis, using the convolution integral.
We propose a method for the quantification of SIMS depth profiles
appropriate to this model, using Maximum Entropy (MaxEnt) reconstruction.
SIMS depth profile data differ significantly from previous applications of the
MaxEnt method: the very high signal to background ratio of the technique has
lead users to plot the results on a logarithmic axis, giving much importance to
extremely small signals.
The noise on SIMS depth profiles has been characterised. A number
of optimisation algorithms have been developed and tested, and the
performance of the MaxEnt method on SIMS data has been assessed. A
novel form of the entropy, particularly suited to SIMS depth profiles, has been
suggested. This form has given excellent results
Bounds on mutual information of mixture data for classification tasks
The data for many classification problems, such as pattern and speech
recognition, follow mixture distributions. To quantify the optimum performance
for classification tasks, the Shannon mutual information is a natural
information-theoretic metric, as it is directly related to the probability of
error. The mutual information between mixture data and the class label does not
have an analytical expression, nor any efficient computational algorithms. We
introduce a variational upper bound, a lower bound, and three estimators, all
employing pair-wise divergences between mixture components. We compare the new
bounds and estimators with Monte Carlo stochastic sampling and bounds derived
from entropy bounds. To conclude, we evaluate the performance of the bounds and
estimators through numerical simulations
Uncertainty Quantification of the Responses of Transient Electromagnetic Disturbance Coupling to Transmission Lines Based on Stochastic Models
L'abstract è presente nell'allegato / the abstract is in the attachmen
Efficient Computational Methods for Structural Reliability and Global Sensitivity Analyses
Uncertainty analysis of a system response is an important part of engineering probabilistic analysis. Uncertainty analysis includes: (a) to evaluate moments of the response; (b) to evaluate reliability analysis of the system; (c) to assess the complete probability distribution of the response; (d) to conduct the parametric sensitivity analysis of the output. The actual model of system response is usually a high-dimensional function of input variables. Although Monte Carlo simulation is a quite general approach for this purpose, it may require an inordinate amount of resources to achieve an acceptable level of accuracy. Development of a computationally efficient method, hence, is of great importance.
First of all, the study proposed a moment method for uncertainty quantification of structural systems. However, a key departure is the use of fractional moment of response function, as opposed to integer moment used so far in literature. The advantage of using fractional moment over integer moment was illustrated from the relation of one fractional moment with a couple of integer moments. With a small number of samples to compute the fractional moments, a system output distribution was estimated with the principle of maximum entropy (MaxEnt) in conjunction with the constraints specified in terms of fractional moments. Compared to the classical MaxEnt, a novel feature of the proposed method is that fractional exponent of the MaxEnt distribution is determined through the entropy maximization process, instead of assigned by an analyst in prior.
To further minimize the computational cost of the simulation-based entropy method, a multiplicative dimensional reduction method (M-DRM) was proposed to compute the fractional (integer) moments of a generic function with multiple input variables. The M-DRM can accurately approximate a high-dimensional function as the product of a series low-dimensional functions. Together with the principle of maximum entropy, a novel computational approach was proposed to assess the complete probability distribution of a system output. Accuracy and efficiency of the proposed method for structural reliability analysis were verified by crude Monte Carlo simulation of several examples.
Application of M-DRM was further extended to the variance-based global sensitivity analysis of a system. Compared to the local sensitivity analysis, the variance-based sensitivity index can provide significance information about an input random variable. Since each component variance is defined as a conditional expectation with respect to the system model function, the separable nature of the M-DRM approximation can simplify the high-dimension integrations in sensitivity analysis. Several examples were presented to illustrate the numerical accuracy and efficiency of the proposed method in comparison to the Monte Carlo simulation method.
The last contribution of the proposed study is the development of a computationally efficient method for polynomial chaos expansion (PCE) of a system's response. This PCE model can be later used uncertainty analysis. However, evaluation of coefficients of a PCE meta-model is computational demanding task due to the involved high-dimensional integrations. With the proposed M-DRM, the involved computational cost can be remarkably reduced compared to the classical methods in literature (simulation method or tensor Gauss quadrature method). Accuracy and efficiency of the proposed method for polynomial chaos expansion were verified by considering several practical examples.1 yea
Investigations on the properties and estimation of earth response operators from EM sounding data
Incl. 3 reprints at backAvailable from British Library Document Supply Centre- DSC:D82993 / BLDSC - British Library Document Supply CentreSIGLEGBUnited Kingdo
- …