On the Sample Information About Parameter and Prediction

The Bayesian measure of sample information about the parameter, known as Lindley's measure, is widely used in various problems such as developing prior distributions, models for the likelihood functions and optimal designs. The predictive information is defined similarly and used for model selection and optimal designs, though to a lesser extent. The parameter and predictive information measures are proper utility functions and have been also used in combination. Yet the relationship between the two measures and the effects of conditional dependence between the observable quantities on the Bayesian information measures remain unexplored. We address both issues. The relationship between the two information measures is explored through the information provided by the sample about the parameter and prediction jointly. The role of dependence is explored along with the interplay between the information measures, prior and sampling design. For the conditionally independent sequence of observable quantities, decompositions of the joint information characterize Lindley's measure as the sample information about the parameter and prediction jointly and the predictive information as part of it. For the conditionally dependent case, the joint information about parameter and prediction exceeds Lindley's measure by an amount due to the dependence. More specific results are shown for the normal linear models and a broad subfamily of the exponential family. Conditionally independent samples provide relatively little information for prediction, and the gap between the parameter and predictive information measures grows rapidly with the sample size.Comment: Published in at http://dx.doi.org/10.1214/10-STS329 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

A Class of Models for Uncorrelated Random Variables

We consider the class of multivariate distributions that gives the distribution of the sum of uncorrelated random variables by the product of their marginal distributions. This class is defined by a representation of the assumption of sub-independence, formulated previously in terms of the characteristic function and convolution, as a weaker assumption than independence for derivation of the distribution of the sum of random variables. The new representation is in terms of stochastic equivalence and the class of distributions is referred to as the summable uncorrelated marginals (SUM) distributions. The SUM distributions can be used as models for the joint distribution of uncorrelated random variables, irrespective of the strength of dependence between them. We provide a method for the construction of bivariate SUM distributions through linking any pair of identical symmetric probability density functions. We also give a formula for measuring the strength of dependence of the SUM models. A final result shows that under the condition of positive or negative orthant dependence, the SUM property implies independence

Multivariate dynamic information

AbstractThis paper develops measures of information for multivariate distributions when their supports are truncated progressively. The focus is on the joint, marginal, and conditional entropies, and the mutual information for residual life distributions where the support is truncated at the current ages of the components of a system. The current ages of the components induce a joint dynamic into the residual life information measures. Our study of dynamic information measures includes several important bivariate and multivariate lifetime models. We derive entropy expressions for a few models, including Marshall–Olkin bivariate exponential. However, in general, study of the dynamics of residual information measures requires computational techniques or analytical results. A bivariate gamma example illustrates study of dynamic information via numerical integration. The analytical results facilitate studying other distributions. The results are on monotonicity of the residual entropy of a system and on transformations that preserve the monotonicity and the order of entropies between two systems. The results also include a new entropy characterization of the joint distribution of independent exponential random variables

Geographical and socioeconomic inequalities in women and children’s nutritional status in Pakistan in 2011: an analysis of data from a nationally representative survey

Background Pakistan has one of the highest levels of child and maternal undernutrition worldwide, but little information about geographical and socioeconomic inequalities is available. We aimed to analyse anthropometric indicators for childhood and maternal nutrition at a district level in Pakistan and assess the association of nutritional status with food security and maternal and household socioeconomic factors. Methods We used data from the 2011 Pakistan National Nutrition Survey, which included anthropometric measurements for 33 638 children younger than 5 years and 24 826 women of childbearing age. We estimated the prevalences of stunting, wasting, and underweight among children and of underweight, overweight, and obesity in women for all 143 districts of Pakistan using a Bayesian spatial technique. We used a mixed-eff ect linear model to analyse the association of nutritional status with individual and household sociodemographic factors and food security. Findings Stunting prevalence in Pakistan’s districts ranged between 22% (95% credible interval 19–26) and 76% (69–83); the lowest fi gures for wasting and underweight were both less than 2·5% and the highest were 42% (34–50) for wasting and 54% (49–59) for underweight. In 106 districts, more women were overweight than were underweight; in 49 of these districts more women were obese than were underweight. Children were better nourished if their mothers were taller or had higher weight, if they lived in wealthier households, and if their mothers had 10 or more years of education. Severe food insecurity was associated with worse nutritional outcomes for both children and women. Interpretation We noted large social and geographical inequalities in child and maternal nutrition in Pakistan, masked by national and provincial averages. Pakistan is also beginning to face the concurrent challenge of high burden of childhood undernutrition and overweight and obesity among women of reproductive age. Planning, implementation, and evaluation of programmes for food and nutrition should be based on district-level needs and outcomes

Network information and connected correlations

Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate the ``connected information'' terms for several examples, and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.Comment: 4 pages, 3 figure