Estimating Mixture Entropy with Pairwise Distances

Mixture distributions arise in many parametric and non-parametric settings -- for example, in Gaussian mixture models and in non-parametric estimation. It is often necessary to compute the entropy of a mixture, but, in most cases, this quantity has no closed-form expression, making some form of approximation necessary. We propose a family of estimators based on a pairwise distance function between mixture components, and show that this estimator class has many attractive properties. For many distributions of interest, the proposed estimators are efficient to compute, differentiable in the mixture parameters, and become exact when the mixture components are clustered. We prove this family includes lower and upper bounds on the mixture entropy. The Chernoff $\alpha$-divergence gives a lower bound when chosen as the distance function, with the Bhattacharyya distance providing the tightest lower bound for components that are symmetric and members of a location family. The Kullback-Leibler divergence gives an upper bound when used as the distance function. We provide closed-form expressions of these bounds for mixtures of Gaussians, and discuss their applications to the estimation of mutual information. We then demonstrate that our bounds are significantly tighter than well-known existing bounds using numeric simulations. This estimator class is very useful in optimization problems involving maximization/minimization of entropy and mutual information, such as MaxEnt and rate distortion problems.Comment: Corrects several errata in published version, in particular in Section V (bounds on mutual information

Nonlinear Information Bottleneck

Information bottleneck (IB) is a technique for extracting information in one random variable $X$ that is relevant for predicting another random variable $Y$. IB works by encoding $X$ in a compressed "bottleneck" random variable $M$ from which $Y$ can be accurately decoded. However, finding the optimal bottleneck variable involves a difficult optimization problem, which until recently has been considered for only two limited cases: discrete $X$ and $Y$ with small state spaces, and continuous $X$ and $Y$ with a Gaussian joint distribution (in which case optimal encoding and decoding maps are linear). We propose a method for performing IB on arbitrarily-distributed discrete and/or continuous $X$ and $Y$, while allowing for nonlinear encoding and decoding maps. Our approach relies on a novel non-parametric upper bound for mutual information. We describe how to implement our method using neural networks. We then show that it achieves better performance than the recently-proposed "variational IB" method on several real-world datasets

Adverse events following influenza immunization reported by healthcare personnel using active surveillance based on text messages

Studies have demonstrated that healthcare personnel (HCP) have concerns about the potential side effects of trivalent inactivate influenza vaccine (IIV3).1-3 A recent metaanalysis of reasons HCP refuse IIV3 indicates the strongest predictors of vaccine acceptance are belief that the vaccine is safe and belief the vaccine does not cause the disease it is meant to prevent.

(p -Cymene)thioglycollatoruthenium(II) dimer; a complex with an ambi-basic S,O-donor ligand

The title compound was prepared from the (p-cymene)ruthenium chloride dimer and thioglycollic acid. The structure is a centrosymmetric dimer bridged by the soft-base S atoms, with the hard-base O atoms of the carboxylate group chelating to form a five-membered twisted-ring. The coordination of the ruthenium atoms is completed by a η6-p-cymene ligand, giving an 18-electron count. The Ru–S bonds are essentially equal at 2.396(1) Å

Caveats for information bottleneck in deterministic scenarios

Information bottleneck (IB) is a method for extracting information from one random variable $X$ that is relevant for predicting another random variable $Y$. To do so, IB identifies an intermediate "bottleneck" variable $T$ that has low mutual information $I(X;T)$ and high mutual information $I(Y;T)$. The "IB curve" characterizes the set of bottleneck variables that achieve maximal $I(Y;T)$ for a given $I(X;T)$, and is typically explored by maximizing the "IB Lagrangian", $I(Y;T) - \beta I(X;T)$. In some cases, $Y$ is a deterministic function of $X$, including many classification problems in supervised learning where the output class $Y$ is a deterministic function of the input $X$. We demonstrate three caveats when using IB in any situation where $Y$ is a deterministic function of $X$: (1) the IB curve cannot be recovered by maximizing the IB Lagrangian for different values of $\beta$; (2) there are "uninteresting" trivial solutions at all points of the IB curve; and (3) for multi-layer classifiers that achieve low prediction error, different layers cannot exhibit a strict trade-off between compression and prediction, contrary to a recent proposal. We also show that when $Y$ is a small perturbation away from being a deterministic function of $X$, these three caveats arise in an approximate way. To address problem (1), we propose a functional that, unlike the IB Lagrangian, can recover the IB curve in all cases. We demonstrate the three caveats on the MNIST dataset

Pigment analysis by Raman microscopy and portable X-ray fluorescence (pXRF) of thirteenth to fourteenth century illuminations and cuttings from Bologna

Non-destructive pigment analysis by Raman microscopy (RM) and portable X-ray fluorescence (pXRF) has been carried out on some Bolognese illuminations and cuttings chosen to represent the beginnings, evolution and height of Bolognese illuminated manuscript production. Dating to the thirteenth and fourteenth centuries and held in a private collection, the study provides evidence for the pigments generally used in this period. The results, which are compared with those obtained for other north Italian artwork, show the developments in usage of artistic materials and technique. Also addressed in this study is an examination of the respective roles of RM and pXRF analysis in this area of technical art history

Recognition of asset impairment: A Comparison of Recognition Criteria

This report examines the accounting for the impairment of long-lived assets and identifiable intangibles as discussed in the Financial Accounting Standards Board Discussion Memorandum No. 098-B. This report specifically analyzes the recognition criteria to be used in determining when an impairment exists. The economic, permanence, and probability criterion are compared and contrasted to ascertain which method is the most feasible. The relevant information for this study ~came from the discussion memorandum, Big Six accounting firms, published literature, and a national student seminar on the topic. The study revealed a great deal of ambiguity in accounting for assets whose value has diminished due to corporate restructuring. This uncertainty has lead to confusion and mistrust among financial statement users. There is, however, a solution to this problem. The use of the probability criterion in recognizing and recording impairments provides adequate disclosures and timely information.B.A. (Bachelor of Arts