Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data

We introduce a procedure to infer the interactions among a set of binary variables, based on their sampled frequencies and pairwise correlations. The algorithm builds the clusters of variables contributing most to the entropy of the inferred Ising model, and rejects the small contributions due to the sampling noise. Our procedure successfully recovers benchmark Ising models even at criticality and in the low temperature phase, and is applied to neurobiological data.Comment: Accepted for publication in Physical Review Letters (2011

Measuring thermodynamic length

Thermodynamic length is a metric distance between equilibrium thermodynamic states. Among other interesting properties, this metric asymptotically bounds the dissipation induced by a finite time transformation of a thermodynamic system. It is also connected to the Jensen-Shannon divergence, Fisher information and Rao's entropy differential metric. Therefore, thermodynamic length is of central interest in understanding matter out-of-equilibrium. In this paper, we will consider how to define thermodynamic length for a small system described by equilibrium statistical mechanics and how to measure thermodynamic length within a computer simulation. Surprisingly, Bennett's classic acceptance ratio method for measuring free energy differences also measures thermodynamic length.Comment: 4 pages; Typos correcte

Quantum Correlations in Large-Dimensional States of High Symmetry

In this article, we investigate how quantum correlations behave for the so-called Werner and pseudo-pure families of states. The latter refers to states formed by mixing any pure state with the totally mixed state. We derive closed expressions for the Quantum Discord (QD) and the Relative Entropy of Quantumness (REQ) for these families of states. For Werner states, the classical correlations are seen to vanish in high dimensions while the amount of quantum correlations remain bounded and become independent of whether or not the the state is entangled. For pseudo-pure states, nearly the opposite effect is observed with both the quantum and classical correlations growing without bound as the dimension increases and only as the system becomes more entangled. Finally, we verify that pseudo-pure states satisfy the conjecture of [\textit{Phys. Rev. A} \textbf{84}, 052110 (2011)] which says that the Geometric Measure of Discord (GD) always upper bounds the squared Negativity of the state

Satellite remote sensing for ice sheet research

Potential research applications of satellite data over the terrestrial ice sheets of Greenland and Antarctica are assessed and actions required to ensure acquisition of relevant data and appropriate processing to a form suitable for research purposes are recommended. Relevant data include high-resolution visible and SAR imagery, infrared, passive-microwave and scatterometer measurements, and surface topography information from laser and radar altimeters

Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

Learning deeper convolutional neural networks becomes a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be gained by simply stacking more layers. In this paper, we consider the issue from an information theoretical perspective, and propose a novel method Relay Backpropagation, that encourages the propagation of effective information through the network in training stage. By virtue of the method, we achieved the first place in ILSVRC 2015 Scene Classification Challenge. Extensive experiments on two challenging large scale datasets demonstrate the effectiveness of our method is not restricted to a specific dataset or network architecture. Our models will be available to the research community later.Comment: Technical report for our submissions to the ILSVRC 2015 Scene Classification Challenge, where we won the first plac

Entropy Rate of Diffusion Processes on Complex Networks

The concept of entropy rate for a dynamical process on a graph is introduced. We study diffusion processes where the node degrees are used as a local information by the random walkers. We describe analitically and numerically how the degree heterogeneity and correlations affect the diffusion entropy rate. In addition, the entropy rate is used to characterize complex networks from the real world. Our results point out how to design optimal diffusion processes that maximize the entropy for a given network structure, providing a new theoretical tool with applications to social, technological and communication networks.Comment: 4 pages (APS format), 3 figures, 1 tabl

Quantum superadditivity in linear optics networks: sending bits via multiple access Gaussian channels

We study classical capacity regions of quantum Gaussian multiple access channels (MAC). In classical variants of such channels, whilst some capacity superadditivity-type effects such as the so called {\it water filling effect} may be achieved, a fundamental classical additivity law can still be identified, {\it viz.} adding resources to one sender is never advantageous to other senders in sending their respective information to the receiver. Here, we show that quantum resources allows violation of this law, by providing two illustrative schemes of experimentally feasible Gaussian MACs.Comment: 4 pages, 2 figure

Parameter estimation in pair hidden Markov models

This paper deals with parameter estimation in pair hidden Markov models (pair-HMMs). We first provide a rigorous formalism for these models and discuss possible definitions of likelihoods. The model being biologically motivated, some restrictions with respect to the full parameter space naturally occur. Existence of two different Information divergence rates is established and divergence property (namely positivity at values different from the true one) is shown under additional assumptions. This yields consistency for the parameter in parametrization schemes for which the divergence property holds. Simulations illustrate different cases which are not covered by our results.Comment: corrected typo

Dissipation: The phase-space perspective

We show, through a refinement of the work theorem, that the average dissipation, upon perturbing a Hamiltonian system arbitrarily far out of equilibrium in a transition between two canonical equilibrium states, is exactly given by $= -\Delta F =kT D(\rho\|\widetilde{\rho})= kT$, where $\rho$ and $\widetilde{\rho}$ are the phase space density of the system measured at the same intermediate but otherwise arbitrary point in time, for the forward and backward process. $D(\rho\|\widetilde{\rho})$ is the relative entropy of $\rho$ versus $\widetilde{\rho}$. This result also implies general inequalities, which are significantly more accurate than the second law and include, as a special case, the celebrated Landauer principle on the dissipation involved in irreversible computations.Comment: 4 pages, 3 figures (4 figure files), accepted for PR

Sky maps without anisotropies in the cosmic microwave background are a better fit to WMAP's uncalibrated time ordered data than the official sky maps

The purpose of this reanalysis of the WMAP uncalibrated time ordered data (TOD) was two fold. The first was to reassess the reliability of the detection of the anisotropies in the official WMAP sky maps of the cosmic microwave background (CMB). The second was to assess the performance of a proposed criterion in avoiding systematic error in detecting a signal of interest. The criterion was implemented by testing the null hypothesis that the uncalibrated TOD was consistent with no anisotropies when WMAP's hourly calibration parameters were allowed to vary. It was shown independently for all 20 WMAP channels that sky maps with no anisotropies were a better fit to the TOD than those from the official analysis. The recently launched Planck satellite should help sort out this perplexing result.Comment: 11 pages with 1 figure and 2 tables. Extensively rewritten to explain the research bette