858 research outputs found

    Improving variational methods via pairwise linear response identities

    Get PDF
    nference methods are often formulated as variational approximations: these approxima-tions allow easy evaluation of statistics by marginalization or linear response, but theseestimates can be inconsistent. We show that by introducing constraints on covariance, onecan ensure consistency of linear response with the variational parameters, and in so doinginference of marginal probability distributions is improved. For the Bethe approximationand its generalizations, improvements are achieved with simple choices of the constraints.The approximations are presented as variational frameworks; iterative procedures relatedto message passing are provided for finding the minim

    Cycle-based Cluster Variational Method for Direct and Inverse Inference

    Get PDF
    We elaborate on the idea that loop corrections to belief propagation could be dealt with in a systematic way on pairwise Markov random fields, by using the elements of a cycle basis to define region in a generalized belief propagation setting. The region graph is specified in such a way as to avoid dual loops as much as possible, by discarding redundant Lagrange multipliers, in order to facilitate the convergence, while avoiding instabilities associated to minimal factor graph construction. We end up with a two-level algorithm, where a belief propagation algorithm is run alternatively at the level of each cycle and at the inter-region level. The inverse problem of finding the couplings of a Markov random field from empirical covariances can be addressed region wise. It turns out that this can be done efficiently in particular in the Ising context, where fixed point equations can be derived along with a one-parameter log likelihood function to minimize. Numerical experiments confirm the effectiveness of these considerations both for the direct and inverse MRF inference.Comment: 47 pages, 16 figure

    A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines

    Full text link
    Restricted Boltzmann machines (RBMs) are energy-based neural-networks which are commonly used as the building blocks for deep architectures neural architectures. In this work, we derive a deterministic framework for the training, evaluation, and use of RBMs based upon the Thouless-Anderson-Palmer (TAP) mean-field approximation of widely-connected systems with weak interactions coming from spin-glass theory. While the TAP approach has been extensively studied for fully-visible binary spin systems, our construction is generalized to latent-variable models, as well as to arbitrarily distributed real-valued spin systems with bounded support. In our numerical experiments, we demonstrate the effective deterministic training of our proposed models and are able to show interesting features of unsupervised learning which could not be directly observed with sampling. Additionally, we demonstrate how to utilize our TAP-based framework for leveraging trained RBMs as joint priors in denoising problems

    Viscosity and scale invariance in the unitary Fermi gas

    Full text link
    We compute the shear viscosity of the unitary Fermi gas above the superfluid transition temperature, using a diagrammatic technique that starts from the exact Kubo formula. The formalism obeys a Ward identity associated with scale invariance which guarantees that the bulk viscosity vanishes identically. For the shear viscosity, vertex corrections and the associated Aslamazov-Larkin contributions are shown to be crucial to reproduce the full Boltzmann equation result in the high-temperature, low fugacity limit. The frequency dependent shear viscosity η(ω)\eta(\omega) exhibits a Drude-like transport peak and a power-law tail at large frequencies which is proportional to the Tan contact. The weight in the transport peak is given by the equilibrium pressure, in agreement with a sum rule due to Taylor and Randeria. Near the superfluid transition the peak width is of the order of 0.5TF0.5 T_F, thus invalidating a quasiparticle description. The ratio η/s\eta/s between the static shear viscosity and the entropy density exhibits a minimum near the superfluid transition temperature whose value is larger than the string theory bound /(4πkB)\hbar/(4\pi k_B) by a factor of about seven.Comment: 34 pages, 9 figures; final form (contains new derivation of sum rule), accepted for publication in Annals of Physic
    corecore