121,902 research outputs found
Relative Entropy Relaxations for Signomial Optimization
Signomial programs (SPs) are optimization problems specified in terms of
signomials, which are weighted sums of exponentials composed with linear
functionals of a decision variable. SPs are non-convex optimization problems in
general, and families of NP-hard problems can be reduced to SPs. In this paper
we describe a hierarchy of convex relaxations to obtain successively tighter
lower bounds of the optimal value of SPs. This sequence of lower bounds is
computed by solving increasingly larger-sized relative entropy optimization
problems, which are convex programs specified in terms of linear and relative
entropy functions. Our approach relies crucially on the observation that the
relative entropy function -- by virtue of its joint convexity with respect to
both arguments -- provides a convex parametrization of certain sets of globally
nonnegative signomials with efficiently computable nonnegativity certificates
via the arithmetic-geometric-mean inequality. By appealing to representation
theorems from real algebraic geometry, we show that our sequences of lower
bounds converge to the global optima for broad classes of SPs. Finally, we also
demonstrate the effectiveness of our methods via numerical experiments
Relative Entropy Relaxations for Signomial Optimization
Signomial programs (SPs) are optimization problems specified in terms of signomials,
which are weighted sums of exponentials composed with linear functionals of a decision variable. SPs
are nonconvex optimization problems in general, and families of NP-hard problems can be reduced
to SPs. In this paper we describe a hierarchy of convex relaxations to obtain successively tighter
lower bounds of the optimal value of SPs. This sequence of lower bounds is computed by solving
increasingly larger-sized relative entropy optimization problems, which are convex programs specified
in terms of linear and relative entropy functions. Our approach relies crucially on the observation
that the relative entropy function, by virtue of its joint convexity with respect to both arguments,
provides a convex parametrization of certain sets of globally nonnegative signomials with efficiently
computable nonnegativity certificates via the arithmetic-geometric-mean inequality. By appealing to
representation theorems from real algebraic geometry, we show that our sequences of lower bounds
converge to the global optima for broad classes of SPs. Finally, we also demonstrate the effectiveness
of our methods via numerical experiments
Optimal scales in weighted networks
The analysis of networks characterized by links with heterogeneous intensity
or weight suffers from two long-standing problems of arbitrariness. On one
hand, the definitions of topological properties introduced for binary graphs
can be generalized in non-unique ways to weighted networks. On the other hand,
even when a definition is given, there is no natural choice of the (optimal)
scale of link intensities (e.g. the money unit in economic networks). Here we
show that these two seemingly independent problems can be regarded as
intimately related, and propose a common solution to both. Using a formalism
that we recently proposed in order to map a weighted network to an ensemble of
binary graphs, we introduce an information-theoretic approach leading to the
least biased generalization of binary properties to weighted networks, and at
the same time fixing the optimal scale of link intensities. We illustrate our
method on various social and economic networks.Comment: Accepted for presentation at SocInfo 2013, Kyoto, 25-27 November 2013
(http://www.socinfo2013.org
A Survey on Soft Subspace Clustering
Subspace clustering (SC) is a promising clustering technology to identify
clusters based on their associations with subspaces in high dimensional spaces.
SC can be classified into hard subspace clustering (HSC) and soft subspace
clustering (SSC). While HSC algorithms have been extensively studied and well
accepted by the scientific community, SSC algorithms are relatively new but
gaining more attention in recent years due to better adaptability. In the
paper, a comprehensive survey on existing SSC algorithms and the recent
development are presented. The SSC algorithms are classified systematically
into three main categories, namely, conventional SSC (CSSC), independent SSC
(ISSC) and extended SSC (XSSC). The characteristics of these algorithms are
highlighted and the potential future development of SSC is also discussed.Comment: This paper has been published in Information Sciences Journal in 201
Entropic multipliers method for langevin diffusion and weighted log sobolev inequalities
In his work about hypocercivity, Villani [18] considers in particular
convergence to equilibrium for the kinetic Langevin process. While his
convergence results in L 2 are given in a quite general setting, convergence in
entropy requires some boundedness condition on the Hessian of the Hamiltonian.
We will show here how to get rid of this assumption in the study of the
hypocoercive entropic relaxation to equilibrium for the Langevin diffusion. Our
method relies on a generalization to entropy of the multipliers method and an
adequate functional inequality. As a byproduct, we also give tractable
conditions for this functional inequality, which is a particular instance of a
weighted logarithmic Sobolev inequality, to hold
A vector quantization approach to universal noiseless coding and quantization
A two-stage code is a block code in which each block of data is coded in two stages: the first stage codes the identity of a block code among a collection of codes, and the second stage codes the data using the identified code. The collection of codes may be noiseless codes, fixed-rate quantizers, or variable-rate quantizers. We take a vector quantization approach to two-stage coding, in which the first stage code can be regarded as a vector quantizer that “quantizes” the input data of length n to one of a fixed collection of block codes. We apply the generalized Lloyd algorithm to the first-stage quantizer, using induced measures of rate and distortion, to design locally optimal two-stage codes. On a source of medical images, two-stage variable-rate vector quantizers designed in this way outperform standard (one-stage) fixed-rate vector quantizers by over 9 dB. The tail of the operational distortion-rate function of the first-stage quantizer determines the optimal rate of convergence of the redundancy of a universal sequence of two-stage codes. We show that there exist two-stage universal noiseless codes, fixed-rate quantizers, and variable-rate quantizers whose per-letter rate and distortion redundancies converge to zero as (k/2)n -1 log n, when the universe of sources has finite dimension k. This extends the achievability part of Rissanen's theorem from universal noiseless codes to universal quantizers. Further, we show that the redundancies converge as O(n-1) when the universe of sources is countable, and as O(n-1+ϵ) when the universe of sources is infinite-dimensional, under appropriate conditions
Reduced perplexity: Uncertainty measures without entropy
Conference paper presented at Recent Advances in Info-Metrics, Washington, DC, 2014. Under review for a book chapter in "Recent innovations in info-metrics: a cross-disciplinary perspective on information and information processing" by Oxford University Press.A simple, intuitive approach to the assessment of probabilistic inferences is introduced. The Shannon information metrics are translated to the probability domain. The translation shows that the negative logarithmic score and the geometric mean are equivalent measures of the accuracy of a probabilistic inference. Thus there is both a quantitative reduction in perplexity as good inference algorithms reduce the uncertainty and a qualitative reduction due to the increased clarity between the original set of inferences and their average, the geometric mean. Further insight is provided by showing that the Renyi and Tsallis entropy functions translated to the probability domain are both the weighted generalized mean of the distribution. The generalized mean of probabilistic inferences forms a Risk Profile of the performance. The arithmetic mean is used to measure the decisiveness, while the -2/3 mean is used to measure the robustness
- …