17,251 research outputs found
Compressing Probability Distributions
We show how to store good approximations of probability distributions in
small space
A Method for Compressing Parameters in Bayesian Models with Application to Logistic Sequence Prediction Models
Bayesian classification and regression with high order interactions is
largely infeasible because Markov chain Monte Carlo (MCMC) would need to be
applied with a great many parameters, whose number increases rapidly with the
order. In this paper we show how to make it feasible by effectively reducing
the number of parameters, exploiting the fact that many interactions have the
same values for all training cases. Our method uses a single ``compressed''
parameter to represent the sum of all parameters associated with a set of
patterns that have the same value for all training cases. Using symmetric
stable distributions as the priors of the original parameters, we can easily
find the priors of these compressed parameters. We therefore need to deal only
with a much smaller number of compressed parameters when training the model
with MCMC. The number of compressed parameters may have converged before
considering the highest possible order. After training the model, we can split
these compressed parameters into the original ones as needed to make
predictions for test cases. We show in detail how to compress parameters for
logistic sequence prediction models. Experiments on both simulated and real
data demonstrate that a huge number of parameters can indeed be reduced by our
compression method.Comment: 29 page
Cryptography from Information Loss
© Marshall Ball, Elette Boyle, Akshay Degwekar, Apoorvaa Deshpande, Alon Rosen, Vinod. Reductions between problems, the mainstay of theoretical computer science, efficiently map an instance of one problem to an instance of another in such a way that solving the latter allows solving the former.1 The subject of this work is “lossy” reductions, where the reduction loses some information about the input instance. We show that such reductions, when they exist, have interesting and powerful consequences for lifting hardness into “useful” hardness, namely cryptography. Our first, conceptual, contribution is a definition of lossy reductions in the language of mutual information. Roughly speaking, our definition says that a reduction C is t-lossy if, for any distribution X over its inputs, the mutual information I(X; C(X)) ≤ t. Our treatment generalizes a variety of seemingly related but distinct notions such as worst-case to average-case reductions, randomized encodings (Ishai and Kushilevitz, FOCS 2000), homomorphic computations (Gentry, STOC 2009), and instance compression (Harnik and Naor, FOCS 2006). We then proceed to show several consequences of lossy reductions: 1. We say that a language L has an f-reduction to a language L0 for a Boolean function f if there is a (randomized) polynomial-time algorithm C that takes an m-tuple of strings X = (x1, . . ., xm), with each xi ∈ {0, 1}n, and outputs a string z such that with high probability, L0(z) = f(L(x1), L(x2), . . ., L(xm)) Suppose a language L has an f-reduction C to L0 that is t-lossy. Our first result is that one-way functions exist if L is worst-case hard and one of the following conditions holds: f is the OR function, t ≤ m/100, and L0 is the same as L f is the Majority function, and t ≤ m/100 f is the OR function, t ≤ O(m log n), and the reduction has no error This improves on the implications that follow from combining (Drucker, FOCS 2012) with (Ostrovsky and Wigderson, ISTCS 1993) that result in auxiliary-input one-way functions. 2. Our second result is about the stronger notion of t-compressing f-reductions – reductions that only output t bits. We show that if there is an average-case hard language L that has a t-compressing Majority reduction to some language for t = m/100, then there exist collision-resistant hash functions. This improves on the result of (Harnik and Naor, STOC 2006), whose starting point is a cryptographic primitive (namely, one-way functions) rather than average-case hardness, and whose assumption is a compressing OR-reduction of SAT (which is now known to be false unless the polynomial hierarchy collapses). Along the way, we define a non-standard one-sided notion of average-case hardness, which is the notion of hardness used in the second result above, that may be of independent interest
The work value of information
We present quantitative relations between work and information that are valid
both for finite sized and internally correlated systems as well in the
thermodynamical limit. We suggest work extraction should be viewed as a game
where the amount of work an agent can extract depends on how well it can guess
the micro-state of the system. In general it depends both on the agent's
knowledge and risk-tolerance, because the agent can bet on facts that are not
certain and thereby risk failure of the work extraction. We derive strikingly
simple expressions for the extractable work in the extreme cases of effectively
zero- and arbitrary risk tolerance respectively, thereby enveloping all cases.
Our derivation makes a connection between heat engines and the smooth entropy
approach. The latter has recently extended Shannon theory to encompass finite
sized and internally correlated bit strings, and our analysis points the way to
an analogous extension of statistical mechanics.Comment: 5 pages, 4 figure
Intrinsic viscosity of a suspension of weakly Brownian ellipsoids in shear
We analyze the angular dynamics of triaxial ellipsoids in a shear flow
subject to weak thermal noise. By numerically integrating an overdamped angular
Langevin equation, we find the steady angular probability distribution for a
range of triaxial particle shapes. From this distribution we compute the
intrinsic viscosity of a dilute suspension of triaxial particles. We determine
how the viscosity depends on particle shape in the limit of weak thermal noise.
While the deterministic angular dynamics depends very sensitively on particle
shape, we find that the shape dependence of the intrinsic viscosity is weaker,
in general, and that suspensions of rod-like particles are the most sensitive
to breaking of axisymmetry. The intrinsic viscosity of a dilute suspension of
triaxial particles is smaller than that of a suspension of axisymmetric
particles with the same volume, and the same ratio of major to minor axis
lengths.Comment: 14 pages, 6 figures, 1 table, revised versio
Large Alphabets and Incompressibility
We briefly survey some concepts related to empirical entropy -- normal
numbers, de Bruijn sequences and Markov processes -- and investigate how well
it approximates Kolmogorov complexity. Our results suggest th-order
empirical entropy stops being a reasonable complexity metric for almost all
strings of length over alphabets of size about when surpasses
- …