64 research outputs found
Linear Programming Decoding of Spatially Coupled Codes
For a given family of spatially coupled codes, we prove that the LP threshold
on the BSC of the graph cover ensemble is the same as the LP threshold on the
BSC of the derived spatially coupled ensemble. This result is in contrast with
the fact that the BP threshold of the derived spatially coupled ensemble is
believed to be larger than the BP threshold of the graph cover ensemble as
noted by the work of Kudekar et al. (2011, 2012). To prove this, we establish
some properties related to the dual witness for LP decoding which was
introduced by Feldman et al. (2007) and simplified by Daskalakis et al. (2008).
More precisely, we prove that the existence of a dual witness which was
previously known to be sufficient for LP decoding success is also necessary and
is equivalent to the existence of certain acyclic hyperflows. We also derive a
sublinear (in the block length) upper bound on the weight of any edge in such
hyperflows, both for regular LPDC codes and for spatially coupled codes and we
prove that the bound is asymptotically tight for regular LDPC codes. Moreover,
we show how to trade crossover probability for "LP excess" on all the variable
nodes, for any binary linear code.Comment: 37 pages; Added tightness construction, expanded abstrac
Communication Complexity of Permutation-Invariant Functions
Motivated by the quest for a broader understanding of communication
complexity of simple functions, we introduce the class of
"permutation-invariant" functions. A partial function is permutation-invariant if for every bijection
and every , it is the case that . Most of the commonly studied functions
in communication complexity are permutation-invariant. For such functions, we
present a simple complexity measure (computable in time polynomial in given
an implicit description of ) that describes their communication complexity
up to polynomial factors and up to an additive error that is logarithmic in the
input size. This gives a coarse taxonomy of the communication complexity of
simple functions. Our work highlights the role of the well-known lower bounds
of functions such as 'Set-Disjointness' and 'Indexing', while complementing
them with the relatively lesser-known upper bounds for 'Gap-Inner-Product'
(from the sketching literature) and 'Sparse-Gap-Inner-Product' (from the recent
work of Canonne et al. [ITCS 2015]). We also present consequences to the study
of communication complexity with imperfectly shared randomness where we show
that for total permutation-invariant functions, imperfectly shared randomness
results in only a polynomial blow-up in communication complexity after an
additive overhead
The Power of Shared Randomness in Uncertain Communication
In a recent work (Ghazi et al., SODA 2016), the authors with Komargodski and Kothari initiated the study of communication with contextual uncertainty, a setup aiming to understand how efficient communication is possible when the communicating parties imperfectly share a huge context. In this setting, Alice is given a function f and an input string x, and Bob is given a function g and an input string y. The pair (x,y) comes from a known distribution mu and f and g are guaranteed to be close under this distribution. Alice and Bob wish to compute g(x,y) with high probability. The lack of agreement between Alice and Bob on the function that is being computed captures the uncertainty in the context. The previous work showed that any problem with one-way communication complexity k in the standard model (i.e., without uncertainty, in other words, under the promise that f=g) has public-coin communication at most O(k(1+I)) bits in the uncertain case, where I is the mutual information between x and y. Moreover, a lower bound of Omega(sqrt{I}) bits on the public-coin uncertain communication was also shown.
However, an important question that was left open is related to the power that public randomness brings to uncertain communication. Can Alice and Bob achieve efficient communication amid uncertainty without using public randomness? And how powerful are public-coin protocols in overcoming uncertainty? Motivated by these two questions:
- We prove the first separation between private-coin uncertain communication and public-coin uncertain communication. Namely, we exhibit a function class for which the communication in the standard model and the public-coin uncertain communication are O(1) while the private-coin uncertain communication is a growing function of n (the length of the inputs). This lower bound (proved with respect to the uniform distribution) is in sharp contrast with the case of public-coin uncertain communication which was shown by the previous work to be within a constant factor from the certain communication. This lower bound also implies the first separation between public-coin uncertain communication and deterministic uncertain communication. Interestingly, we also show that if Alice and Bob imperfectly share a sequence of random bits (a setup weaker than public randomness), then achieving a constant blow-up in communication is still possible.
- We improve the lower-bound of the previous work on public-coin uncertain communication. Namely, we exhibit a function class and a distribution (with mutual information I approx n) for which the one-way certain communication is k bits but the one-way public-coin uncertain communication is at least Omega(sqrt{k}*sqrt{I}) bits.
Our proofs introduce new problems in the standard communication complexity model and prove lower bounds for these problems. Both the problems and the lower bound techniques may be of general interest
LP/SDP Hierarchy Lower Bounds for Decoding Random LDPC Codes
Random (dv,dc)-regular LDPC codes are well-known to achieve the Shannon
capacity of the binary symmetric channel (for sufficiently large dv and dc)
under exponential time decoding. However, polynomial time algorithms are only
known to correct a much smaller fraction of errors. One of the most powerful
polynomial-time algorithms with a formal analysis is the LP decoding algorithm
of Feldman et al. which is known to correct an Omega(1/dc) fraction of errors.
In this work, we show that fairly powerful extensions of LP decoding, based on
the Sherali-Adams and Lasserre hierarchies, fail to correct much more errors
than the basic LP-decoder. In particular, we show that:
1) For any values of dv and dc, a linear number of rounds of the
Sherali-Adams LP hierarchy cannot correct more than an O(1/dc) fraction of
errors on a random (dv,dc)-regular LDPC code.
2) For any value of dv and infinitely many values of dc, a linear number of
rounds of the Lasserre SDP hierarchy cannot correct more than an O(1/dc)
fraction of errors on a random (dv,dc)-regular LDPC code.
Our proofs use a new stretching and collapsing technique that allows us to
leverage recent progress in the study of the limitations of LP/SDP hierarchies
for Maximum Constraint Satisfaction Problems (Max-CSPs). The problem then
reduces to the construction of special balanced pairwise independent
distributions for Sherali-Adams and special cosets of balanced pairwise
independent subgroups for Lasserre.
Some of our techniques are more generally applicable to a large class of
Boolean CSPs called Min-Ones. In particular, for k-Hypergraph Vertex Cover, we
obtain an improved integrality gap of that holds after a
\emph{linear} number of rounds of the Lasserre hierarchy, for any k = q+1 with
q an arbitrary prime power. The best previous gap for a linear number of rounds
was equal to and due to Schoenebeck.Comment: 23 page
Communication with Contextual Uncertainty
We introduce a simple model illustrating the role of context in communication
and the challenge posed by uncertainty of knowledge of context. We consider a
variant of distributional communication complexity where Alice gets some
information and Bob gets , where is drawn from a known
distribution, and Bob wishes to compute some function (with high
probability over ). In our variant, Alice does not know , but only
knows some function which is an approximation of . Thus, the function
being computed forms the context for the communication, and knowing it
imperfectly models (mild) uncertainty in this context.
A naive solution would be for Alice and Bob to first agree on some common
function that is close to both and and then use a protocol for
to compute . We show that any such agreement leads to a large overhead
in communication ruling out such a universal solution.
In contrast, we show that if has a one-way communication protocol with
complexity in the standard setting, then it has a communication protocol
with complexity in the uncertain setting, where denotes
the mutual information between and . In the particular case where the
input distribution is a product distribution, the protocol in the uncertain
setting only incurs a constant factor blow-up in communication and error.
Furthermore, we show that the dependence on the mutual information is
required. Namely, we construct a class of functions along with a non-product
distribution over for which the communication complexity is a single
bit in the standard setting but at least bits in the
uncertain setting.Comment: 20 pages + 1 title pag
On the Power of Learning from k-Wise Queries
Several well-studied models of access to data samples, including statistical queries, local differential privacy and low-communication algorithms rely on queries that provide information about a function of a single sample. (For example, a statistical query (SQ) gives an estimate of Ex_{x ~ D}[q(x)] for any choice of the query function q mapping X to the reals, where D is
an unknown data distribution over X.) Yet some data analysis algorithms rely on properties of functions that depend on multiple samples. Such algorithms would be naturally implemented using k-wise queries each of which is specified by a function q mapping X^k to the reals. Hence it is natural to ask whether algorithms using k-wise queries can solve learning problems more efficiently and by how much.
Blum, Kalai and Wasserman (2003) showed that for any weak PAC learning problem over a fixed distribution, the complexity of learning with k-wise SQs is smaller than the (unary) SQ complexity by a factor of at most 2^k. We show that for more general problems over distributions the picture is substantially richer. For every k, the complexity of distribution-independent PAC learning with k-wise queries can be exponentially larger than learning with (k+1)-wise queries. We then give two approaches for simulating a k-wise query using unary queries. The first approach exploits the structure of the
problem that needs to be solved. It generalizes and strengthens (exponentially)
the results of Blum et al.. It allows us to derive strong lower bounds for
learning DNF formulas and stochastic constraint satisfaction problems that hold
against algorithms using k-wise queries. The second approach exploits the
k-party communication complexity of the k-wise query function
Computational aspects of communication amid uncertainty
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 203-215).This thesis focuses on the role of uncertainty in communication and effective (computational) methods to overcome uncertainty. A classical form of uncertainty arises from errors introduced by the communication channel but uncertainty can arise in many other ways if the communicating players do not completely know (or understand) each other. For example, it can occur as mismatches in the shared randomness used by the distributed agents, or as ambiguity in the shared context or goal of the communication. We study many modern models of uncertainty, some of which have been considered in the literature but are not well-understood, while others are introduced in this thesis: Uncertainty in Shared Randomness -- We study common randomness and secret key generation. In common randomness generation, two players are given access to correlated randomness and are required to agree on pure random bits while minimizing communication and maximizing agreement probability. Secret key generation refers to the setup where, in addition, the generated random key is required to be secure against any eavesdropper. These setups are of significant importance in information theory and cryptography. We obtain the first explicit and sample-efficient schemes with the optimal trade-offs between communication, agreement probability and entropy of generated common random bits, in the one-way communication setting. -- We obtain the first decidability result for the computational problem of the noninteractive simulation of joint distributions, which asks whether two parties can convert independent identically distributed samples from a given source of correlation into another desired form of correlation. This class of problems has been well-studied in information theory and its computational complexity has been wide open. Uncertainty in Goal of Communication -- We introduce a model for communication with functional uncertainty. In this setup, we consider the classical model of communication complexity of Yao, and study how this complexity changes if the function being computed is not completely known to both players. This forms a mathematical analogue of a natural situation in human communication: Communicating players do not a priori know what the goal of communication is. We design efficient protocols for dealing with uncertainty in this model in a broad setting. Our solution relies on public random coins being shared by the communicating players. We also study the question of relaxing this requirement and present several results answering different aspects of this question. Uncertainty in Prior Distribution -- We study data compression in a distributed setting where several players observe messages from an unknown distribution, which they wish to encode, communicate and decode. In this setup, we design and analyze a simple, decentralized and efficient protocol. In this thesis, we study these various forms of uncertainty, and provide novel solutions using tools from various areas of theoretical computer science, information theory and mathematics."This research was supported in part by an NSF STC Award CCF 0939370, NSF award numbers CCF-1217423, CCF-1650733 and CCF-1420692, an Irwin and Joan Jacobs Presidential Fellowship and an IBM Ph.D. Fellowship"--Page 7.by Badih Ghazi.Ph. D
Pure-DP Aggregation in the Shuffle Model: Error-Optimal and Communication-Efficient
We obtain a new protocol for binary counting in the -shuffle-DP
model with error and expected communication
messages per user. Previous
protocols incur either an error of with
messages per user (Ghazi et al., ITC 2020) or an error
of with messages per user (Cheu and
Yan, TPDP 2022). Using the new protocol, we obtained improved
-shuffle-DP protocols for real summation and histograms
- …