873 research outputs found
Polar Codes: Robustness of the Successive Cancellation Decoder with Respect to Quantization
Polar codes provably achieve the capacity of a wide array of channels under
successive decoding. This assumes infinite precision arithmetic. Given the
successive nature of the decoding algorithm, one might worry about the
sensitivity of the performance to the precision of the computation.
We show that even very coarsely quantized decoding algorithms lead to
excellent performance. More concretely, we show that under successive decoding
with an alphabet of cardinality only three, the decoder still has a threshold
and this threshold is a sizable fraction of capacity. More generally, we show
that if we are willing to transmit at a rate below capacity, then we
need only bits of precision, where is a universal
constant.Comment: In ISIT 201
Near-Optimal Active Learning of Halfspaces via Query Synthesis in the Noisy Setting
In this paper, we consider the problem of actively learning a linear
classifier through query synthesis where the learner can construct artificial
queries in order to estimate the true decision boundaries. This problem has
recently gained a lot of interest in automated science and adversarial reverse
engineering for which only heuristic algorithms are known. In such
applications, queries can be constructed de novo to elicit information (e.g.,
automated science) or to evade detection with minimal cost (e.g., adversarial
reverse engineering). We develop a general framework, called dimension coupling
(DC), that 1) reduces a d-dimensional learning problem to d-1 low dimensional
sub-problems, 2) solves each sub-problem efficiently, 3) appropriately
aggregates the results and outputs a linear classifier, and 4) provides a
theoretical guarantee for all possible schemes of aggregation. The proposed
method is proved resilient to noise. We show that the DC framework avoids the
curse of dimensionality: its computational complexity scales linearly with the
dimension. Moreover, we show that the query complexity of DC is near optimal
(within a constant factor of the optimum algorithm). To further support our
theoretical analysis, we compare the performance of DC with the existing work.
We observe that DC consistently outperforms the prior arts in terms of query
complexity while often running orders of magnitude faster.Comment: Accepted by AAAI 201
How to Achieve the Capacity of Asymmetric Channels
We survey coding techniques that enable reliable transmission at rates that
approach the capacity of an arbitrary discrete memoryless channel. In
particular, we take the point of view of modern coding theory and discuss how
recent advances in coding for symmetric channels help provide more efficient
solutions for the asymmetric case. We consider, in more detail, three basic
coding paradigms.
The first one is Gallager's scheme that consists of concatenating a linear
code with a non-linear mapping so that the input distribution can be
appropriately shaped. We explicitly show that both polar codes and spatially
coupled codes can be employed in this scenario. Furthermore, we derive a
scaling law between the gap to capacity, the cardinality of the input and
output alphabets, and the required size of the mapper.
The second one is an integrated scheme in which the code is used both for
source coding, in order to create codewords distributed according to the
capacity-achieving input distribution, and for channel coding, in order to
provide error protection. Such a technique has been recently introduced by
Honda and Yamamoto in the context of polar codes, and we show how to apply it
also to the design of sparse graph codes.
The third paradigm is based on an idea of B\"ocherer and Mathar, and
separates the two tasks of source coding and channel coding by a chaining
construction that binds together several codewords. We present conditions for
the source code and the channel code, and we describe how to combine any source
code with any channel code that fulfill those conditions, in order to provide
capacity-achieving schemes for asymmetric channels. In particular, we show that
polar codes, spatially coupled codes, and homophonic codes are suitable as
basic building blocks of the proposed coding strategy.Comment: 32 pages, 4 figures, presented in part at Allerton'14 and published
in IEEE Trans. Inform. Theor
Construction of Polar Codes with Sublinear Complexity
Consider the problem of constructing a polar code of block length for the
transmission over a given channel . Typically this requires to compute the
reliability of all the synthetic channels and then to include those that
are sufficiently reliable. However, we know from [1], [2] that there is a
partial order among the synthetic channels. Hence, it is natural to ask whether
we can exploit it to reduce the computational burden of the construction
problem.
We show that, if we take advantage of the partial order [1], [2], we can
construct a polar code by computing the reliability of roughly a fraction
of the synthetic channels. In particular, we prove that
is a lower bound on the number of synthetic channels to be
considered and such a bound is tight up to a multiplicative factor . This set of roughly synthetic channels is universal, in
the sense that it allows one to construct polar codes for any , and it can
be identified by solving a maximum matching problem on a bipartite graph.
Our proof technique consists of reducing the construction problem to the
problem of computing the maximum cardinality of an antichain for a suitable
partially ordered set. As such, this method is general and it can be used to
further improve the complexity of the construction problem in case a new
partial order on the synthetic channels of polar codes is discovered.Comment: 9 pages, 3 figures, presented at ISIT'17 and submitted to IEEE Trans.
Inform. Theor
The Space of Solutions of Coupled XORSAT Formulae
The XOR-satisfiability (XORSAT) problem deals with a system of Boolean
variables and clauses. Each clause is a linear Boolean equation (XOR) of a
subset of the variables. A -clause is a clause involving distinct
variables. In the random -XORSAT problem a formula is created by choosing
-clauses uniformly at random from the set of all possible clauses on
variables. The set of solutions of a random formula exhibits various
geometrical transitions as the ratio varies.
We consider a {\em coupled} -XORSAT ensemble, consisting of a chain of
random XORSAT models that are spatially coupled across a finite window along
the chain direction. We observe that the threshold saturation phenomenon takes
place for this ensemble and we characterize various properties of the space of
solutions of such coupled formulae.Comment: Submitted to ISIT 201
- …