The multivariate covering lemma states that given a collection of $k$
codebooks, each of sufficiently large cardinality and independently generated
according to one of the marginals of a joint distribution, one can always
choose one codeword from each codebook such that the resulting $k$-tuple of
codewords is jointly typical with respect to the joint distribution. We give a
proof of this lemma for weakly typical sets. This allows achievability proofs
that rely on the covering lemma to go through for continuous channels (e.g.,
Gaussian) without the need for quantization. The covering lemma and its
converse are widely used in information theory, including in rate-distortion
theory and in achievability results for multi-user channels.Comment: 10 page

Effros, Michelle

Langberg, Michael

Noorzad, Parham

English

arXiv

The multivariate covering lemma states that given a collection of k codebooks, each of sufficiently large cardinality and independently generated according to one of the marginals of a joint distribution, one can always choose one codeword from each codebook such that the resulting k-tuple of codewords is jointly typical with respect to the joint distribution. We give a proof of this lemma for weakly typical sets. This allows achievability proofs that rely on the covering lemma to go through for continuous channels (e.g., Gaussian) without the need for quantization. The covering lemma and its converse are widely used in information theory, including in rate-distortion theory and in achievability results for multi-user channels

Caltech Authors - Main

ar
X
iv
:1
50
8.
03
34
9v
2 
 [
cs
.I
T
] 
 2
1 
Ja
n 
20
16
THE MULTIVARIATE COVERING LEMMA
AND ITS CONVERSE
PARHAM NOORZAD, MICHELLE EFFROS, AND MICHAEL LANGBERG
Abstract. The multivariate covering lemma states that given a collection of
k codebooks, each of sufficiently large cardinality and independently generated
according to one of the marginals of a joint distribution, one can with prob-
ability arbitrarily close to one choose one codeword from each codebook such
that the resulting k-tuple of codewords is jointly typical with respect to the
joint distribution. Prior proofs of the multivariate covering lemma primarily
employ strong typicality. We give a proof of this lemma for weakly typical sets.
This allows achievability proofs that rely on the covering lemma to go through
for continuous (e.g., Gaussian) channels without the need for quantization.
The covering lemma and its converse are widely used in information theory,
including in rate-distortion theory and in achievability results for multi-user
channels.
1. Introduction
The covering lemma and its extensions play a crucial role in achievability results
in network information theory. Covering lemmas are useful for enabling network
nodes to transmit codewords that “look like” they are generated from a dependent
distribution, whereas in reality, they are carefully selected from sufficiently large
codebooks that are independently generated. This allows nodes to obtain the ben-
efits of both independent and dependent codewords: like independent codewords,
such codewords can be decoded in different locations; like dependent codewords
they have the potential to achieve rates higher than those achieved by independent
codewords. This benefit, however, comes at a cost in rate. Thus the strategy is
useful when the benefit transmitting dependent codewords exceeds its cost.
In the context of the covering lemma, the concept of “looking like” dependent
codewords is captured by the notion of being jointly typical with respect to a
dependent distribution. As there are various ways to define the typical set (here
we specifically focus on weakly typical [2] and strongly typical sets [3]), one may
ask whether a specific version of the covering lemma holds for a given definition of
the typical set. The weakly typical set has two advantages over the strongly typical
set. First, it is easily defined for continuous (e.g., Gaussian) distributions. Second,
the weakly typical set has a simple one-shot counterpart, which allows proofs using
the weakly typical set to be written in the one-shot framework in a simple manner.
On the other hand, some results hold for the strongly typical set that do not hold
for the weakly typical set. Thus it is helpful to review the covering lemma and its
extensions and see for which definition of the typical set each result is currently
known to hold.
The simplest case of the covering lemma is the situation where given a random
vector and an independently generated codebook, a node looks for a codeword in
1
THE MULTIVARIATE COVERING LEMMA 2
the codebook that is jointly typical (with respect to a dependent distribution) with
the given random vector. The result obtained in this case, simply referred to as the
“covering lemma”, appears in the achievability proof of the rate distortion theorem
using weakly typical sets [2]. The second case, called the “mutual covering lemma,”
treats the case where given two independently generated codebooks, a node looks
for a jointly typical pair of codewords, where each codeword is from one of the
codebooks. This result is used in Marton’s inner bound for the two-user broadcast
channel and is proved for strongly typical sets [4, 7]. Recently, by extending the
proof of [2], the authors of [6, 8] prove a one-shot version of the mutual covering
lemma. This proof can be used to show the validity of the mutual covering lemma
for weakly typical sets in the asymptotic setting. The proof in [6, 8], however,
requires stronger independence assumptions on the codebooks than the proof using
strongly typical sets in [3, 4]. Finally, the “multivariate covering lemma” is the
extension of the mutual covering lemma to k independently generated codebooks,
and can be used to obtain an inner bound on the broadcast channel with k users
[3]. As stated in [3], one can show this result holds for strongly typical sets by
extending the proof of the mutual covering lemma [4].
In this work, using the general strategy of El Gamal and Van der Meulen [4]
and some ideas regarding weakly typical sets from Koetter, Effros, and Médard
[5], we give a proof of the multivariate covering lemma for weakly typical sets.
We also provide a converse, a special case of which is usually referred to as the
packing lemma [3]. We remark that while similar to the argument in [4], we use
Chebyshev’s inequality for the direct result (Section 4), it is also possible to use
the Cauchy-Schwarz inequality (see Appendix A), which leads to a more accurate
upper bound.
2. Problem Statement
For every positive integer n, define the set [n] = {1, . . . , n}. Let k be a positive
integer and
p(u0, u1, . . . , uk, uk+1)
be a probability distribution on the set
k+1
∏
j=0
Uj .
For every nonempty S ⊆ [k] define
US =
∏
j∈S
Uj .
For every j ∈ [k], let Mj be a nonnegative integer. For every nonempty S ⊆ [k],
define the set MS as
MS =
∏
j∈S
[Mj].
and let M = M[k]. For every m = (m1, . . . ,mk) ∈ M, let the random vector
(U0, U1(m1), . . . , Uk(mk), Uk+1)
have distribution
p(u0)
k+1
∏
j=1
p(uj|u0),
THE MULTIVARIATE COVERING LEMMA 3
where p(u0) and each p(uj|u0) are the conditional marginals of p(u0, . . . , uk+1). In
addition, let F be an arbitrary subset of U0 × U[k+1]. We want to find upper and
lower bounds on the probability
P
{
∀m ∈ M :
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
/∈ F
}
.
We derive the lower bound (Section 3) using the union bound, which does not
depend on the statistical dependencies of the vectors
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
for different values of m. For the upper bound (Section 4), which leads to the
multivariate covering lemma, we require a stronger assumption, which we next
describe.
Let m = (mj)j∈[k] and m
′ = (m′j)j∈[k] be in M. Define the set Sm,m′ as
Sm,m′ =
{
j ∈ [k] : mj = m
′
j
}
.
When m and m′ are clear from context, we denote Sm,m′ with S. In the proof of
the upper bound we require
P
{
∀j ∈ [k] : Uj(mj) = uj and Uj(m
′
j) = u
′
j
∣
∣U0 = u0, Uk+1 = uk+1
}
=
k
∏
j=1
p(uj|u0)×
∏
j∈Sc
p(u′j |u0),
for all u0 and all (uj)j and (u
′
j)j such that if j ∈ S, then uj = u
′
j (Assumption I).
Note that if there exists a j ∈ S where uj 6= u
′
j then the probability on the left
hand side equals zero.
In the corresponding asymptotic problem (Section 5), we apply our bounds to
P
{
∀m :
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
/∈ A
(n)
δ
}
,
where for every m,
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
is simply n i.i.d. copies of the original random vector
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
,
(Assumption II) and A
(n)
δ is the weakly typical set for the distribution p(u0, u1, . . . , uk, uk+1).
Our main result follows.
Theorem 1 (Multivariate Covering Lemma). Suppose Assumptions (I) and (II)
hold for the joint distribution of
Un0 ,
{
Un1 (m1), . . . , U
n
k (mk)
}
m
, Unk+1.
For the direct part, suppose for all j ∈ [k], Mj ≥ enRj . If for all nonempty S ⊆ [k],
∑
j∈S
Rj >
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1) + (8k − 2|S|+ 10)δ, (1)
then
lim
n→∞
P
{
∃m :
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
∈ A
(n)
δ
}
= 1. (2)
THE MULTIVARIATE COVERING LEMMA 4
For the converse, assume for all j ∈ [k], Mj ≤ enRj . If Equation (2) holds, then
∑
j∈S
Rj ≥
∑
j∈S
H(Uj |U0)−H(US|U0, Uk+1)− 2(|S|+ 1)δ,
for all nonempty S ⊆ [k].
In the direct part of Theorem 1, we can weaken the lower bound on
∑
j∈S Rj
when S = [k]. Specifically, we can replace Equation (1) with
k
∑
j=1
Rj >
k
∑
j=1
H(Uj |U0)−H(U[k]|U0, Uk+1) + 2(k + 1)δ.
for S = [k].
3. The Lower Bound
For every S ⊆ [k], define FS as the projection of F on U0 × US × Uk+1. Then
for every (u0, uS , uk+1) ∈ FS , let F(u0, uS , uk+1) be the set of all uSc such that
(u0, u[k], uk+1) ∈ F . In addition, for every nonempty S ⊆ [k], let αS and βS be
constants such that
αS ≤ log
p(uS |u0, uk+1)
∏
j∈S p(uj |u0)
for all (u0, uS , uk+1) ∈ FS and
βS ≤ log
p(uS |u0, uSc , uk+1)
∏
j∈S p(uj |u0)
for all (u0, uS , uSc , uk+1) ∈ F . Furthermore, let the constant γ satisfy
γ ≥ log
p(u[k]|u0, uk+1)
∏
j∈[k] p(uj |u0)
for all (u0, u[k], uk+1) ∈ F .
For every m = (m1, . . . ,mk) ∈ M, define the random variable Zm as
Zm = 1
{
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
∈ F
}
and set
Z =
∑
m∈M
Zm.
Our aim is to find a lower bound for P{Z = 0}. Note that for every nonempty
S ⊆ [k],
P
{
∃m : Zm = 1
}
= P
{
∃m :
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
∈ F
}
≤ P
{
∃m :
(
U0,
(
Uj(mj)
)
j∈S
, Uk+1
)
∈ FS
}
≤ |MS |
∑
FS
p(u0, uk+1)
∏
j∈S
p(uj |u0)
≤ |MS |e
−αS
∑
FS
p(u0, uS , uk+1)
≤ |MS |e
−αS .
THE MULTIVARIATE COVERING LEMMA 5
Thus
P{Z = 0} = 1−P
{
∃m : Zm = 1
}
≥ 1− min
|S|6=∅
|MS |e
−αS . (3)
4. The Upper Bound
In deriving our upper bound on P{Z = 0}, we apply conditioning and Cheby-
shev’s inequality. Thus, the factor
1
(
P{F(u0, uk+1)}
)2
appears, where
P{F(u0, uk+1)} = P
{
U[k] ∈ F(u0, uk+1)|U0 = u0, Uk+1 = uk+1
}
=
∑
u[k]∈F(u0,uk+1)
p(u[k]|u0, uk+1)
and F(u0, uk+1) (Section 3) is simply the set of all u[k]’s that satisfy (u0, u[k], uk+1) ∈
F . Thus to get a reasonably accurate upper bound, we require P{F(u0, uk+1)} to
be large. However, as we cannot guarantee this for all (u0, uk+1), we partition the
(u0, uk+1) pairs into “good” and “bad” sets, corresponding to large and small val-
ues of P{F(u0, uk+1)}, respectively. The probability of the good set is large when
P{(U0, U[k], Uk+1) ∈ F} is sufficiently large. To see this, fix ǫ > 0 and following
Appendix III of [5], define the set G ⊆ U0 × Uk+1 as
G =
{
(u0, uk+1) : P{F(u0, uk+1)} ≥ 1− ǫ
}
,
Note that G is the set of all good (u0, uk+1) pairs as defined above. We have
P
{
(U0, U[k], Uk+1) ∈ F
}
=
∑
u0,uk+1
∑
u[k]∈F(u0,uk+1)
p(u0, uk+1)p(u[k]|u0, uk+1)
=
∑
u0,uk+1
p(u0, uk+1)P{F(u0, uk+1)}
≤ (1− ǫ)P{(U0, Uk+1) /∈ G}+P{(U0, Uk+1) ∈ G}
= 1− ǫP{(U0, Uk+1) /∈ G}.
Thus
P{(U0, Uk+1) /∈ G} ≤
1
ǫ
P
{
(U0, U[k], Uk+1) /∈ F
}
. (4)
Our aim is to find an upper bound for P{Z = 0}. To do this, we write
P{Z = 0} =
∑
u0,uk+1
p(u0, uk+1)P{Z = 0|u0, uk+1}
≤
1
ǫ
P
{
(U0, U[k], Uk+1) /∈ F
}
+
∑
(u0,uk+1)∈G
p(u0, uk+1)P{Z = 0|u0, uk+1},
(5)
where the inequality follows from Equation (4). Therefore, to find an upper bound
on P{Z = 0}, it suffices to find an upper bound on P{Z = 0|U0 = u0, , Uk+1 =
uk+1} for all (u0, uk+1) ∈ G. Fix (u0, uk+1) ∈ G. We use Chebyshev’s inequality
to find an upper bound on P{Z = 0|U0 = u0, Uk+1 = uk+1}. Thus we need to
THE MULTIVARIATE COVERING LEMMA 6
calculate E[Z|U0 = u0, Uk+1 = uk+1] and E[Z2|U0 = u0, Uk+1 = uk+1]. For a given
m, from the definition of γ (Section 3) it follows
E[Zm|u0, uk+1] = P
{
(
U1(m1), . . . , Uk(mk)
)
∈ F(u0, uk+1)
∣
∣u0, uk+1
}
=
∑
F(u0,uk+1)
p(u1|u0) . . . p(uk|u0)
≥
∑
F(u0,uk+1)
e−γp(u[k]|u0, uk+1)
= e−γ P{F(u0, uk+1)} ≥ (1 − ǫ)e
−γ .
where the last inequality follows from the fact that (u0, uk+1) ∈ G. Thus, by
linearity of expectation,
E[Z|U0 = u0, Uk+1 = uk+1] ≥ |M|e
−γ(1− ǫ). (6)
Next, we find an upper bound on E[Z2|U0 = u0, Uk+1 = uk+1]. We have
Z2 =
∑
m
Z2
m
+
∑
m6=m′
ZmZm′ = Z +
∑
m6=m′
ZmZm′ ,
since Z2
m
= Zm and Z =
∑
m
Zm. Thus
E[Z2|u0, uk+1] = E[Z|u0, uk+1] + E
[
∑
m6=m′
ZmZm′
∣
∣u0, uk+1
]
For any pair of distinct m and m′ with nonempty S = Sm,m′ , we have
E
[
ZmZm′ |u0, uk+1
]
=
∑
FS(u0,uk+1)
∏
i∈S
p(ui|u0)
(
∑
uSc∈F(u0,uS ,uk+1)
∏
j∈Sc
p(uj |u0)
)2
≤ e−αS−2βSc
∑
FS(u0,uk+1)
p(uS |u0, uk+1)
(
∑
uSc∈F(u0,uS,uk+1)
p(uSc |u0, uS , uk+1)
)2
≤ e−αS−2βSc ,
where FS(u0, uk+1) is the set of all uS that satisfy (u0, uS , uk+1) ∈ FS . On the other
hand, if S = Sm,m′ is empty, then Zm and Z
′
m
are independent given (U0, Uk+1) =
(u0, uk+1), and
E
[
ZmZm′ |u0, uk+1
]
=
(
E[Zm|u0, uk+1]
)2
.
Thus (assume |M∅| = 1)
E[Z2|u0, uk+1] = E[Z|u0, uk+1] +
∑
S⊂[k]
|MS |
∏
j∈Sc
(
|Mj|
2 − |Mj |
)
E[ZmZm′ |u0, uk+1]
≤ E[Z|u0, uk+1] +
(
E[Z|u0, uk+1]
)2
+
∑
∅⊂S⊂[k]
|MS ||MSc |
2e−αS−2βSc ,
(7)
THE MULTIVARIATE COVERING LEMMA 7
where the notation ∅ ⊂ S ⊂ [k] means that S is a nonempty proper subset of [k].
We have
P
{
Z = 0|u0, uk+1
}
≤ P
{
∣
∣Z − E[Z|u0, uk+1]
∣
∣ ≥ E[Z|u0, uk+1]
∣
∣
∣
u0, uk+1
}
(a)
≤
Var(Z|u0, uk+1)
(
E[Z|u0, uk+1]
)2 =
E[Z2|u0, uk+1]
(
E[Z|u0, uk+1]
)2 − 1
(b)
≤
1
1− ǫ
|M|−1eγ +
1
(1− ǫ)2
∑
∅⊂S⊂[k]
|MS |
−1e−αS−2βSc+2γ ,
where (a) follows from Chebyshev’s inequality and (b) follows from Equations (6)
and (7). Now using Equation (5), we get
P{Z = 0} ≤
1
ǫ
P{Fc}+
1
1− ǫ
|M|−1eγ +
1
(1 − ǫ)2
∑
∅⊂S⊂[k]
|MS |
−1e−αS−2βSc+2γ .
(8)
5. The Asymptotic Result
In this section, using our lower and upper bounds, we prove Theorem 1. We first
prove the direct part using our upper bound from Section 4. Set F = A
(n)
δ and for
every j ∈ [k], choose an integer Mj ≥ enRj . Choose a sequence {ǫn}n such that
lim
n→∞
1
ǫn
P
{
(A
(n)
δ )
c
}
= 0.
This is simple to do, since P
{
(A
(n)
δ )
c
}
decays exponentially in n (see Appendix B).
Fix a nonempty S ⊆ [k]. Notice that if
(
Un0 , (U
n
j )j∈S , U
n
k+1
)
∈ FS, then
∣
∣
∣
log
p(unS |u
n
0 , u
n
k+1)
∏
j∈S p(u
n
j |u
n
0 )
− n
(
∑
j∈S
H(Uj|U0)−H(US |U0, Uk+1)
)∣
∣
∣
≤ 2n(|S|+ 1)δ.
Thus we may choose
αS = n
(
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1)− 2(|S|+ 1)δ
)
and
γ = n
(
k
∑
j=1
H(Uj |U0)−H(U[k]|U0, Uk+1) + 2(k + 1)δ
)
.
Similarly, for every nonempty S ⊆ [k], we choose βS as
βS = n
(
∑
j∈S
H(Uj |U0)−H(US |U0, USc , Uk+1)− 2(|S|+ 1)δ)
)
,
since for every
(
Un0 , (U
n
j )j∈S , (U
n
j )j∈Sc
)
∈ F ,
∣
∣
∣
log
p(unS |u
n
0 , u
n
Sc , u
n
k+1)
∏
j∈S p(u
n
j |u
n
0 )
−n
(
∑
j∈S
H(Uj |U0)−H(US |U0, USc , Uk+1)
)
∣
∣
∣
≤ 2n(|S|+1)δ.
THE MULTIVARIATE COVERING LEMMA 8
From our upper bound, Equation (8), it now follows that if for all nonempty S ⊂ [k],
∑
j∈S
Rj >
1
n
(2γ − αS − 2βSc)
= 2
k
∑
j=1
H(Uj |U0)− 2H(U[k]|U0, Uk+1)−
∑
j∈S
H(Uj |U0) +H(US |U0, Uk+1)
− 2
∑
j∈Sc
H(Uj |U0) + 2H(USc |U0, US , Uk+1) + (8k − 2|S|+ 10)δ
=
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1) + (8k − 2|S|+ 10)δ,
and for S = [k],
k
∑
j=1
Rj >
1
n
γ =
k
∑
j=1
H(Uj |U0)−H(U[k]|U0, Uk+1)− 2(k + 1)δ,
then
lim
n→∞
P
{
∃m :
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
∈ A
(n)
δ
}
= 1. (9)
Next we prove the converse. Suppose for each j ∈ [k], Mj ≤ enRj and Equation
(9) holds. Then from our lower bound, Equation (3), it follows
∑
j∈S
Rj ≥
1
n
αS =
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1)− 2(|S|+ 1)δ,
for all nonempty S ⊆ [k].
Appendix A. Cauchy-Schwarz Inequality
Let Z be any random variable that is nonnegative with probability one and has
positive first and second moments. Then
Z = Z1{Z > 0}
almost surely. Thus
E[Z] = E
[
Z1{Z > 0}
]
≤
√
E[Z2]×P{Z > 0},
where the inequality follows from Cauchy-Schwarz. Hence
P{Z > 0} ≥
(
E[Z]
)2
E[Z2]
and
P{Z = 0} ≤ 1−
(
E[Z]
)2
E[Z2]
.
On the other hand, using Chebyshev’s inequality we get
P{Z = 0} = P
{
|Z − E[Z]| ≥ E[Z]
}
≤
Var(Z)
(
E[Z]
)2 =
E[Z2]
(
E[Z]
)2 − 1.
THE MULTIVARIATE COVERING LEMMA 9
Now note that the bound resulting from Cauchy-Schwarz is stronger, since for any
t > 0,
1− t ≤
1
t
− 1.
Appendix B. Large Deviations
The moment generating function of a random variable X is defined as
M(t) = E[etX ]
for all real t for which the expectation on the right hand side is finite. If M is defined
on a neighborhood of 0, say (−t0, t0) for some t0 > 0, then it has a Taylor series
expansion with a positive radius of convergence [1, pp. 278-280]. In particular,
d
dt
M(t)
∣
∣
t=0
= E[X ].
We want to find an upper bound for P{X ≥ a} for some real number a. Choose
t > 0. Using Markov’s inequality, we get
P{X ≥ a} = P{tX ≥ ta}
= P{etX ≥ eta}
≤ e−taE[etX ]
= elogM(t)−ta
Since t > 0 was arbitrary, we get
P{X ≥ a} ≤ einft>0(logM(t)−ta).
Define the function f as
f(t) = logM(t)− ta.
Then f(0) = 0 and f ′(0) = E[X ]− a. Thus if a > E[X ],
inf
t>0
(
logM(t)− ta
)
< 0. (10)
If we apply the same inequality to the random variable
1
n
n
∑
i=1
Xi,
where the Xi’s are i.i.d. copies of X , we get
P
{
n
∑
i=1
Xi ≥ na
}
≤ en inft>0(logM(t)−ta). (11)
Now consider a random vector (U1, . . . , Uk) with distribution p(u1, . . . , uk). For
every nonempty S ⊆ [k], let US denote the random vector (Uj)j∈S . Let (U
n
1 , . . . , U
n
k )
be n i.i.d. copies of (U1, . . . , Uk). By applying inequality (11) to the random vari-
ables {log 1
p(USi)
}ni=1 and setting a = H(US) + ǫ for some ǫ > 0, we get
P
{
n
∑
i=1
log
1
p(USi)
≥ n(H(US) + ǫ)
}
≤ e−nIS(ǫ), (12)
where IS(ǫ) is given by
IS(ǫ) = inf
t>0
{
t
(
H(US) + ǫ
)
− logE
[
p(US)
−t
]
}
THE MULTIVARIATE COVERING LEMMA 10
By the union bound we get
P
{
(Un1 , . . . , U
n
k ) /∈ A
(n)
ǫ (U1, . . . , Uk)
}
≤ 2
∑
∅(S⊆[k]
e−nIS(ǫ)
≤ 2(2k − 1)e−nminS IS(ǫ)
≤ e−nI(ǫ),
where
I(ǫ) = min
S⊆[k]
IS(ǫ) + o
( 1
n
)
.
Finally, note that by Equation (10), each IS(ǫ) is positive, thus so is I(ǫ).
References
[1] Patrick Billingsley, Probability and measure, 3rd ed., SIAM, 1995.
[2] Thomas M. Cover and Joy A. Thomas, Elements of information theory, 2nd ed., Wiley, 2006.
[3] Abbas El Gamal and Young-Han Kim, Network information theory, 2nd ed., Cambridge Uni-
versity Press, 2012.
[4] Abbas El Gamal and Edward C. van der Meulen, A proof of Marton’s coding theorem for the
discrete memoryless broadcast channel, IEEE Trans. Inf. Theory IT-27 (1981), no. 1, 120–122.
[5] Ralf Koetter, Michelle Effros, and Muriel Médard, A theory of network equivalence—Part II:
Multiterminal channels, IEEE Trans. Inf. Theory 60 (2014), no. 7, 3709–3732.
[6] Jingbo Liu, Paul Cuff, and Sergio Verdú, One-shot mutual covering lemma and Marton’s inner
bound with a common message, Proc. IEEE Int. Symp. Information Theory, 2015.
[7] Katalin Marton, A coding theorem for the discrete memoryless broadcast channel, IEEE Trans.
Inf. Theory IT-25 (1979), no. 3, 306–311.
[8] Sergio Verdú, Non-asymptotic covering lemmas, IEEE Information Theory Workshop (ITW),
2015.
California Institute of Technology
E-mail address: parham@caltech.edu
California Institute of Technology
E-mail address: effros@caltech.edu
State University of New York at Buffalo
E-mail address: mikel@buffalo.edu


The Multivariate Covering Lemma and its Converse

ar
X
iv
:1
50
8.
03
34
9v
2 
 [c
s.I
T]
  2
1 J
an
 20
16
THE MULTIVARIATE COVERING LEMMA
AND ITS CONVERSE
PARHAM NOORZAD, MICHELLE EFFROS, AND MICHAEL LANGBERG
Abstract. The multivariate covering lemma states that given a collection of
k codebooks, each of sufficiently large cardinality and independently generated
according to one of the marginals of a joint distribution, one can with prob-
ability arbitrarily close to one choose one codeword from each codebook such
that the resulting k-tuple of codewords is jointly typical with respect to the
joint distribution. Prior proofs of the multivariate covering lemma primarily
employ strong typicality. We give a proof of this lemma for weakly typical sets.
This allows achievability proofs that rely on the covering lemma to go through
for continuous (e.g., Gaussian) channels without the need for quantization.
The covering lemma and its converse are widely used in information theory,
including in rate-distortion theory and in achievability results for multi-user
channels.
1. Introduction
The covering lemma and its extensions play a crucial role in achievability results
in network information theory. Covering lemmas are useful for enabling network
nodes to transmit codewords that “look like” they are generated from a dependent
distribution, whereas in reality, they are carefully selected from sufficiently large
codebooks that are independently generated. This allows nodes to obtain the ben-
efits of both independent and dependent codewords: like independent codewords,
such codewords can be decoded in different locations; like dependent codewords
they have the potential to achieve rates higher than those achieved by independent
codewords. This benefit, however, comes at a cost in rate. Thus the strategy is
useful when the benefit transmitting dependent codewords exceeds its cost.
In the context of the covering lemma, the concept of “looking like” dependent
codewords is captured by the notion of being jointly typical with respect to a
dependent distribution. As there are various ways to define the typical set (here
we specifically focus on weakly typical [2] and strongly typical sets [3]), one may
ask whether a specific version of the covering lemma holds for a given definition of
the typical set. The weakly typical set has two advantages over the strongly typical
set. First, it is easily defined for continuous (e.g., Gaussian) distributions. Second,
the weakly typical set has a simple one-shot counterpart, which allows proofs using
the weakly typical set to be written in the one-shot framework in a simple manner.
On the other hand, some results hold for the strongly typical set that do not hold
for the weakly typical set. Thus it is helpful to review the covering lemma and its
extensions and see for which definition of the typical set each result is currently
known to hold.
The simplest case of the covering lemma is the situation where given a random
vector and an independently generated codebook, a node looks for a codeword in
1
THE MULTIVARIATE COVERING LEMMA 2
the codebook that is jointly typical (with respect to a dependent distribution) with
the given random vector. The result obtained in this case, simply referred to as the
“covering lemma”, appears in the achievability proof of the rate distortion theorem
using weakly typical sets [2]. The second case, called the “mutual covering lemma,”
treats the case where given two independently generated codebooks, a node looks
for a jointly typical pair of codewords, where each codeword is from one of the
codebooks. This result is used in Marton’s inner bound for the two-user broadcast
channel and is proved for strongly typical sets [4, 7]. Recently, by extending the
proof of [2], the authors of [6, 8] prove a one-shot version of the mutual covering
lemma. This proof can be used to show the validity of the mutual covering lemma
for weakly typical sets in the asymptotic setting. The proof in [6, 8], however,
requires stronger independence assumptions on the codebooks than the proof using
strongly typical sets in [3, 4]. Finally, the “multivariate covering lemma” is the
extension of the mutual covering lemma to k independently generated codebooks,
and can be used to obtain an inner bound on the broadcast channel with k users
[3]. As stated in [3], one can show this result holds for strongly typical sets by
extending the proof of the mutual covering lemma [4].
In this work, using the general strategy of El Gamal and Van der Meulen [4]
and some ideas regarding weakly typical sets from Koetter, Effros, and Médard
[5], we give a proof of the multivariate covering lemma for weakly typical sets.
We also provide a converse, a special case of which is usually referred to as the
packing lemma [3]. We remark that while similar to the argument in [4], we use
Chebyshev’s inequality for the direct result (Section 4), it is also possible to use
the Cauchy-Schwarz inequality (see Appendix A), which leads to a more accurate
upper bound.
2. Problem Statement
For every positive integer n, define the set [n] = {1, . . . , n}. Let k be a positive
integer and
p(u0, u1, . . . , uk, uk+1)
be a probability distribution on the set
k+1∏
j=0
Uj .
For every nonempty S ⊆ [k] define
US =
∏
j∈S
Uj .
For every j ∈ [k], let Mj be a nonnegative integer. For every nonempty S ⊆ [k],
define the set MS as
MS =
∏
j∈S
[Mj].
and let M =M[k]. For every m = (m1, . . . ,mk) ∈M, let the random vector
(U0, U1(m1), . . . , Uk(mk), Uk+1)
have distribution
p(u0)
k+1∏
j=1
p(uj|u0),
THE MULTIVARIATE COVERING LEMMA 3
where p(u0) and each p(uj|u0) are the conditional marginals of p(u0, . . . , uk+1). In
addition, let F be an arbitrary subset of U0 × U[k+1]. We want to find upper and
lower bounds on the probability
P
{
∀m ∈M :
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
/∈ F
}
.
We derive the lower bound (Section 3) using the union bound, which does not
depend on the statistical dependencies of the vectors(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
for different values of m. For the upper bound (Section 4), which leads to the
multivariate covering lemma, we require a stronger assumption, which we next
describe.
Let m = (mj)j∈[k] and m
′ = (m′j)j∈[k] be in M. Define the set Sm,m′ as
Sm,m′ =
{
j ∈ [k] : mj = m
′
j
}
.
When m and m′ are clear from context, we denote Sm,m′ with S. In the proof of
the upper bound we require
P
{
∀j ∈ [k] : Uj(mj) = uj and Uj(m
′
j) = u
′
j
∣∣U0 = u0, Uk+1 = uk+1}
=
k∏
j=1
p(uj|u0)×
∏
j∈Sc
p(u′j |u0),
for all u0 and all (uj)j and (u
′
j)j such that if j ∈ S, then uj = u
′
j (Assumption I).
Note that if there exists a j ∈ S where uj 6= u
′
j then the probability on the left
hand side equals zero.
In the corresponding asymptotic problem (Section 5), we apply our bounds to
P
{
∀m :
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
/∈ A
(n)
δ
}
,
where for every m, (
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
is simply n i.i.d. copies of the original random vector(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
,
(Assumption II) and A
(n)
δ is the weakly typical set for the distribution p(u0, u1, . . . , uk, uk+1).
Our main result follows.
Theorem 1 (Multivariate Covering Lemma). Suppose Assumptions (I) and (II)
hold for the joint distribution of
Un0 ,
{
Un1 (m1), . . . , U
n
k (mk)
}
m
, Unk+1.
For the direct part, suppose for all j ∈ [k], Mj ≥ enRj . If for all nonempty S ⊆ [k],∑
j∈S
Rj >
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1) + (8k − 2|S|+ 10)δ, (1)
then
lim
n→∞
P
{
∃m :
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
∈ A
(n)
δ
}
= 1. (2)
THE MULTIVARIATE COVERING LEMMA 4
For the converse, assume for all j ∈ [k], Mj ≤ enRj . If Equation (2) holds, then∑
j∈S
Rj ≥
∑
j∈S
H(Uj |U0)−H(US|U0, Uk+1)− 2(|S|+ 1)δ,
for all nonempty S ⊆ [k].
In the direct part of Theorem 1, we can weaken the lower bound on
∑
j∈S Rj
when S = [k]. Specifically, we can replace Equation (1) with
k∑
j=1
Rj >
k∑
j=1
H(Uj |U0)−H(U[k]|U0, Uk+1) + 2(k + 1)δ.
for S = [k].
3. The Lower Bound
For every S ⊆ [k], define FS as the projection of F on U0 × US × Uk+1. Then
for every (u0, uS , uk+1) ∈ FS , let F(u0, uS , uk+1) be the set of all uSc such that
(u0, u[k], uk+1) ∈ F . In addition, for every nonempty S ⊆ [k], let αS and βS be
constants such that
αS ≤ log
p(uS |u0, uk+1)∏
j∈S p(uj |u0)
for all (u0, uS , uk+1) ∈ FS and
βS ≤ log
p(uS |u0, uSc , uk+1)∏
j∈S p(uj |u0)
for all (u0, uS , uSc , uk+1) ∈ F . Furthermore, let the constant γ satisfy
γ ≥ log
p(u[k]|u0, uk+1)∏
j∈[k] p(uj |u0)
for all (u0, u[k], uk+1) ∈ F .
For every m = (m1, . . . ,mk) ∈M, define the random variable Zm as
Zm = 1
{(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
∈ F
}
and set
Z =
∑
m∈M
Zm.
Our aim is to find a lower bound for P{Z = 0}. Note that for every nonempty
S ⊆ [k],
P
{
∃m : Zm = 1
}
= P
{
∃m :
(
U0, U1(m1), . . . , Uk(mk), Uk+1
)
∈ F
}
≤ P
{
∃m :
(
U0,
(
Uj(mj)
)
j∈S
, Uk+1
)
∈ FS
}
≤ |MS |
∑
FS
p(u0, uk+1)
∏
j∈S
p(uj |u0)
≤ |MS |e
−αS
∑
FS
p(u0, uS , uk+1)
≤ |MS |e
−αS .
THE MULTIVARIATE COVERING LEMMA 5
Thus
P{Z = 0} = 1−P
{
∃m : Zm = 1
}
≥ 1− min
|S|6=∅
|MS |e
−αS . (3)
4. The Upper Bound
In deriving our upper bound on P{Z = 0}, we apply conditioning and Cheby-
shev’s inequality. Thus, the factor
1(
P{F(u0, uk+1)}
)2
appears, where
P{F(u0, uk+1)} = P
{
U[k] ∈ F(u0, uk+1)|U0 = u0, Uk+1 = uk+1
}
=
∑
u[k]∈F(u0,uk+1)
p(u[k]|u0, uk+1)
and F(u0, uk+1) (Section 3) is simply the set of all u[k]’s that satisfy (u0, u[k], uk+1) ∈
F . Thus to get a reasonably accurate upper bound, we require P{F(u0, uk+1)} to
be large. However, as we cannot guarantee this for all (u0, uk+1), we partition the
(u0, uk+1) pairs into “good” and “bad” sets, corresponding to large and small val-
ues of P{F(u0, uk+1)}, respectively. The probability of the good set is large when
P{(U0, U[k], Uk+1) ∈ F} is sufficiently large. To see this, fix ǫ > 0 and following
Appendix III of [5], define the set G ⊆ U0 × Uk+1 as
G =
{
(u0, uk+1) : P{F(u0, uk+1)} ≥ 1− ǫ
}
,
Note that G is the set of all good (u0, uk+1) pairs as defined above. We have
P
{
(U0, U[k], Uk+1) ∈ F
}
=
∑
u0,uk+1
∑
u[k]∈F(u0,uk+1)
p(u0, uk+1)p(u[k]|u0, uk+1)
=
∑
u0,uk+1
p(u0, uk+1)P{F(u0, uk+1)}
≤ (1− ǫ)P{(U0, Uk+1) /∈ G}+P{(U0, Uk+1) ∈ G}
= 1− ǫP{(U0, Uk+1) /∈ G}.
Thus
P{(U0, Uk+1) /∈ G} ≤
1
ǫ
P
{
(U0, U[k], Uk+1) /∈ F
}
. (4)
Our aim is to find an upper bound for P{Z = 0}. To do this, we write
P{Z = 0} =
∑
u0,uk+1
p(u0, uk+1)P{Z = 0|u0, uk+1}
≤
1
ǫ
P
{
(U0, U[k], Uk+1) /∈ F
}
+
∑
(u0,uk+1)∈G
p(u0, uk+1)P{Z = 0|u0, uk+1},
(5)
where the inequality follows from Equation (4). Therefore, to find an upper bound
on P{Z = 0}, it suffices to find an upper bound on P{Z = 0|U0 = u0, , Uk+1 =
uk+1} for all (u0, uk+1) ∈ G. Fix (u0, uk+1) ∈ G. We use Chebyshev’s inequality
to find an upper bound on P{Z = 0|U0 = u0, Uk+1 = uk+1}. Thus we need to
THE MULTIVARIATE COVERING LEMMA 6
calculate E[Z|U0 = u0, Uk+1 = uk+1] and E[Z2|U0 = u0, Uk+1 = uk+1]. For a given
m, from the definition of γ (Section 3) it follows
E[Zm|u0, uk+1] = P
{(
U1(m1), . . . , Uk(mk)
)
∈ F(u0, uk+1)
∣∣u0, uk+1}
=
∑
F(u0,uk+1)
p(u1|u0) . . . p(uk|u0)
≥
∑
F(u0,uk+1)
e−γp(u[k]|u0, uk+1)
= e−γ P{F(u0, uk+1)} ≥ (1 − ǫ)e
−γ .
where the last inequality follows from the fact that (u0, uk+1) ∈ G. Thus, by
linearity of expectation,
E[Z|U0 = u0, Uk+1 = uk+1] ≥ |M|e
−γ(1− ǫ). (6)
Next, we find an upper bound on E[Z2|U0 = u0, Uk+1 = uk+1]. We have
Z2 =
∑
m
Z2
m
+
∑
m6=m′
ZmZm′ = Z +
∑
m6=m′
ZmZm′ ,
since Z2
m
= Zm and Z =
∑
m
Zm. Thus
E[Z2|u0, uk+1] = E[Z|u0, uk+1] + E
[ ∑
m6=m′
ZmZm′
∣∣u0, uk+1]
For any pair of distinct m and m′ with nonempty S = Sm,m′ , we have
E
[
ZmZm′ |u0, uk+1
]
=
∑
FS(u0,uk+1)
∏
i∈S
p(ui|u0)
( ∑
uSc∈F(u0,uS ,uk+1)
∏
j∈Sc
p(uj |u0)
)2
≤ e−αS−2βSc
∑
FS(u0,uk+1)
p(uS |u0, uk+1)
( ∑
uSc∈F(u0,uS,uk+1)
p(uSc |u0, uS , uk+1)
)2
≤ e−αS−2βSc ,
where FS(u0, uk+1) is the set of all uS that satisfy (u0, uS , uk+1) ∈ FS . On the other
hand, if S = Sm,m′ is empty, then Zm and Z
′
m
are independent given (U0, Uk+1) =
(u0, uk+1), and
E
[
ZmZm′ |u0, uk+1
]
=
(
E[Zm|u0, uk+1]
)2
.
Thus (assume |M∅| = 1)
E[Z2|u0, uk+1] = E[Z|u0, uk+1] +
∑
S⊂[k]
|MS |
∏
j∈Sc
(
|Mj|
2 − |Mj |
)
E[ZmZm′ |u0, uk+1]
≤ E[Z|u0, uk+1] +
(
E[Z|u0, uk+1]
)2
+
∑
∅⊂S⊂[k]
|MS ||MSc |
2e−αS−2βSc ,
(7)
THE MULTIVARIATE COVERING LEMMA 7
where the notation ∅ ⊂ S ⊂ [k] means that S is a nonempty proper subset of [k].
We have
P
{
Z = 0|u0, uk+1
}
≤ P
{∣∣Z − E[Z|u0, uk+1]∣∣ ≥ E[Z|u0, uk+1]∣∣∣u0, uk+1}
(a)
≤
Var(Z|u0, uk+1)(
E[Z|u0, uk+1]
)2 = E[Z2|u0, uk+1](
E[Z|u0, uk+1]
)2 − 1
(b)
≤
1
1− ǫ
|M|−1eγ +
1
(1− ǫ)2
∑
∅⊂S⊂[k]
|MS |
−1e−αS−2βSc+2γ ,
where (a) follows from Chebyshev’s inequality and (b) follows from Equations (6)
and (7). Now using Equation (5), we get
P{Z = 0} ≤
1
ǫ
P{Fc}+
1
1− ǫ
|M|−1eγ +
1
(1 − ǫ)2
∑
∅⊂S⊂[k]
|MS |
−1e−αS−2βSc+2γ .
(8)
5. The Asymptotic Result
In this section, using our lower and upper bounds, we prove Theorem 1. We first
prove the direct part using our upper bound from Section 4. Set F = A
(n)
δ and for
every j ∈ [k], choose an integer Mj ≥ enRj . Choose a sequence {ǫn}n such that
lim
n→∞
1
ǫn
P
{
(A
(n)
δ )
c
}
= 0.
This is simple to do, since P
{
(A
(n)
δ )
c
}
decays exponentially in n (see Appendix B).
Fix a nonempty S ⊆ [k]. Notice that if
(
Un0 , (U
n
j )j∈S , U
n
k+1
)
∈ FS, then∣∣∣ log p(unS |un0 , unk+1)∏
j∈S p(u
n
j |u
n
0 )
− n
(∑
j∈S
H(Uj|U0)−H(US |U0, Uk+1)
)∣∣∣ ≤ 2n(|S|+ 1)δ.
Thus we may choose
αS = n
(∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1)− 2(|S|+ 1)δ
)
and
γ = n
( k∑
j=1
H(Uj |U0)−H(U[k]|U0, Uk+1) + 2(k + 1)δ
)
.
Similarly, for every nonempty S ⊆ [k], we choose βS as
βS = n
(∑
j∈S
H(Uj |U0)−H(US |U0, USc , Uk+1)− 2(|S|+ 1)δ)
)
,
since for every
(
Un0 , (U
n
j )j∈S , (U
n
j )j∈Sc
)
∈ F ,
∣∣∣ log p(unS |un0 , unSc , unk+1)∏
j∈S p(u
n
j |u
n
0 )
−n
(∑
j∈S
H(Uj |U0)−H(US |U0, USc , Uk+1)
)∣∣∣ ≤ 2n(|S|+1)δ.
THE MULTIVARIATE COVERING LEMMA 8
From our upper bound, Equation (8), it now follows that if for all nonempty S ⊂ [k],∑
j∈S
Rj >
1
n
(2γ − αS − 2βSc)
= 2
k∑
j=1
H(Uj |U0)− 2H(U[k]|U0, Uk+1)−
∑
j∈S
H(Uj |U0) +H(US |U0, Uk+1)
− 2
∑
j∈Sc
H(Uj |U0) + 2H(USc |U0, US , Uk+1) + (8k − 2|S|+ 10)δ
=
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1) + (8k − 2|S|+ 10)δ,
and for S = [k],
k∑
j=1
Rj >
1
n
γ =
k∑
j=1
H(Uj |U0)−H(U[k]|U0, Uk+1)− 2(k + 1)δ,
then
lim
n→∞
P
{
∃m :
(
Un0 , U
n
1 (m1), . . . , U
n
k (mk), U
n
k+1
)
∈ A
(n)
δ
}
= 1. (9)
Next we prove the converse. Suppose for each j ∈ [k], Mj ≤ enRj and Equation
(9) holds. Then from our lower bound, Equation (3), it follows∑
j∈S
Rj ≥
1
n
αS =
∑
j∈S
H(Uj |U0)−H(US |U0, Uk+1)− 2(|S|+ 1)δ,
for all nonempty S ⊆ [k].
Appendix A. Cauchy-Schwarz Inequality
Let Z be any random variable that is nonnegative with probability one and has
positive first and second moments. Then
Z = Z1{Z > 0}
almost surely. Thus
E[Z] = E
[
Z1{Z > 0}
]
≤
√
E[Z2]×P{Z > 0},
where the inequality follows from Cauchy-Schwarz. Hence
P{Z > 0} ≥
(
E[Z]
)2
E[Z2]
and
P{Z = 0} ≤ 1−
(
E[Z]
)2
E[Z2]
.
On the other hand, using Chebyshev’s inequality we get
P{Z = 0} = P
{
|Z − E[Z]| ≥ E[Z]
}
≤
Var(Z)(
E[Z]
)2 = E[Z2](
E[Z]
)2 − 1.
THE MULTIVARIATE COVERING LEMMA 9
Now note that the bound resulting from Cauchy-Schwarz is stronger, since for any
t > 0,
1− t ≤
1
t
− 1.
Appendix B. Large Deviations
The moment generating function of a random variable X is defined as
M(t) = E[etX ]
for all real t for which the expectation on the right hand side is finite. IfM is defined
on a neighborhood of 0, say (−t0, t0) for some t0 > 0, then it has a Taylor series
expansion with a positive radius of convergence [1, pp. 278-280]. In particular,
d
dt
M(t)
∣∣
t=0
= E[X ].
We want to find an upper bound for P{X ≥ a} for some real number a. Choose
t > 0. Using Markov’s inequality, we get
P{X ≥ a} = P{tX ≥ ta}
= P{etX ≥ eta}
≤ e−taE[etX ]
= elogM(t)−ta
Since t > 0 was arbitrary, we get
P{X ≥ a} ≤ einft>0(logM(t)−ta).
Define the function f as
f(t) = logM(t)− ta.
Then f(0) = 0 and f ′(0) = E[X ]− a. Thus if a > E[X ],
inf
t>0
(
logM(t)− ta
)
< 0. (10)
If we apply the same inequality to the random variable
1
n
n∑
i=1
Xi,
where the Xi’s are i.i.d. copies of X , we get
P
{ n∑
i=1
Xi ≥ na
}
≤ en inft>0(logM(t)−ta). (11)
Now consider a random vector (U1, . . . , Uk) with distribution p(u1, . . . , uk). For
every nonempty S ⊆ [k], let US denote the random vector (Uj)j∈S . Let (U
n
1 , . . . , U
n
k )
be n i.i.d. copies of (U1, . . . , Uk). By applying inequality (11) to the random vari-
ables {log 1
p(USi)
}ni=1 and setting a = H(US) + ǫ for some ǫ > 0, we get
P
{
n∑
i=1
log
1
p(USi)
≥ n(H(US) + ǫ)
}
≤ e−nIS(ǫ), (12)
where IS(ǫ) is given by
IS(ǫ) = inf
t>0
{
t
(
H(US) + ǫ
)
− logE
[
p(US)
−t
]}
THE MULTIVARIATE COVERING LEMMA 10
By the union bound we get
P
{
(Un1 , . . . , U
n
k ) /∈ A
(n)
ǫ (U1, . . . , Uk)
}
≤ 2
∑
∅(S⊆[k]
e−nIS(ǫ)
≤ 2(2k − 1)e−nminS IS(ǫ)
≤ e−nI(ǫ),
where
I(ǫ) = min
S⊆[k]
IS(ǫ) + o
( 1
n
)
.
Finally, note that by Equation (10), each IS(ǫ) is positive, thus so is I(ǫ).
References
[1] Patrick Billingsley, Probability and measure, 3rd ed., SIAM, 1995.
[2] Thomas M. Cover and Joy A. Thomas, Elements of information theory, 2nd ed., Wiley, 2006.
[3] Abbas El Gamal and Young-Han Kim, Network information theory, 2nd ed., Cambridge Uni-
versity Press, 2012.
[4] Abbas El Gamal and Edward C. van der Meulen, A proof of Marton’s coding theorem for the
discrete memoryless broadcast channel, IEEE Trans. Inf. Theory IT-27 (1981), no. 1, 120–122.
[5] Ralf Koetter, Michelle Effros, and Muriel Médard, A theory of network equivalence—Part II:
Multiterminal channels, IEEE Trans. Inf. Theory 60 (2014), no. 7, 3709–3732.
[6] Jingbo Liu, Paul Cuff, and Sergio Verdú, One-shot mutual covering lemma and Marton’s inner
bound with a common message, Proc. IEEE Int. Symp. Information Theory, 2015.
[7] Katalin Marton, A coding theorem for the discrete memoryless broadcast channel, IEEE Trans.
Inf. Theory IT-25 (1979), no. 3, 306–311.
[8] Sergio Verdú, Non-asymptotic covering lemmas, IEEE Information Theory Workshop (ITW),
2015.
California Institute of Technology
E-mail address: parham@caltech.edu
California Institute of Technology
E-mail address: effros@caltech.edu
State University of New York at Buffalo
E-mail address: mikel@buffalo.edu


https://authors.library.caltech.edu/94381/1/1508.03349.pdf

The Multivariate Covering Lemma and its Converse

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main

Caltech Authors - Main