We show the capacity of a generalized pulse-position modulation (PPM) channel, where the input vectors may be any set that allows a transitive group of coordinate permutations, is achieved by a uniform input distribution. We derive a simple expression in terms of the Kullback Leibler distance for the binary case, and the asymptote in the PPM order. We prove a sub-additivity result for the PPM channel and use it to show PPM capacity is monotonic in the order

Hamkins, J.

Klimesh, M.

McElience, R.

Moision, B.

NASA Technical Reports Server

IPN Progress Report 42-164 February 15, 2006
Capacity of the Generalized Pulse-Position
Modulation Channel
J. Hamkins,1 M. Klimesh,1 R. McEliece,1 and B. Moision1
We show the capacity of a generalized pulse-position modulation (PPM) channel,
where the input vectors may be any set that allows a transitive group of coordinate
permutations, is achieved by a uniform input distribution. We derive a simple
expression in terms of the Kullback–Leibler distance for the binary case, and ﬁnd
the asymptote in the PPM order. We prove a sub-additivity result for the PPM
channel and use it to show PPM capacity is monotonic in the order.
I. Introduction
NASA is currently developing the ﬁrst operational deep-space optical communications link for launch
on the Mars Telesat Orbiter in 2009.2 The deep-space optical channel is well modeled as memoryless
and operates eﬃciently at large peak-to-average power ratios, which may be eﬃciently implemented with
pulse-position modulation (PPM) [1,2], in which each channel symbol is a unit vector. PPM satisﬁes
the property that each symbol may be obtained as a permutation of the coordinates of another. We
consider a generalization of this, where the input vectors may be any set that allows a transitive group
of coordinate permutations. We derive an expression for the capacity of this generalized PPM channel in
the binary case, and examine the behavior of the capacity of the PPM channel as a function of the PPM
order.
In Section II we show that, for a memoryless generalized PPM channel, capacity is achieved with
equiprobable inputs. We show that a simple expression for the capacity follows for the binary case, and
illustrate the asymptotic behavior of the PPM channel as the PPM size tends to inﬁnity. In Section III
we prove a sub-additivity result for the PPM channel and show that certain monotonic behavior follows.
II. Capacity of Generalized PPM
We use X,Y to denote random variables and x, y their realizations. Similarly, we let X,Y and x,y
denote n-vectors of random variables and their realizations. Let pY |X(y|x) be the conditional density (or
probability mass) function of a memoryless channel, and pY|X(y|x) its nth extension. When it’s clear
from the context, we simply write p(y|x) or p(y|x) for pY |X(y|x) and pY|X(y|x).
1 Communications Architectures and Research Section.
2 This mission has recently been canceled.
The research described in this publication was carried out by the Jet Propulsion Laboratory, California Institute of
Technology, under a contract with the National Aeronautics and Space Administration.
1
https://ntrs.nasa.gov/search.jsp?R=20060008610 2019-08-29T19:39:22+00:00Z
Let S = {x1,x2, · · · ,xs} be a set of length n vectors, pX(·) a probability distribution on S, and I(X;Y)
the mutual information between X and Y. The S-capacity of the channel is deﬁned as
CS = max
pX
I(X;Y)
i.e., the capacity with inputs restricted to S. Let G be a group of coordinate permutations that ﬁx S,
i.e., such that for each g ∈ G, σgS = S, where σg is the mapping imposed by g. If, in addition, G acts
transitively on S, then we call S a transitive set. (G acts transitively on S if for each xi,xj ∈ S there
exists g ∈ G such that xi = σg(xj).)
The capacity of a channel whose input is a transitive set follows from the well-known fact that I(X;Y)
is convex-∩ in the input distribution p [3, Theorem 4.4.2].
Theorem 1. If S is a transitive set, then CS is achieved by a uniform distribution on S.
Proof. Let p be a distribution on S and for g ∈ G let pg be given by pg(xi) = p
(
σg(xi)
)
. Clearly
any pg produces the same mutual information as p. Thus by Jensen’s inequality, (1/|G|) ∑g∈G pg yields
mutual information greater than or equal to that yielded by p. But this new distribution is simply the
uniform distribution: for any xi we have
1
|G|
∑
g∈G
pg(xi) =
1
|G|
∑
g∈G
p
(
σg(xi)
)
and as g ranges over G, σg(xi) ranges over each element of S the same number of times (by the Orbit
Stabilizer Theorem, e.g., [4, Theorem 8.2]); thus the above quantity is equal to 1/s. ❐
A. Binary Inputs
With binary inputs, CS reduces to a simple expression. Let the input alphabet be {0, 1}, and I0(x)
and I1(x) the collection of indices of the 0’s and 1’s in x, respectively. For example, I0(101) = 2,
I1(101) = {1, 3}. Let N1 = |I1(x)|, a constant for each x ∈ S, and D(·||·) be the Kullback–Leibler
distance.
Theorem 2. On a binary input channel with p(y|1)/p(y|0) < ∞,
CS = N1D
(
p(y|1)||p(y|0))−D(p(y)||p(y|0))
Proof. With equiprobable inputs from Theorem 1, we have
2
CS =
∫
p(y|x1) log2
(
p(y|x1)
p(y)
)
dy
=
∫
p(y|x1) log2
(∏
i∈I1(x1) p(yi|1)
∏
i∈I0(x1) p(yi|0)
p(y)
)
dy
=
∫
p(y|x1) log2
⎛
⎜⎜⎝
p(y|0)∏i∈I1(x1) p(yi|1)p(yi|0)
p(y)
⎞
⎟⎟⎠ dy
=
∫
p(y|x1) log2
⎛
⎝ ∏
i∈I1(x1)
p(yi|1)
p(yi|0)
⎞
⎠ dy − ∫ p(y) log2
(
p(y)
p(y|0)
)
dy
= N1D
(
p(y|1)||p(y|0))−D(p(y)||p(y|0)) ❐
B. Pulse-Position Modulation
In the remainder, we investigate the behavior of CS as a function of n for the PPM channel. To that
end, let I(n) be the capacity of a memoryless channel with PPM inputs of length n. We ﬁrst treat the
case in Theorem 2 where p(y|1)/p(y|0) is unbounded.
Let U and A be the collections of unambiguous and ambiguous outputs when x = 1 is transmitted:
U =
{
y|p(y|1) = 0, p(y|0) = 0}
A =
{
y|p(y|1) ≥ 0, p(y|0) = 0}
If any coordinate of y belongs to U , the input will be known with certainty. In order to treat the
ambiguous and unambiguous outputs separately, deﬁne a reduced channel p∗(y|x) with output y ∈ A as
follows:
p∗(y|0) = p(y|0)
p∗(y|1) = p(y|1)
p(A|1)
where p(A|1) = ∫
A
p(y|1)dy. Let I∗(X;Y) and I∗(n) be the mutual information and capacity of the
reduced channel, in bits per PPM symbol.
Lemma 1.
I(n) = p(U |1) log2 n + p(A|1)I∗(n)
3
Proof. Let xj be the symbol with I1(xj) = j (a 1 in position j), and Un(xj) and An(xj) the collections
of unambiguous and ambiguous outputs when xj is transmitted:
Un(xj) = {y|yj ∈ U}
An(xj) = {y|yj /∈ U}
Let U˜ =
⋃
x∈S Un(x), A˜ =
⋃
x∈S An(x). Introduce a binary random variable Z as follows:
Z =
{
0, Y ∈ U˜
1, Y ∈ A˜
Since each x contains exactly one nonzero entry, P (Z = 0) = p(U |1) and P (Z = 1) = p(A|1). Note that
p(x|z) = p(x); hence, H(X|Z) = H(X) and
I(X;Y) = I(X;Y, Z)
= I(X;Z) + I(X;Y|Z)
= P (Z = 0)I(X;Y|Z = 0) + P (Z = 1)I(X;Y|Z = 1)
= p(U |1) log2 n + p(A|1)I∗(X;Y)
The lemma follows since the capacity achieving input distribution is uniform for both channels. ❐
Hence we can decompose the n-ary PPM channel into an unambiguous channel, which contributes
p(U |1) log2 n to the capacity, and a reduced channel, with transition probabilities p∗(y|x). In the remain-
der, we assume the channel is reduced, which allows a simple corollary of Theorem 2.
Corollary 1. For the reduced binary PPM channel, I(n) = D
(
p(y|1)||p(y|0))−D(p(y)||p(y|0)).
Corollary 1 allows a straightforward proof of the asymptotic behavior of the memoryless PPM channel.
Theorem 3. limn→∞ I(n) = D
(
p(y|1)||p(y|0)).
Proof. Let 1 denote the unit vector with a 1 in the ﬁrst position:
0 ≤ D(p(y)||p(y|0))
= EY log2
[
p(y)
p(y|0)
]
= EY log2
[∑n
i=1 p(y|xi)p(xi)
p(y|0)
]
= EY log2
[
1
n
n∑
i=1
p(Yi|1)
p(Yi|0)
]
4
where the last inequality follows since
EYi|Xi=0
[
p(Yi|1)
p(Yi|0)
]
=
∫
y:p(y|0)>0
p(y|1) dy ≤ 1
for i = 1. For a reduced channel there exists a constant K such that
EY1|X1=1
[
p(Y1|1)
p(Y1|0)
]
< K
hence,
0 ≤ lim
n→∞D
(
p(y)||p(y|0)) ≤ lim
n→∞ log2
(
K
n
+
n− 1
n
)
= 0
❐
III. Capacity Inequalities
Theorem 4. If n ≤ m, then I(kn)− I(n) ≥ I(km)− I(m).
This theorem says that if we multiply the number of slots by k, the increase in PPM capacity will
be larger if the original number of slots was smaller. The conclusion can be equivalently stated as
I(kn) + I(m) ≥ I(km) + I(n).
Proof. Let Zk, Zm, and Zn be random variables uniformly distributed on {1, · · · , k}, {1, · · · ,m}, and
{1, . . . , n}, respectively. Let Yn be the (random) output vector when Zn drives an n-PPM channel (that
is, the input to the channel is an n-vector with a 1 in position Zn and zeros elsewhere). Similarly, let
Ykn and Ykm be the output vectors when the ordered pairs (Zk, Zn) and (Zk, Zm) drive kn-PPM and
km-PPM channels, respectively. In these two cases, it is useful conceptually to regard the slots as being
arranged in a rectangular grid, with, for example, (Zk, Zn) specifying the column and row of the 1.
Observe that H(Zn|Zk,Ykn) = H(Zn|Yn), because in the left-hand side Zk speciﬁes the column of
n slots in which the 1 is “hiding”; thus the other slots can be ignored. We therefore have
H(Zk, Zn|Ykn) = H(Zn|Zk,Ykn) + H(Zk|Ykn)
= H(Zn|Yn) + H(Zk|Ykn)
We then have
I(kn)− I(n) = log2 kn−H(Zk, Zn|Ykn)− log2 n + H(Zn|Yn)
= log2 k −H(Zk|Ykn)
Similarly, I(km)− I(m) = log2 k −H(Zk|Ykm).
5
All that remains is to show that H(Zk|Ykm) ≥ H(Zk|Ykn). Intuitively, this is clear because there
is more ambiguity about which column the 1 is in when there are more rows. A more formal line of
reasoning would involve introducing side information W in the km-PPM case that speciﬁes a random set
of n rows, one of which contains the 1. Then H(Zk|Ykm) ≥ H(Zk|Ykm,W ) = H(Zk|Ykn). ❐
Theorem 4 has the immediate consequence that if m1m2 = n1n2 and m1 + m2 ≤ n1 + n2, then
I(m1) + I(m2) ≥ I(n1) + I(n2). A special case of the theorem is that I(mn) ≤ I(m) + I(n).
We are also now able to say something interesting about 2k-PPM:
Corollary 2. For k = 1, 2, · · ·, the quantity I(2k)/k is decreasing in k.
Proof. Theorem 4 implies that I(2) − I(1), I(4) − I(2), I(8) − I(4), · · · is a decreasing sequence.
Therefore, the average of the ﬁrst k terms of the sequence is decreasing in k. But since I(1) = 0, the
average of the ﬁrst k terms is simply I(2k)/k. ❐
Corollary 3. For k ∈ N, the bits-per-slot capacity of 2k-PPM on a discrete-time memoryless channel
is monotonically decreasing in k.
Proof. The capacity of 2k-PPM in bits per slot is I(2k)/2k. Thus, this result follows from (and is
much weaker than) Corollary 2. ❐
A close look at Theorem 4 suggests that the following is likely true: The quantity
(
I(k + 1) −
I(k)
)
/
(
log(k + 1) − log k) is decreasing in k. Equivalently, the function I(k) is convex-∩ when plotted
as a function of log k. As of this writing, we have not proven this, so it is still a conjecture. Theorem 4
would essentially be a special case of this result. The result would also imply more general versions of
Corollaries 2 and 3: the quantity I(k)/ log k would be decreasing in k for k ≥ 2, and I(k)/k would be
decreasing in k for k ≥ 3.
IV. Conclusions
We have derived the capacity of a generalized PPM channel. The formulation applies to conventional
PPM and multipulse PPM, among others. We showed that the capacity in bits per slot of conventional
PPM decreases as the modulation order increases, a conclusion consistent with the decreasing average
power of this sequence of modulations.
References
[1] R. G. Lipes, “Pulse-Position-Modulation Coding as Near-Optimum Utilization
of Photon Counting Channel with Bandwidth and Power Constraints,” The Deep
Space Network Progress Report 42-56, January and February 1980, Jet Propul-
sion Laboratory, Pasadena, California, pp. 108–113, April 15, 1980.
http://ipnpr/progress report2/42-56/56N.PDF
[2] A. D. Wyner, “Capacity and Error Exponent for the Direct Detection Photon
Channel—Part I,” IEEE Transactions on Information Theory, vol. 34, pp. 1449–
1461, November 1988.
[3] R. G. Gallager, Information Theory and Reliable Communication, New York:
John Wiley & Sons, 1968.
[4] J. A. Gallian, Contemporary Abstract Algebra, 3rd. ed., Lexington, Massachu-
setts: D. C. Heath and Co., 1994.
6


Capacity of the Generalized Pulse-Position Modulation Channel

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server