It is shown that there are arbitrarily long "good" (in the sense of Gilbert) binary block codes that are preserved under very large permutation groups. This result contrasts sharply with the properties of linear codes: it is conjectured that long cyclic codes are bad, and known that long affine-invariant codes are bad

McEliece, Robert J.

English

Caltech Authors - Main

IEEE TRANS.4CTIONS ON INFORMATION THEORY, VOL. IT-16, NO. 5. SEPTEMBER 1970 
On the Symmetry o f Good Non linear Codes 
ROBERT J. McELIECE, MEMBER, IEEE 
Abstract-It is shown that there are arbitrarily long “good” 
(in the sense of Gilbert) binary block codes that are preserved 
under very large permutation groups. This result contrasts sharply 
with the properties of linear codes: it is conjectured that long 
cyclic codes are bad, and known that long al&e-invariant codes 
are bad. 
1  
T IS the object of this paper to show that there exist 
arbitrarily long binary codes that have large per- 
mutation groups and also satisfy the Gilbert bound. 
Kasami’s result’ that long linear codes of length n = 2”, 
which admit the affine group of order n (n - l), cannot 
satisfy the Gilbert bound, makes it extremely unlikely 
that our result can be extended to linear codes. What 
our result does show, however, is that it cannot be merely 
the presence of a  moderate-sized permutation group that 
forces a code to be bad, just as linearity does not necessarily 
degrade performance. The precise statement of our result 
is given in Theorem 1. 
Theorem 1 
For each n 2 1, suppose P, is a group of permutations 
on {1,2, ... , n), of order pn, and that each nonidentity 
permutation in P, has 5 fn fixed points. 
We  assume that 
logp, = 
fn = 
44, 
o(n). 
Then for any 0 < D < l/2, there exists a sequence of 
codes C,, (C,, has block length n) with rates R,, so that 
Cn admits P,, as a permutation group; 
&in (CJ 2 Dn; 
lim inf R, 2 1 - H, (D), 
where H, (+) is the binary entropy function. 
For example, Theorem 1 permits the following choices 
of groups P,. 
P, = cyclic group of order n. 
Manuscript received July 23, 1969; revised November 3, 1969. 
This research was carried out at the Jet Propulsion Laboratory, 
California Institute of Technology, Pasadena, sponsored by the 
National Aeronautics and Space Administratlon under Contract 
NAS 7-100. 
The author is with Jet Propulsion Laboratory, Pasadena, Calif. 
91103. 
1 T. Kasami, “An upper bound on k/n for affine-invariant codes 
with fixed d/n, “IEEE Trans. Information Theory (Correspondence), 
vol. IT-1.5, pp. 174-176, January 1969. 
p, 
P, = 
affine group of order n (n - 1) if n  = 2” 
some m. 
1 {l) if n  # 2”. 
I 
projective unimodular group PSL (2, 2”‘) 
\ n  = 2’, for some m. 
(11) if n  # 2”. 
for 
if 
The proof of Theorem 1 is based on the usual proof of 
the Gilbert bound, and also on Theorem 2. 
Theorem 2 
Let 
( 
1, 2, ... , n  
7r= 7r(l), 7r(2), ... , 7r(n) > 
be a permutation without fixed points: r(i) #  i for all i. 
For each binary vector X: = (xl, x2, . * . , 2,) let ?rZ = 
(x *(I), XT(Z), * * * , x,(,)), and define iv, (a, n) = number 
of vectors 5 for which d(5, ?rZ) 5 tm (Hamming distance). 
Then if a! 5  l/2, 
N,(a, n) g  2(n’2)(1+Hs(a))m 
In Section II, we show how Theorem 2 may be used 
to prove Theorem 1. Section III is devoted to the proof 
of Theorem 2. 
II. THEOREM 2 IMPLIES THEOREM 1 
Lemma 1 
If P is a group of permutations on ( 1, 2, . . . , n), and 
if each nonidentity permutation of P has _<f fixed points, 
let NP(q n) = number of vectors 5 with d(Z, ~6) 5 cm 
for some 1 # r E P. Then if (I! 5  l/2, f 5  n(1 - 2cu), 
NP(CY, n> i IPI 2’, 
where 
B=(f:(n-i)[l+Il,(*)]. 
Proof: Obviously 
N&, n) I (IPI - 1) mftx N,(a, 4. 
lflr 
Next, observe that if a  permutation n has fr fixed points, 
?r acts as a permutation ?T* without fixed points on n - f T 
points, so that d($, x?) = d(Z*, r*Z*), where X* is a vector 
of length n - fr obtained from II: by deleting those co- 
610 IEEl TRANSACTIONS ON INFORMATION THEORY, SEPTEMBER 1970 
ordinates fixed by ?r. Thus 
N,(a, n) = 2’“N,. 
and so by Theorem 2, NT(a, n) 5 2E”r’, where 
E(f,) = f* + y [l + “~(;EjJj. 
Finally, it is easy to see that E(f) is an increasing function 
off in the range 0 2 f < n(1 - 200, so that if fr 5 f for 
all w E P, N,(a, n) 5 2E(‘). Q.E.D. 
Using Lemma 1, we now can prove Theorem 1. 
For each n, let C,, be a code of largest possible rate R, 
subject to the two restraints of having minimum distance 
2 Dn and being invariant under P,. (The code consisting 
of 00 ... 0 and 11 3.. 1 satisfies these hypotheses, so 
there is no doubt about the existence of such codes.) Our 
goal is to show that lim inf R, 2 1 - H(D). 
We enclose each word Z E C, within a sphere of radius 
[Dn]. The total volume occupied by these spheres is, 
at most, 
2 ?LR” 
co 
; 22 n(R”+H(D)) , k<Dn 
and so there are at least 2” - 2n(R”+H(D’) words at a distance 
2 Dn from C,,. On the other hand, from Lemma 1, for 
sufficiently large n the number of vectors Z with d(Z, 
7KZ) c: Dn for some r E P is no more than 2”E(D+) where 
E(J): 4 = 6, + !,A[,+~(&-)]+lt’ogapm 
with 
4% = ifn. 
Since R, is assumed to be as large as possible, each It: that 
is at distance 2 Dn from C, must have d(Z, G) < Dn 
for some ?r, and so, in particular, 
2n _ y(R,+R(D)) < 2nE(D.n). 
Hence, 
2 n(R,+H(D)) 2 yyl _ 2+1-E)) 
n(R, + H(D)) 2 n + log, (1 - 2--n(1-E)) 
1 -n(l-E) 
2n-gj2 
( 
since log, (1 - y) >_ 
Therefore, 
R, + H(D) 2 1 - & 2--n(1--E). 
i&Y. > 
But our hypotheses are &, + 0, l/n log p, -+ 0, so that 
E(D, n) + $(l + H(D)) < 1 for all D < +. Hence 
cpl-E(n,n), + 0 and so lim inf R, 2 1 - H(D), as asserted. 
III. PROOF OF THEOREM 2 
Let us write the permutation ?r in cycle form: ?r = 
c1c2 . . . c,, with lCil = ni. For example, 
tz3 = (14)(257)(36), 
with n, = 2, na = 3, n3 = 2. Now let d, = number of 
vectors Z for which d(Z, ~2) = m. 
Lemma 2 
D(z) = 2 d,x” 
“SO 
= (1 + 2)” h (1 + (&q). 
i=l 
Proof: Since d(?, ?rX) = w(Z + G), where w is the 
Hamming weight, what is needed is an analysis of the 
weight spectrum of the range of the linear operator 
T, : 5 -+ 5 + ?rZ. Since the range of T, is the set of vectors 
that are orthogonal (inner product: 2. Z = c zixi (mod 2)) 
to the null space of T,, let us first identify the null space. 
If 5 = ?rZ, then x1 = r,(1) = 2,2(,, = . . ., etc., so that 
T,Z = 0 if and only if 2 is constant on the cycles of rr; 
that is, i, j E Ck implies zri = zi. Thus, a vector It: can be 
orthogonal to every vector in the null space of T, if and 
only if 2 has even weight on each cycle; that is, c (x, 1 i E 
Ci) = 0 (mod 2) for each j. Thus, the number of vectors 
of weight m  in the range of T, is the number of vectors 
of weight m  that have even weight on each cycle; and 
this number is clearly 
cl:, = n,+m ,+~+*r=m  tx9 ‘.. ( n:>. (l)
mi all even 
If 
D&) = c @z- = i((1 + 2)“’ + (1 - .q), 
even m  
from (1) we see that d; is the coefficient of zm in the product 
Dl(.W&) . . * D?(z). Furthermore, since there are 2’ 
vectors f such that 2 + ?rZ = 0, each vector of the form 
2 + ~2 has 2’ preimages; that is, cl, = 2’ a,,‘,. Hence, 
D(x) = 2’ $ 0 ((1 + z)“< +.(l - 2)“‘) 
I t 
= (1 + 4” a (1 + (&-j”). Q.E.D. 
Lemma 3 (The Chernoff Bound) 
If 
D(z) = 2 d, zm, 
m=O 
and if d, > 0 for all m, then for every s > 0,O < (Y < 1, 
Proof: 
&-% d, 5 2”“” D(2-7. 
2 aan D(2-“) = c d,,, 2a(a”-m) 2 x d,, 
m  rns (I* 
IEEF TRANSACTIONS ON INFORMATION THEORY. VOL. IT-16. NO. 5, SEPTElMBER 1970 
since 
Nx(a, n) 6: 2,‘,(1 + a-‘)“(, + (&&)‘)“” 
Q.E.D. = 2as,yl/a(1 + 2--2r)n/2~ 
We  can now prove Theorem 2 without difficulty. From 
Lemmas 1 and 3. It is easy to verify that the minimum of this last expression 
N,(cY, n) 5 2”““(1 + 2-“)” g (1 + (e-g)“‘) 
occurs at 
all s > 0. 
First notice that if 0  2 0, m  2 2, 1  + 0” 5 (1 + 02)m’a, and is 2(n’Z’(1’H(u’). 
and since we have assumed that ni 2  2 for all i, Q.E.D. 
Sequentia l Decoding o f Systematic and  Nonsystematic 
Convolutional Codes W ith  Arbitra ry Decoder Bias 
EDWARD A. BUCHER, MEMHEEL,  IEEE 
A&s&act-This paper presents several results involving Fano’s 
sequential decoding algorithm for convolutional codes. An upper 
bound to the ath moment of decoder computation is obtained for 
arbitrary decoder bias B and a 2 1. An upper bound on error proba- 
bility with sequential decoding is derived for both systematic and 
nonsystematic convolutional codes. This error bound involves the 
exact value of the decoder bias B. It is shown that there is a trade-off 
between sequential decoder computation and error probability as the 
bias B is varied. It is also shown that for many values of B, sequential 
decoding of systematic convolutional codes gives an exponentially 
larger error probability than sequential decoding of nonsystematic 
convolutional codes when both codes are designed with exponentially 
equal optinzum decoder error probabilities. 
RECENT paper [l] extends Viterbi’s [9] upper 
and lower bounds to error probability for con- 
volutional codes to include systematic as well 
as nonsystematic convolutional codes with optimum de- 
coding. These results are of the form 
5 P(E) 5 f(L) exp {--V.&(R)) (1) 
where N., the effective constraint length, is the number 
of channel symbols directly affected by a given informa- 
tion symbol after the symbol’s first appearance in the 
codeword. The function O(N,) approaches zero as N, 
Manuscript received April 1, 1969; revised January 6, 1970. 
This work is part. of the author’s Ph.D. dissertation, Department, 
of Electrical Engineering, Massachusetts Institute of Technology, 
Cambridge, Mass. 
The author is with the Lincoln Laboratory, Ylassachuset,ts 
Instit,ute of Technology, Lexington, Mass. 02173. 
approaches infinity and f(L) is a  linear function of the 
information block length L. 
This paper examines the performance of sequential 
decoding with systematic and nonsystematic convolu- 
tional codes. The Fano algorithm [2] sequential decoder 
provides a practical yet powerful method of decoding 
convolutional codes [3]. A combination of results by Jacobs 
and Berlekamp [4], Savage [5], Falconer [6], and Jelinek 
[7] shows that the number of sequential decoder computa- 
tions required to decode an information symbol in an 
infinite constraint length convolutional code has a Pareto 
distribution. Thus, 
P[number of computations > N] z AVL’” (2) 
for large N. The Pareto exponent 01 is obtained by noting 
that the ath moment of computation is bounded for all 
positive a < o( and unbounded for a 2 (II. Thus, a is the 
smallest positive a for which the ath moment of computa- 
tion is unbounded. 
We  show that there is a trade-off between sequential 
decoder computation and error probability for convolu- 
tional codes. This trade-off involves the value of the 
decoder bias B. Setting B equal to the data rate R maxi- 
mizes (Y but gives an upper bound on error probability in 
which the sequential decoding upper-bound error ex- 
ponent E,,(R, B) is substantially smaller than E,(R) 
the optimum decoder upper-bound error exponent. On the 
other hand, making B somewhat larger than R decreases 
error probability until EU8(R, B) = E,(R), but this 
increase in B implies a smaller LY and more decoder com- 
putation. We  also find that E,,(R, B) is often substantially 


On the symmetry of good nonlinear codes

https://authors.library.caltech.edu/6936/1/MCEieeetit70.pdf

On the symmetry of good nonlinear codes

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main