Algorithms for convolutional codes and development of linear sequential machine

Massey, J. L.

English

NASA Technical Reports Server

F I N A L  REPORT F O R  FIRST YEAR 
(Sept. 16 ,  1967 t o  Sept.  15,  1968) 
NASA GRANT NGL 15-004-026 
"CONVOLUTIONAL CODING TECHNIQUES 
FOR DATA PROTECTION"  
https://ntrs.nasa.gov/search.jsp?R=19690009661 2020-03-12T07:46:20+00:00Z
February 3 ,  1969 
FINAL REPORT FOR FIRST YEAR 
(Sept. 16 ,  1967 t o  Sept.  15, 1968) 
NASA GRANT NGL 15-004-026 
"CONVOLUTIONAL CODING TECHNIQUES 
FOR DATA PROTECTION" 
Submitted t o :  
Info. copy to :  
Submitted by: 
National Aeronautics E Space Administration 
Office of S c i e n t i f i c  & Technical Information [Code US) 
Washington, D .  C .  20546 
F l i g h t  Data Systems Branch 
NASA Goddard Space F l i g h t  Center 
Greenbelt Md .. 20771 
ATT: D r ,  Robert W. Rochelle (Code 710) 
D r  James L . Massey (P r inc ipa l  Inves t iga to r  1 
Professor  of Electrical Engineering 
Universi ty  of Notre D a m e  
Notre D a m e ,  Ind,  46556 
ABSTRACT 
The p r i n c i p a l  r e s u l t s  obtained under NASA Grant N G R - 1 5 - 0 0 4 - 0 2 6  
during t h e  per iod Sept .  16 ,  1967 t o  Sept .  15, 1968 were the following: 
(1) Eas i ly  programmable algori thms f o r  t h e  syn thes i s  of good, long 
convolut ional  codes, 
( 2 )  A un i f i ed  ana lys i s  of decoders f o r  convolut ional  codes based on 
t h e  use of s t o c h a s t i c  s equen t i a l  machines models, 
( 3 )  Demonstration of t h e  ex is tence  of 'tgoodft classes of convolut ional  
codes with "bad" e r r o r  propagation p rope r t i e s  , 
( 4 )  Development of necessary and s u f f i c i e n t  condi t ions f o r  l i n e a r  
s equen t i a l  machines (:.e. convolut ional  encoders) t o  have feedforward 
inve r ses ,  
( 5 )  Development of a simple conceptual approach t o  obta in  t h e  pro- 
p e r t i e s  of t h e  Fano sequen t i a l  decoding a lgor i thm, ,  
( 6 )  New bounds on minimum d i s t ance  f o r  convolut ional  codes, 
(7 )  Development of a complete and ur, if ied theory  for inverses  of 
l i n e a r ,  t ime-invariant  dynamical systems, 
and (8 )  Further  development and simplification between decoding pro- 
cedures for t h e  Bose-Chaudhuri-Hocquenghem codes and t h e  syn thes i s  of 
minimal length l i n e a r  feedback s h i f t - r e g i s t e r s .  
I n  addi t ion  t o  descr ib ing  t h e  above research ,  t h i s  r e p o r t  includes 
a l i s t i n g  of a l l  personnel involved i n  t h e  research  performed under t h i s  
g ran t .  
I. Summary of Main Research Resul t s  
(1) Synthesis  of Good, Long Convolutional Codes 
I n  r ecogn i t ion  of t h e  fact that f u t u r e  space communications 
systems may w e l l  r e q u i r e  t h e  use of convolut ional  codes w i t h  con- 
s t r a i n t  l eng ths  w e l l  beyond those  for which good codes w e r e  formerly 
known, an e f f o r t  w a s  made t o  develop algori thms f o r  ob ta in ing  good, 
long codes with a modest amount of computer search t i m e ,  These 
e f f o r t s  m e t  wi th  considerable  success  y i e ld ing  algori thms which have 
been programmed, f o r  i n s t ance ,  t o  ob ta in  good ra te  1 / 2  b inary  codes 
out  t o  a c o n s t r a i n t  l eng th  of  144 b i t s  (72 information b i t s ) ,  w e l l  
beyond twice t h e  c o n s t r a i n t  l ength  of codes obtained by earlier inves- 
t i g a t o r s .  Details of t h i s  work may be found in :  
D .  J. Cos te l lo ,  Jr., "A Construction Technique for Random-Error 
Correct ing Convolutional Codes," Tech. Rpt,  No, EE-685, U ,  of Notre 
D a m e ,  Notre D a m e ,  I nd , ,  May 1, 1968. (This paper has been accepted 
for  pub l i ca t ion  i n  IEEE Transact ions on Information Theory,)  
( 2 )  A Unified Analysis of Decoders for, Convolutional Codes 
It w a s  evident  t o  s tuden t s  of convolut ional  coding, t h a t  t h e  con- 
cep tua l  foundat ions of t h i s  sub jec t  were i n  a p r imi t ive  s t a g e  compared 
t o  t h a t  f o r  t h e  earlier, but  less e f f i c i e n t ,  block coding. 
decoding procedures,  v i z .  s equen t i a l  decoding, feedback threshold  
decoding, d e f i n i t e  decoding, semi-def ini te  decoding, V i t e r b i  s e m i -  
s equen t i a l  decoding, e t c , ,  had been advanced f o r  convolut ional  codes 
with very l i t t l e  understanding of t h e  r e l a t i o n s h i p s  between t h e s e  
var ious  forms of decoders.  
Various 
To reduce t h e  conceptual confusion, an e f f o r t  w a s  made t o  f i n d  a 
common framework t h a t  could accommodate a l l  t h e s e  va r i ed  forms of  de- 
coders.  The s t o c h a s t i c  s equen t i a l  machine model was u l t ima te ly  s e t t l e d  
upon and proved t o  be except iona l ly  simple and f l e x i b l e  f o r  t h e  t a s k  of 
descr ib ing  decoders for convolut ional  codes,  
model, t h e  first p r e c i s e  d e f i n i t i o n  and exac t  c a l c u l a t i o n  of t h e  s teady-  
state e r r o r  p r o b a b i l i t y  f o r  a feedback decoder w a s  obtained.  Conditions 
were a l s o  obtained which guarantee the  s u p e r i o r i t y  of feedback decoding 
t o  d e f i n i t e  decoding i n  most p r a c t i c a l  s i t u a t i o n s .  
shown t h a t  f o r  t h e  V i t e r b i  algorithm (which i s  a maximum Likelihood 
Through the  use of t h i s  
It w a s  f u r t h e r  
1- 
decoding procedure f o r  f i n i t e  sequences) t h a t  t h e  decoding error 
p r o b a b i l i t y  always tends  t o  a d e f i n i t e  l i m i t  for decoding of i n f i n i t e  
sequences, and t h a t  under c e r t a i n  condi t ions  t h i s  s teady-s ta te  e r r o r  
p r o b a b i l i t y  is  an exponent ia l ly  decreasing func t ion  of t h e  code con- 
s t r a i n t  l ength .  
L 
Further  d e t a i l s  may be found i n :  
T. N .  Morrissey, Jr., "Analysis of Decoders f o r  Convolutional 
Codes by S tochas t i c  Sequent ia l  Machine Methods," Tech. Rpt,  No. EE-682, 
Dept. of  E l e c .  Engr. ,  U.  of Notre D a m e ,  Notre D a m e ,  Ind . ,  May 1, 1968. 
T.  N .  Morrissey, Jr., "A Unified Markovian Analysis of Decoders 
for Convolutional Codes," Tech. Rpt. No. EE-687, Dept. of Elec. Engr , ,  
U. of Notre D a m e ,  Notre D a m e ,  Ind . ,  O c t .  24, 1968. (This paper has 
been accepted f o r  pub l i ca t ion  i n  the IEEE Transact ions on Information 
Theory. ) 
(i 3 )  "Good" Codes with rrBad'r Er ror  Propagation P rope r t i e s  
The p o s s i b i l i t y  of ca t a s t roph ic  decoding error-propagat ion,  ? . e ,  of 
t h e  p o s s i b i l i t y  t h a t  a f i n i t e  number of  channel e r r o r s  could t r i g g e r  i n -  
f i n i t e l y  many decoding e r r o r s ,  has been a haunting cons idera t ion  f o r  u se r s  
of convolut ional  codes with feedback decoders.  
t h a t  such a ca tas t rophe  might occur only with a poor code or with a 
poor decoding algori thm f o r  a good code. 
a r e sea rch  e f f o r t  t h a t  culminated i n  t h e  demonstration of i n f i n i t e  
It was sometimes thought 
These cons idera t ions  l e d  t o  
classes of good codes Ctfgood" i n  t h e  sense t h a t  t h e i r  feedback-decoding 
minimum d i s t a n c e  i s  a t  least ha l f  t h a t  guaranteed obta inable  by t h e  
Gi lbe r t  bound) t h a t  necessa r i ly  e x h i b i t  ca t a s t roph ic  e r r o r  propagation 
when decoded by any reasonable  feedback decoder. It was f u r t h e r  shown 
t h a t  t h e  de fec t  i n  t h e s e  codes which g ives  r ise t o  t h e  ca t a s t roph ic  
f a i l u r e  i s  t h e i r  s m a l l  def ini te-decoding minimum d i s t ance .  
be found i n :  
J. L. Massey, "Catastrophic  Error-Propagation i n  Convolutional Codes," 
Details may 
Proceedings of 11 th  Midwest C i r c u i t  Theory Symposium, U ,  of Notre Dame, 
Notre D a m e ,  I nd . ,  May 13-14, 1968, pp. 583-587, 
-- 
(4 )  Feedforward Inverses  of  Linear Sequent ia l  C i r c u i t s  
It had earlier been pointed out  by the p r i n c i p a l  i nves t iga to r  
t h a t  a second kind of ca t a s t roph ic  behavior is poss ib l e  with convolut ional  
codes, v i z .  t h e  occurrence of i n f i n i t e l y  many errors Ln t h e  decoded infor.;: 
mation d i g i t s  when only a f i n i t e  number of  e r r o r s  had been made i n  decoding 
t h e  channel sequences, and t h a t  t h i s  ca tas t rophe  w a s  avoidable i f  and 
-2- 
only if t h e  l i n e a r  s e q u e n t i a l  machine which se rves  as t h e  convolut ional  
encoder possesses  a feedforward (;.e. feedback-free) inverse .  This  fact  
l e d  t o  a success fu l  e f f o r t  t o  ob ta in  t h e  gene ra l  condi'tions f o r  t h e  ex is tence  
of a feedforward inve r se  of a l i n e a r  s equen t i a l  c i r c u i t .  
found in :  
Details may be 
R. R.  Olson, "Note on Feedforward Inverses  of  Linear Sequent ia l  C i r -  
c u i t s , "  Tech. Rpt. No. EE-684, Dept ,  of Elec, Engr., U ,  of Notre D a m e ,  
Apr i l  1, 1968. 
- on Computers 
(This  paper has been submitted t o  t h e  IEEE Transact ions 
with t h e  dec is ion  on pub l i ca t ion  s t i l l  pending,)  
( 5 )  A Simple Conceptual Approach t o  t h e  Fano Algorithm 
The Fano algori thm has for seve ra l  yea r s  been accepted as the b e s t  
a v a i l a b l e  sequen t i a l  decoding algori thm, but  understanding of t h e  search  
p r o p e r t i e s  of t h i s  a lgori thm has proved e l u s i v e ,  
has l e d  t o  an a n a l y s i s  of t h e  Fano algori thm which makes de r iva t ion  of t h e  
search  p r o p e r t i e s  bo th  simple and i n t u i t i v e l y  p l eas ing .  
t h i s  procedure were first t o  consider  t h e  "sear.chi.ngr1 of a s i n g l e  path or 
t runk  i n  which t h e r e  are no a l t e r n a t e  branches stemming from t h e  nodes 
thereon ,  and second, t o  i s o l a t e  a superpos i t ion  - p r i n c i p l e  by which t h e  
gene ra l  t ree-searching  p r o p e r t i e s  of t h e  algorithm can reacii ly be in fe r r ed  
from i ts  trunk-searching p rope r t i e s .  
Research under t h i s  g r a n t  
The b a s i c  s t e p s  i n  
Details may be found i n :  
J. L .  Massey and M .  K.  Sain,  I'Trunk and Tree Searching P rope r t i e s  of 
t h e  Fano Sequent ia l  Decoding Algorithm," Elec.  Engr. Memo. No. EE-6817, 
Univ. of Motre D a m e ,  Notre Dame, Ind. ,  O c t .  1, 1968. Also appearing i n  
Proceedings of 6 th  Annual Al le r ton  Conf. on Ckt,  and System Th., Univ. of 
I l l i n o i s ,  O c t .  2-4, 1968, pp. 153-160. 
_I -- - - 
This approach w a s  a l s o  used as t h e  foundation of an a n a l y s i s  of t h e  
s ta t i s t ica l  p r o p e r t i e s  of t h e  Fano algori thm t h a t  w i l l  be d c s c r a e d  i n  a 
forthcoming t e c h n i c a l  r e p o r t .  This r e s u l t  w a s  descr ibed o r z l l y  as: 
J. L .  Massey and M. K. Sa in ,  "Dis t r ibu t ion  of t h e  Minimum Cumulative 
Metric f o r  Sequent ia l  Decoding," presented a t  IEEE I n t e r n a t i o n a l  Symposium 
on Information Theory, E l l e n v i l l e ,  N .  Y . ,  J an ,  28-31, 1969. 
(6 )  Bounds on Minimum Distance of  Convolutional Codes 
One of t h e  r e sea rch  goa l s  of t h i s  gran t  is the obta in ing  of  good bounds 
-3- 
f o r  t h e  minimum d i s t a n c e  of convolut ional  codes t h a t  can be used f o r  
t h e  eva lua t ion  of  known codes. 
improved Plotkin- type upper bound on t h e  feedback-decoding minimum 
d i s t ance  of convolut ional  codes, and t o  a Gilber t - type lower bound on 
t h e  def ini te-decoding minimum d i s t ance .  
i a l l y  important s ince  it proves t h e  ex i s t ence  of good convolut ional  
codes t h a t  are free of both ca t a s t roph ic  forms of f a i l u r e  descr ibed 
i n  s e c t i o n s  (3) and (4)  above. 
t h i s  bound was based on t h e  demonstration tha ta lmost  a l l  l i n e a r  feed-  
back s h i f t - r e g i s t e r s  produce no non-zero output  sequence with a spa r se  
populat ion of non-zero d i g i t s ,  
encoder f o r  a c y c l i c  block code, and conversely,  t h i s  a n a l y s i s  is 
expected t o  be u s e f u l  i n  a t t ack ing  t h e  e l u s i v e  quest ion as t o  whether 
t h e r e  e x i s t  a r b i t r a r i l y  long c y c l i c  codes wi th  good d i s t ance  p r o p e r t i e s ,  
Details may be found i n :  
This r e sea rch  has l e d  t o  a simple and 
This  la t ter  bound is  espec- 
Moreover, t h e  method of proof f o r  
Since each such s h i f t - r e g i s t e r  i s  an 
J .  L. Massey, "Some Algebraic and Dj-stance P rope r t i e s  of  Convo- 
l u t i o n a l  Codes," appearing i n  Error CoAmecting Codes (Edi tor :  H.  B ,  
Mann), John Wiley and Sons, I n c . ,  New Pork, 1968, pp. 89-109. 
( 7 )  Inverses  of Dynamical Systems 
An unexpected bonus of the  work performed under t h i s  g r a n t  a rose  
from t h e  r e a l i z a t i o n  by t h e  i n v e s t i g a t o r s  t h a t  t h e i r  work on feedfor -  
ward inverses  of  l i n e a r  s equen t i a l  c i r c u i t s  could be e a s i l y  genera l ized  
t o  a complete theory  f o r  inversesof  t ime=invariant  dynamical systems, 
both time-continuous and t imeyd i sc re t e .  Such inve r ses ,  commonly known 
as "whitening f i l ters"  from t h e i r  app l i ca t ion  i n  f i l t e r i n g  and pre- 
d i c t i o n  theory ,  p lay  important r o l e s  i n  many cu r ren t  i n v e s t i g a t i o n s  i n  
s ta t i s t ica l  communication theory and opt imal  c o n t r o l  theory ,  The gen-. 
e r a l i z a t i o n  w a s  c a r r i e d  ou t  and r e s u l t e d  i n  c r i te r ia  f o r  t h e  ex is tence  
of inverses  considerably simpler than those  prev ious ly  known, and a l s o  
t o  a simple,  gene ra l  procedure for t h e  cons t ruc t ion  of t h e  inverse  
system from knowledge of  t h e  s t r u c t u r a l  matrices of  t h e  o r i g i n a l  system. 
This  research  a l s o  i s o l a t e d  a fundamental new parameter,  c a l l e d  t h e  
inherent  i n t e g r a t i o n  of  a system, which is t h e  least i n t e g r a l  of t h e  
inpu t  t o  t h e  system t h a t  can be recovered as t h e  output  of a second 
-4- 
l i n e a r  t ime-invariant  dynamical system i n  cascade with t h e  first 
system. Details of t h i s  research  may be found i n :  
M. K .  Sain and J. L.  Massey, V n v e r t i b i l i t y  of Linear Time- 
Inva r i an t  Dynamical Systems," E l e c .  Engr. Memo. No. EE-687, U. of 
Notre D a m e ,  Notre D a m e ,  Ind . ,  Aug. 8 ,  1968. {This paper has  been 
accepted f o r  pub l i ca t ion  i n  IEEE Transact ions - on Automatic Control . )  
(8) Bose-Chaudhuri-Hocquenghem Codes and Linear Shi f t -Regis te rs  
During t h e  per iod repor ted  he re in ,  t h e  p r i n c i p a l  i nves t iga to r  
achieved s u b s t a n t i a l  s imp l i f i ca t ion  of h i s  work r e l a t i n g  t h e  syn- 
t h e s i s  of minimal length  l i n e a r  feedback s h i f t - r e g i s t e r s  for producing 
a given f i n i t e  sequence of d i g i t s  t o  t h e  problem of decoding t h e  Bose- 
Chaudhuri-Hocquenghem codes. This work demonstrates t h a t  t h e  Berlekamp 
decoding algori thm, which i s  t h e  most e f f i c i e n t  known genera l  algorithm 
f o r  decoding these  c y c l i c  codes, is  more gene ra l ly  j u s t  an algorithm 
t o  solve t h e  s h i f t - r e g i s t e r  synthes is  problem c i t e d  above. 
may be found i n :  
Details 
J. L .  Massey, "Shi f t  Regis ter  Synthesis  and BCH Decoding," Elec.  
Engr. Memo. No. EE-684, U. of Notre Daxe, Notre Dame,  Ind. ,  Juiie 18,  
1968. 
ac t ions  - on Information Theory.) 
(This paper has been accepted f o r  publ ica t ion  i n  IEEE Trans- 
(9 )  Other Ac t iv i ty  
I n  add i t ion  t o  the  t h e o r e t i c a l  research  described above, during 
t h e  period repor ted  he re in ,  t h e  i n v e s t i g a t o r s  completed t h e  assembly 
langugage programming of a channel s imulator  and Fano sequen t i a l  
decoder for t h e  UNIVAC 1107 computer i n  t h e  Universi ty  of Notre Dame 
Computing Center.  
binary symmetric channel or an add i t ive  Gaussian noise  channel for 
use i n  conjunction with t h e  sequen t i a l  decoding of any convolut ional  
or - and cons t r a in t  l ength  of 72 information b i t s  1 1  code of rates - - 2, 3'  
or less. 
pe r  computation as computations are defined with sequen t i a l  decoding. 
Extensive s imulat ions have s ince  been c a r r i e d  out  on t h i s  f a c i l i t y  and 
t h e  r e s u l t s  w i l l  be repor ted  i n  a f u t u r e  t e c h n i c a l  r e p o r t .  
These programs permit t h e  s imulat ion of e i t h e r  a 
1 
4 
The operat ion of t hese  programs r e q u i r e s  about 100 microseconds 
-5- 
11. Personnel Engaged i n  Reseapch Under t h i s  Grant 
T a b l e  I g ives  a complete accounting of a l l  personnel  engaged 
i n  r e sea rch  under t h i s  g ran t  during t h e  per iod  covered by t h i s  r e p o r t .  
Source of support  i s  shown, if d i f f e r e n t  from t h i s  g r a n t ,  for graduate  
r e sea rch  s tuden t s  performing r e sea rch  under t h e  d i r e c t i o n  of t h e  pr in-  
c i p a l  i n v e s t i g a t o r .  
-6- 
n 
v), x ,  
p:: 
c ,  w .  
p: . 
I 
. -  
a, 
a G  
a,? 
??I  
.A 
a, 
c) 
a, 
p: 
n n 
c) 
a, E El p: 
co 
ro 
\ 
m 
r-l 
\ 
co 
0 
-I-, 
co co 
W W 
\ 
m m 
r-l rl 
\ \ m m 
0 0 
tJ c, 
0 
-r 
0 
tJ 
P- 
W 
\ 
W 
r-l 
\ m 
b 
(D 
\ 
a 
rl 
\ 
W 
c 
H 
8 
tJ 
Id 
M 
.rl 
c, 
rn 
d 
G 
H 
w 
l.4 a s c, 
c, 
rn 
5 
c, 
c, 
rn 
4 
m 
rn c 
% 
4J 
c, 
rn 
.rl 
rn 
rn c 
5 
3.1 
c3 w 
E s 
.rl 
rn 
tn c 
c, 
c, 
rn 
.rl 
rn 
rn e 
5 % 
f;i 
'a, 
rn 
a, 
p: 
c 
4 
Id 
u3 rn 
Id c 
x a, 
rn 
rn 
.rl 
Fc 
c 0 
rn 8 4  
0 
r-l 
rl 
a, 
c, 
rn 
0 
0 
M 
c 
0 
5 t-2 
tn 
b !=l E- p: 
F-l Fc n c 
n n 
W F 
W W 
5 
Fc x 
co 
A 
W 
Fc 
R 
Fc n Fc n Fc x Fc e 
n 
4 
v 
n 
01 
W 
n 
m 
v 
3 
W 


Convolutional coding techniques for data protection Final report, 16 Sep. 1967 - 15 Sep. 1968

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server