Contains reports on two research projects.Lincoln Laboratory, Purchase Order DDL-B158Department of the ArmyDepartment of the NavyDepartment of the Air Force under Contract AF 19(122)-45

Unger, S. H.

Wozencraft, J. M.

DSpace@MIT

IX. PROCESSING AND TRANSMISSION OF INFORMATION*Prof. P. Elias E. Ferretti E. T. SchoenProf. R. M. Fano K. Joannou F. F. TungProf. D. A. Huffman R. M. Lerner S. H. UngerProf. C. E. Shannon J. B. O'Loughlin M. L. WetherellDr. M. V. Cerrillo L. S. Onyshkevych J. M. WozencraftW. G. Breckenridge A. J. Osborne W. A. YoungbloodW. W. PetersonA. ROW ASSIGNMENTS IN SEQUENTIAL SWITCHING CIRUCITSThe terminal characteristics of asynchronous sequential switching circuits can bespecified by means of flow tables (1). In an analogous manner, the external behaviorof synchronous (or clocked) sequential circuits can be described by means of similartables (2, 3), as shown in Fig. IX-1.A general synthesis procedure for either circuit can be carried out as follows.1. Construction and simplification of the flow table (1).2. Assignment to each row of the table of one or more states of a set of binary-valued variables, which will be called "state variables." In the asynchronous case, thesestates must be so chosen that critical races never occur. That is, if any transition inthe table calls for changing more than one state variable, then the order in which thechanges occur must not affect the row that is ultimately reached. This requirementoften necessitates the use of more than one state per row (4). No restriction of thiskind applies in the case of synchronous systems; hence there is little reason to assignmore than one state to a row.3. Derivation from the flow table and row assignment of the specifications for acombinational circuit with feedback, as shown in reference 1; derivation by an analogousprocess for synchronous systems.4. Synthesis of this combinational circuit by any convenient method.This report will discuss step 2. Let us first consider the synchronous case, andassume that one state is to be assigned to each row; and, for the sake of simplicity,that m, the number of rows in the table, is an integral power of two. Thus, if m = 2nthen n-state variables will yield the required number of states. Since any state can beassigned to any row, the number of possible arrangements is m!. Three of the 24possible assignments are shown for the same table in Fig. IX-1. However, from aphysical viewpoint, many of these assignments are essentially equivalent to one another,since they differ only in that some of the state variables have merely been relabelled.For example, the assignment in Fig. IX-1c can be obtained from the one in Fig. IX-laby complementing y 2 and then interchanging yl and y 2It will be shown that the number of distinct assignments (those which, in general,may lead to different circuit specifications in step 3) is (2n - 1)!/n! for a 2n-rowsynchronous flow table. In the case of asynchronous circuits, the problem of raceThis work was supported in part by Purchase Order DDL-B158. (See the Introduction.)(IX. PROCESSING AND TRANSMISSION OF INFORMATION)Inputx1x2:StateylY2: 0000 11001 1011 1010 1001 11 10 Row1 10 20  12 20  3 231 21 4 0  34 0 30 10 4(a)Inputx1x2:StateYIY2: 00000 1101 1010 1011 1001 11 10Inputx1x2Stateyl2: 00 01Row10 1 20  121 20  3 23 21 4 0  34 0 30 10 411 101 0 0 01 1 1 210 21 20 3110 31 21 4010 40 30 10(c)Fig. IX-1. Synchronous flow tables. The numbers with superscripts indicatethe next row; the superscripts correspond to the output. Rowassignments in (a) and (b) are distinct; those in (a) and (c) areequivalent.conditions restricts the allowable number of assignments to the extent that there are nonontrivial transformations that can always be made. In specific cases, however, severaldistinct assignments are possible.These statements will be proved by using some elementary principles of group theory(see ref. 5). If r represents a particular row assignment for an m-row synchronous table(where m = 2 n), and if p represents one of the m! permutation transformations that canbe applied to a set of m elements, then the expression rp can be used to designate thenew row assignment that results from permuting assignment r in accordance with trans-formation p. To take an example: if the zeros and ones that describe each state arethought of as forming binary numbers, then the states can be compactly written as thedecimal equivalents of these numbers, so that, in Fig. IX-la, the internal states areRow(IX. PROCESSING AND TRANSMISSION OF INFORMATION)0, 1, 3, and 2, reading from the top down. If we apply the permutation (0)(1)(32), theassignment of Fig. IX-lb is obtained; the permutation (0132) transforms the same assign-ment into that of Fig. IX-ic. The set G of all permutations p (including the identityI) can easily be shown to be a noncomutative group with m! members.A permutation s [such as (0132)] that results when the state variables are inter-changed and/or complemented, will be called "symmetric." The set of all of thesetransformations (members of which will henceforth be referred to as si, i = 1, 2, ... ) isa subgroup of G containing n! 2 n members, and it constitutes the set of all transforma-tions, leaving distances on an n-dimensional hypercube invariant (6).Two row assignments, rl and r 2 , will be said to be equivalent (written rl r 2 ), ifr = r2 si. Similarly, two transformations tl and t2 will be defined as being equivalentif tl = t 2 si. It follows that, if a row assignment is operated upon separately by twoequivalent transformations, the two resultant assignments will be equivalent.For each element p in G, form a subset of G that consists of the elements psi forevery si in S. These subsets are called "left cosets of S."Suppose that q 1 and q 2 belong to the same coset. Then, there are elements sl ands2 in S, and an element p in G so that q 1 = psl' q 2 = ps 2 and s 3 = s2s 1 (s2 is theinverse of s2). Since q 2 s 3 = (PS 2 ) (S2Sl) = p(s 2 s) s1 = = q1 , it follows that q 1 ~ q 2Cosets possess the following properties (proved in sec. 9 of ref. 5):1. Every member of G belongs to some coset of S.2. Each coset contains the same number of elements as S.3. Two cosets of S are either disjoint or identical.4. S is a coset of itself.We have shown that members of the same coset are equivalent. Now we shall demon-strate that the converse is true. Suppose that q1 - q 2. Then, q1 = q 2s 3 and from prop-erty 1, q 2 = ps 2 for some p in G. If s1 = s2s3 , then q1 = ps2s 3 = psl so that q 1 and q2belong to a common coset. Therefore, two transformations are distinct if and only ifthey belong to different cosets. But according to Lagrange's theorem (which can bederived from the first three properties of the cosets listed above) the number of distinctcosets of G is equal to the number of elements in G divided by the number of elementsin S, or(2n)! _ (2n - 1)2nn! n (1)Hence, the number of distinct row assignment transformations and the number of dis-tinct assignments are equal to term 1.For the special case of the four-row table, the problem of finding a set of threedistinct assignments is quite simple. If we start with any arbitrary assignment (as inFig. IX-la) and interchange the states assigned to the third and fourth rows, we have(IX. PROCESSING AND TRANSMISSION OF INFORMATION)one distinct assignment (Fig. IX-Ib). A third assignment, distinct from each of thefirst two, can then be found by interchanging the states of rows two and three in the firstassignment. Symbolically, the two transformations can be written as (0)(1)(32) and(0)(13)(2), respectively. It might be well to note here that, in any particular problem,the number of row assignments that lead to physically different circuits may be less thanthe number of distinct transformations given in term 1, owing to symmetries within theparticular flow table.Now consider an asynchronous circuit in which transitions between all pairs of adja-cent internal states (those states differing in the values of one variable only) occur atsome point in the flow table. Any row assignment transformation that does not preservedistances on the n-dimensional hypercube whose vertices correspond to internal statesof the system will introduce race conditions. Thus, only members of S can be used.This implies that, in general, no nontrivial transformations are possible, although theymay exist for special examples.In the special case of a four-row asynchronous flow table that is strongly connected(in the sense that there are input sequences connecting all pairs of internal states),similar reasoning indicates that no nontrivial row assignment transformations exist.When nontrivial transformations are available, it would be highly desirable to havea method for ascertaining a priori which assignment is the best one - according to somecriterion such as the minimization of the number of necessary combinational elements.The usefulness of a process of this kind can be appreciated, if it is realized that thereare 840 distinct row assignments for any eight-row synchronous flow table. Unfortu-nately, this seems to be an exceedingly difficult problem and I have no suggestions asto how it can be approached.S. H. UngerReferences1. D. A. Huffman, Technical Report 274, Research Laboratory of Electronics, M.I.T.Jan. 10, 1954.2. D. A. Huffman, Proceedings of the Symposium on Information Networks, PolytechniInstitute of Brooklyn, New York, April 12-14, 1954.3. E. F. Moore, Gedanken-experiments on Sequential Machines, Automata Studies(Princeton University Press, 1956).4. D. A. Huffman, Technical Report 293, Research Laboratory of Electronics, M.I.T.March 14, 1955.5. G. D. Birkhoff and S. MacLane, A Survey of Modern Algebra (Macmillan Company,New York, 1953), Chap. VI.6. D. Slepian, Can. J. Math. 5, 185-193 (1954).yc,(IX. PROCESSING AND TRANSMISSION OF INFORMATION)B. CODING FOR BINARY SYMMETRIC CHANNELRecent work by C. E. Shannon and P. Elias has established bounds on the exponentialbehavior of probability of error for finite-length messages transmitted over idealizednoisy channels. They have also shown that random coding is exponentially equivalent tothe best-possible coding, for transmission rates near to capacity (1, 2).The physical implementation of a random-coding procedure is difficult because thenumber of possible messages increases exponentially with message length. Even withmoderate message lengths, this number can become enormous for reasonable rates oftransmission. Small message lengths, however, preclude the strong, large-sample,statistical assertions that are necessary to assure a small probability of error.Research is in progress toward the determination of a physically realizable codingand decoding procedure for the idealized Binary Symmetric Channel. This procedurewill be a compromise between the demands of reasonable equipment on the one hand,and acceptable performance on the other. A random-code message development, usinga sliding parity-check generator, is envisioned at the transmitter. The transmittedmessage of length n is divided for coding and decoding purposes into several smallersublengths n 1 < n 2 = n 1 + A 1 < n 3 = n 2 + A 2 < ... < n = ni- 1 + Ai - 1 < .'". n. In goingfrom any length n i to ni+ 1 , the transmitter inserts a number 5. < A. of completely arbi-trary information digits, filling the rest of the increment Ai with "random" check digitsgenerated as the mod 2 sum of the sliding check pattern and as much of the message ashas already been determined.Correspondingly, at each stage n i of decoding, the receiver progressively comparesthe received message with the set of possible transmitted messages, and discards thosemembers of the set that are beneath a certain threshold of probability. At each stage,a number of possible messages whose individual a posteriori probabilities sum to anacceptable aggregate is retained. In the next stage, then, the receiver need only computethose possible messages that have prefixes which are members of the stored subset.In the end, the process is made to converge to the selection of a single message ofhigh probability. The over-all probability of error is equal to one minus the cumulativeprobability of having retained the correct message at each stage of the procedure.It has been shown that the number m. of prefixes that should be retained at the ith1stage can be made to vary as l/n i by adopting suitable constraints on the incrementalrate R i = 5i/ i . As ni increases, R i can also increase, giving an effective average rateR reasonably close to the maximum rate that is permitted under ideal coding for thespecified design probability of error and block-length n.J. M. WozencraftReferences1. C. E. Shannon, The rate of approach to ideal coding (Abstract), IRE ConventionRecord, Part 4, 1955.2. P. Elias, Coding for noisy channel, IRE Convention Record, Part 4, 1955.

Processing and Transmission of Information

http://dspace.mit.edu/bitstream/1721.1/51959/1/RLE_QPR_042_IX.pdf

Processing and Transmission of Information

Abstract

Similar works

Full text

Available Versions

DSpace@MIT