Contains research objectives and reports on two research projects

Elias, Peter

Huffman, D. A.

Unger, S. H.

Contains research objectives and reports on three research projects

DSpace@MIT

XII. PROCESSING AND TRANSMISSION OF INFORMATIONProf. P. Elias E. Ferretti A. J. OsborneProf. R. M. Fano K. Joannou E. T. SchoenProf. D. A. Huffman J. B. O'Loughlin F. F. TungDr. M. V. Cerrillo L. S. Onyshkevych S. H. UngerW. G. Breckenridge M. L. WetherellRESEARCH OBJECTIVESThe fundamental problem of the transmission of information is divided in two parts.The first can be described loosely as the task of eliminating redundancy in naturalmessages - for instance, in speech and in pictures - in order to achieve an economicalrepresentation. This has been called the problem of bandwidth compression. Thesecond can be described loosely as the operation of putting redundancy into an economi-cally represented message in order to match it to a noisy channel and to improve thereliability of communication in the presence of random disturbances. These two aspectsof the problem are intimately related to the two fundamental results of informationtheory.In both of these tasks, switching theory is needed to implement the required codingand decoding devices; however, it is our feeling that there are more fundamental rela-tionships between switching theory and information theory. On the one hand, there arequestions arising from tracing information flow in switching networks, and defining suchnetworks as information-lossless or information-lossy. On the other hand, there arestrong connections between the problem of sending correct information over a noisychannel and the problem of building reliable switching devices from unreliable compo-nents. These are the reasons that led to the merging last year of the switching theoryand information theory groups of this Laboratory.Of necessity, a full investigation of the properties of switching circuits must beundertaken before we shall know which of these properties should be exploited to makecoders for the transmission of information. During the past year a study of the specialproperties of linear switching circuits began, and some means for their orderly realiza-tion were found. These linear circuits are especially promising because they areinformation-lossless and because their force-free responses form a set of multiple-error correcting codes and are nearly noiselike in their correlation properties.A way of designing switching circuits from elements having one particular kind ofunideal behavior was found last year and described in a paper, "The Design and Use ofHazard-Free Switching Networks" (presented at the A. C. M. meeting, September 1955).In this paper it was shown that lack of equality in the reaction times of several switchingelements that are controlled by a single switching signal could be compensated for byadditional paralleled elements with proper control. Success in solving this problemleads us to hope for the achievement of ways of compensating for other kinds of unidealbehavior.Research on the measurement of probability distributions of picture intensities wascompleted last year and will be reported in two forthcoming technical reports (No. 296,by J. Capon, and No. 302, by J. C. Stoddard). Further measurements of the statisticsof pictures and efforts at bandwidth compression continue. A new program of researchon picture processing has been undertaken. The objective of this program is to treatpictures in such a way that detail is reduced without displeasing the observer (forexample, by eliminating photographic grain and still leaving sharp edges). This workwill include the examination of the effect on two-dimensional signals of some operations,such as filtering, clipping, and noise-suppression, which are already familiar in theirapplication to one-dimensional time functions.Two devices for speech compression are under investigation. One is a modifiedvocoder, designed by MUller (Quarterly Progress Report, April 15, 1954, p. 46) theconstruction and testing of which are being completed. The other is a device that trans-mits only one pitch-period of a speech waveform and repeats it several times at thereceiver. Some results from the second of these devices, and perhaps from the108(XII. PROCESSING AND TRANSMISSION OF INFORMATION)first, should be forthcoming.Work on coding for noisy channels continues. During the past year there wasan appreciable amount of progress in the theory of this field, in relating the transmis-sion rate, the channel capacity, the error probability of received messages, and thepermissible time delay in coding and decoding. The task of constructing coders anddecoders, generally speaking, remains difficult, but for one kind of channel, the erasurechannel, it is becoming manageable. The possibility of making use of the erasurechannel to combat the fading problem in scatter transmission is being investigated.P. Elias, D. A. HuffmanA. TIME-VARYING LINEAR BINARY CIRCUITSBinary circuits synthesized from modulo-two adders and unit delays that may containfeedback loops have been shown to be information-lossless networks and to exhibitresponse properties similar to those of conventional linear electric networks (1).Perhaps their most interesting characteristic is the possibility of exciting in themforce-free, steady-state periodic outputs. These outputs correspond to the nondecayingsteady-state responses of an electrical filter composed only of inductances and capaci-tances.A more general class of linear binary circuits utilizes not only delays and modulo-two adders but also a "commutator" element. The latter element is represented inFig. XII-1. Its input is connected to each one of N outputs once every cycle of N unitsof time. We define alternately N corresponding multiplicative operators C , C 1 , C ....C , CN- 2 CN-1 so that only one at a time is equal to unity and so that C j+1 = 1 justone unit of time after C. 1. (Note also that C = 1 one time unit after C N - 1 = 0.) Thus,C 0 if j kThe product of operators C. and D (Dq is a delay of q time units) represents a cas-cading of the corresponding circuit elements. It may be reasoned physically (seeFig. XII-2) that C.D q  D C. because the binary signal D C.X cannot possibly equalJ th 3unity unless the commutator wiping arm is in the (j+q) position. (Since the commu-tator arm starts another cycle after the (N-l) position, the number j + q must be con-sidered to have been taken modulo N.)cIXXX DX CjDXC D Xx----4 .r j C j X [ -n DqI C, X = C D I XCN-2X i I -Fig. XII-1. A "commutator element." Fig. XII-2. Illustrating that CD q DC109(XII. PROCESSING AND TRANSMISSION OF INFORMATION)x o + D + z XFig. XII-3. Circuit realization of Fig. XII-4. Realization ofZ = X + D(Z + CoX). Z 2 30oX = C +C D+C D +C D.X o 1 2 3The terminal description of a time-varying linear binary circuit can be accomplishedby means of a transfer function written as the ratio of two polynomials in the delay oper-ator D and whose coefficients are C operators. Particular care must be taken in thealgebraic synthesis procedure, since the C operators form a "ring" (the cancellationlaw does not always hold) and the order of forming products is important.The equations below represent steps in the complete synthesis of a typical circuit.The derived circuit diagram is given in Fig. XII-3. The circuit has a "behavior-period"of two units of time; thus there are only two C operators: C and C . The transferratio is to be2z C0 D +D+CZ o 2X C D 3 +C D 2 +Io 1which may be reduced toC 1  + D +D+ I)(CD 2 + D + C 1  C1 D + C 1  C(D+I)X 3 D+I -D+IC 1 D2 + D + I) (CoD3 + C I D2 + I)(the common right-factor (D+I) may not be cancelled!) andZ = DZ + C1DX + X = X + D(Z + CoX)Time-varying circuits of this class can be information-lossy, and for this reasonmay not have inverses. Their responses to input impulses need not be periodic (as inthe circuits of ref. 1) but may actually terminate after a finite interval of time.The algebra of time-varying binary circuits allows us to express in compact formprocedures that are likely to be useful in the coding of information to be transmittedthrough noisy binary channels. For instance, a circuit that takes every fourth input andrepeats it four times has a transfer ratioZ 2 3Z C +C D+CD +CDX o 1 2 3110(XII. PROCESSING AND TRANSMISSION OF INFORMATION)We can show algebraically that only one delay element is necessary for synthesis. Thus,C CD+CZ=C +C D+C D +C D= +C +C )D+ +C 3 )D+IX o 1 2 3 (C 1 +C +C 3) D + I (C2 3) D + IThe last ratio can be realized in the form given in Fig. XII-4.D. A. HuffmanReferences1. D. A. Huffman, The synthesis of linear binary coding networks, Thirdon Information Theory, London, September 1955.SymposiumB. DESIGN OF LINEAR SEQUENCE FILTERS FOR ARBITRARYIMPULSE RESPONSEThe discussion that follows demonstrates that any single-input, single-output, non-time-varying linear sequence filter can be synthesized from delay elements and modulo-two adders only. A similar conclusion holds for (possibly) time-varying circuits,including those that have more than one input and more than one output.We consider that a linear sequence filter is completely specified if its response toan input sequence that contains but a single "1" symbol (the "impulse") is given. Inorder for the filter to be physically realizable the l's in the response sequence cannotoccur before the input "impulse" occurs. Furthermore, if the response is to be thatof a finite filter the impulse response must ultimately become zero or become periodicwith a finite period. As an example, note th t t e arbitrarily pecified i pulse r s onse, ZAs an example, note that the arbitrarily specified impulse response, Z , ofFig. XII-5 can be resolved into two parts.X*: --- 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0---Z*---0000 100100 1 1001 1001 1001 I ---Z*: --- 0 00,1 00 11 00 1, 1 00 I,1 00 II 00 1,1---Z* --- 0 0 0 I I 0 0 0 0 0 0 0 0 0 o 0 0 o00 0---Fig. XII-5. Resolution of an impulseresponse into periodic andtransient components.a consequence of a subfilter (Fig. XII-6b)a single feedback path and produces fromOne of these (Z ) is the ultimately periodicPresponse behavior extended back to thetime of the input impulse. The other com-ponent (Z ) is the transient part of thetresponse and is chosen so that Zp +Z = Zp tThe transient impulse-response (Z )can be considered a consequence of a sub-filter that has feed-forward paths only andof the form given in Fig. XII-6a. Theperiodic response (Z ) can be consideredpthat is resolved into two parts: the first hasthe input impulse a train of I's at equallyspaced intervals equal to the ultimate repetition period of the response; the second111~qa~ ___ _ I I -- I --- Iraab(XII. PROCESSING AND TRANSMISSION OF INFORMATION)X Z I + D + D 3)z l(a)+D+D)X(a)*-x_XI+D+DX ZI + D(C)Fig. XII-6. Synthesis of a binary filter to meet impulse response specifications.Fig. XII-7. Chain realization of a binary sequence filter.produces, from each 1 symbol put into it, a finite response equal to the (repeated)typical cycle of the ultimate response.The composite filter that has the desired impulse response is given in Fig. XII-6c.This filter will not, in general, be economical in. its use of delay elements. The fol-lowing algebraic simplification procedure and the final circuit of Fig. XII-7 are onefurther example of the usefulness of an algebra of polynomials in a delay operator:Z 3 I+D 3X II+D+D +X I+D 4Z D(I + D 3 + D 4 + D )XI+ D4Z D(I + D + D 2 + D4 + D5)X I + D + D D3I+D+D2+D 3Z + DX = DZ + D - DX + D Z + D 2  + DX + D 4 D DX + D 5 DXZ + DZ = D fZ + DX + D fZ + DX + D fZ + D fDX + D"- DX}}}1112(XII. PROCESSING AND TRANSMISSION OF INFORMATION)The design of the more general multi-input, multi-output, time-varying circuit willbe the subject of a forthcoming paper, "An Algebraic Synthesis Procedure for LinearBinary Sequence Filters." The effect of a single input digit at one of the inputs of sucha filter may be "smeared" over the various output sequences in an arbitrarily specifiedmanner, hence it will be foreseen that such filters (especially when the number of out-puts exceeds the number of inputs) may be particularly useful in the redundant codingof information so that it will be less vulnerable when transmitted through noisy channels.D. A. HuffmanC. TIMING CONSIDERATIONS IN CLOCKED SWITCHING CIRCUITSIn a sequential switching circuit, either clocked or synchronous, the problem of raceconditions is solved by permitting changes of state to occur only during certain limited,and generally periodic, time intervals defined by pulses generated for this purpose. Tosecure reliable system operation certain constraints must be satisfied by relevantcircuit parameters, the duration and period of the clock pulses, and the tolerancesinvolved. In the discussion of these relations an idealized model of a clocked systemwill be used and a procedure for obtaining the maximum operating speed will bedeveloped.Figure XII-8 shows the kind of system that is being studied. The box outlined bydotted lines can be considered one of several binary memory elements that can berealized physically in many different ways.The system operates as follows. Between clock pulses there is no output andthere is no connection between the combinational circuit and the inputs to the memoryelements. The local feedback loop around each amplifier holds the output of the corre-sponding D1 constant. When a clock pulse (CP) arrives, the combinational circuitoperates on both the system input (assumed to be at the proper level when the CP goeson) and the output of the memory to produce both the system output and a set of inputstates fed to the memory elements through the multiplier gates controlled by the clockpulses. The delays D 1 and D 2 keep the memory output from changing while the CP ison. (D 2 alone would suffice; D 1 is the unavoidable delay associated with the amplifier.)Notice that in this mode of operation the delays serve a secondary purpose, the princi-pal storage function being accomplished by the feedback loop.The symbols used are as follows: TD1 TDZ, and T c represent the average valuesof the delays, D 1 and D 2 , and the CP width, respectively. The minimum allowable timebetween leading edges of adjacent clock pulses is T p. The maximum normalized devia-tions of D1 , D 2 and the pulsewidth are designated ED 1 , EDZ2 and E c , respectively.For example, the delays D 1 are assumed to be within the range from TD1(1 - ED1) to113(XII. PROCESSING AND TRANSMISSION OF INFORMATION)SYSTEMOUTPUTi DELAY~-- - DC AMPLIFIERIIXJ---LOGICAL MULTIPLIER-~--LOGICAL ADDERI I YFCLALELEMENT -- LOGICAL INVERTERFig. XII-8. Clocked sequential circuit.TDl(1 + ED1) under all conditions. Obviously, the E's must lie between zero and one.It is appropriate to inquire what relations must exist among these quantities if thesystem is to operate properly regardless of the number of memory elements that changetheir states simultaneously. First of all, the inputs to the combinational circuit fromthe memory must not change while the CP is on:TDl(1 - ED1) + TD2(1 - ED2) > Tc(1 + Ec) (1)Furthermore, the outputs of the D 1 elements must change before the CP ends (if theyare to change at all):TDI(1 + ED1) < Tc(1 - E c )  (2)Finally, enough time must be allowed between the leading edges of successive CP's topermit the effects of changes of state while the first CP is on to emerge from the appro-priate D 's before the next CP starts:TDI(1 + E+) + TD2(1 + ED2) < Tp (3)In designing this kind of system it is necessary to find TD2, Tc, and Tp as functionsof TD1 and to determine the estimated tolerances. Generally, it is desirable to mini-mize Tp so as to maximize the operating speed, which would indicate that the equalitysign should be used in Eq. 3. Next, solve Eq. 1 for TD2 to obtainTc(1 + E c ) - TDl (1 - ED1 (4)D2 1 - E (4)The equality sign in Eq. 4 minimizes TD2 , which is a necessary step in minimizingTp, as shown by Eq. 3. Substituting Eq. 4 in Eq. 3 and simplifying the new expressionyields114(XII. PROCESSING AND TRANSMISSION OF INFORMATION)2(ED1 - ED2) (1 + ED2)(1 + Ec)T = ED T + 1T (5)p 1-ED2 D1 1 - ED2 cThis in turn shows that Tc should also be made as small as the constraint of Eq. 2 willallow. Substituting that minimum value in Eq. 5 reveals that the smallest allowable Tis equal to an unpleasant-looking, positive constant multiplied by TD1. Thus TD1 shouldbe minimized; and T c, TD2, and Tp can be found by successive substitutions in Eq. 2(using the equality sign), Eq. 4, and Eq. 3 (using the equality sign), respectively.To take an example, suppose that D 1 is conservatively estimated to lie between0.06 psec and 0.14 psec. Then TD1 =(0.06+0.14)/2 = 0.1 sec, and ED1 = (0. 1 -0. 6)/0. 1= 0.4 psec. Furthermore, let us assume that D 2 can be constructed to operate reliablywith a tolerance of +20 per cent and that deviations in the clock pulsewidth can beheld within +15 per cent. Then ED2 = 0. 2, and E c = 0.15. From Eq. 2 we findTc = [(1 + EDI)T D1/1 - Ec = 0. 165 Lsec. Equation 4 and the previous result indi-cate that TD 2 should equal 0. 163 sec; combining these results with Eq. 3 yieldsT = 0. 34 p sec.PThus we find that the maximum operating rate of this system would be obtained byusing a value of 0. 163 psec for D 2 and a clock source that produced pulses 0. 165 1 secin width at a rate never exceeding one every 0. 34 Isec.S. H. Unger115

Processing and Transmission of Information

XI. PROCESSING AND TRANSMISSION OF INFORMATIONProf. S. C. Caldwell W. G. Breckenridge A. J. OsborneProf. P. Elias J. Capon J. C. StoddardProf. R. M. Fano E. Ferretti S. H. UngerProf. D. A. Huffman E. J. McCluskey, Jr. W. A. YoungbloodRESEARCH OBJECTIVESIn the Quarterly Progress Report of January 15, 1954, R. M. Fano gave an intro-ductory discussion of the two kinds of problems that arise in applying information the-ory. The first is the determination of the amount of information generated by a messagesource, such as a speaker or a television camera. The second is the representation ofa message in the best form for economical transmission over an imperfect channel.In connection with problems of the first kind, work on speech and picture compres-sion was discussed. These programs are nearing the end of the first phase of equipmentconstruction. During the coming year descriptions of equipment, calibration procedures,and preliminary measurements will be reported as progress is made.Work on problems of the second kind is also continuing. This work is centered onthe simplest noisy channel: a binary channel that transmits zeros or ones and has theprobability po of transmitting any digit incorrectly. The channel is so simple that it ispossible to compute answers to a variety of detailed questions about error-correctingcodes designed for it. It is also possible to build coders and decoders out of relays andtrigger-pairs. This means that such codes can be tested, and some preliminary con-struction of apparatus for this purpose is under way.Since information is a measurable quantity it can be considered somewhat analogousto energy (1). For example, certain switching circuits realized by means of delay linesand adders have this property: they may alternately store and give up the informationsupplied to them in the form of a sequence of binary digits in the same way that networkswith inductive and capacitive elements store and give up the energy supplied to them inthe form of time "sequences" of voltage and current. If certain restrictions are met,such information filters may be "lossless"; the analogous electrical-energy filters wouldbe termed "reactive."An intensive study of the properties and possible syntheses of information-filterswas undertaken in the belief that proper internal feedback within such filters may sta-bilize them against the effects of component failure and external noise. From the view-point discussed above, the problem of obtaining error-free information from a noisycommunication channel can be related to the problem of designing a feedback networkwhich is lossless for the transmitted signal but lossy for external disturbances. Somedetails of the work mentioned above will soon appear as a technical report.A practical realization of information filters might be in terms of sequentialswitching circuits. Switching theory is also developing useful techniques for the treat-ment of indefinitely extended sequences of binary variables. Therefore, steps have beentaken to insure close cooperation among people with interests in both information theoryand switching theory.P. Elias, D. A. HuffmanA. THE SYNTHESIS OF ITERATIVE SWITCHING CIRCUITSOne of the properties of an iterative, or "positional, " switching circuit is that itsability to transmit may, in general, depend upon a large number of binary switchingvariables. For example, the extended relay-contact network shown in Fig. XI-1 trans-mits a ground to the output terminal if and only if the number of operated relays in the-63-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)X 2XI X2CIRCUIT Xi Xn-I XINPUTSFig. XI-1An "even" circuit.circuit is even. (Unprimed contacts are normally closed; primed contacts are normallyopen.) It is clear what the fundamental structure of a typical cell of this network is, andthis structure (as in all iterative networks) remains the same throughout the networkeven if the number of variables is large.In the synthesis of iterative switching networks (2) it has been difficult to be certainthat the specification of terminal requirements is exact. One conceivable (and accurate)way to describe the transmission of a network with n binary variables would be to listan output of either one or zero (ground or no ground for a contact network) for each ofthe 2n possible input states in a table of combinations (truth table). Clearly this tablewould become excessively large for relatively small values of n. (For example,210 = 1024.)What is needed is a method of specification that becomes complex only if the struc-ture of a typical cell in the network is complex, and from which the typical cell itselfmay be synthesized directly.The synthesis method proposed here satisfies these requirements and is based uponthe fact that the variables in an iterative circuit are ordered from left to right, just asthe input to a sequential switching circuit is a sequence of digits ordered from the pastto the future. Therefore, an iterative circuit can be considered a sequential circuit withthe variables ordered in space rather than in time. And we should be able to describean iterative circuit by means of a flow table (3).As an example we shall synthesize a contact network that transmits a ground if andonly if at least two of the relay variables are equal to unity. A typical cell in such anetwork must "know," by means of the grounds on its input leads, how many variablesto its left have the value unity and must transmit this knowledge - modified accordingto its own binary value - to the right (to the next cell).Only three classifications of data need be distinguished by such a cell. Either1. no variables to the left have the value one, or2. one variable to the left has the value one, or3. two or more variables to the left have the value one.-64-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)These three mutually exclusive states have been assigned the state-designators, "0,"i1," and "2," respectively, and are associated with the three rows of the flow table inFig. XI-2(a). The two columns of the table correspond to X. = 0 and X. = 1.1 1The six entries within the flow table itself specify the way in which the state of thecircuit is modified when the value of the variable within a new typical cell is considered.For example, the entry "2" in the row associated with state "1" (the middle row) andwith Xi = 1 (the right-hand column) is interpreted as follows: If to the left of a typicalcell there is just one variable with the value unity, and if the value of the variable in thecell is also unity, then the data to be sent on to the next cell to the right are that twovariables have the value one.The output data in Fig. XI-2(a) tell whether or not the circuit should transmit aground when the contact network is terminated just to the right of the typical cell underconsideration.The knowledge about the circuit state which is given to a typical cell is coded as aset of grounds on one or more input leads to the cell. The knowledge given to the nextcell to the right is coded in exactly the same way. In our present example (and also inFig. XI-1) two leads are sufficient. They correspond to states "1" and "2." That is,the state "O" is associated with no grounds on the cell leads; the state "1," with a groundon the first lead; the state "2, " with a ground on the second lead.Now the data in the flow table must be used to show how to connect the contacts withina typical cell (see Fig. XI-2(b)). The bottom pair of "2" entries in the flow table tells usthat if the second input lead is grounded, the second output lead is to be grounded,regardless of whether X i =0 or X. = i. Thus the second input lead is tied directly to thesecond output lead.The entry "2" in the middle row of the flow table instructs us to ground the secondoutput terminal if the first input lead is grounded and if X. 1. The diagonal X. con-1 1tact in the typical cell accomplishes this.The entry "1" in the top row tells us that when neither input lead is grounded andwhen Xi = 1, the first output lead should be grounded. The proper result is obtained bymeans of the X. contact, which has one terminal connected to ground. The other entry1"1" (in the X. = 0 column) corresponds to the X' contact.1 1When an indefinite number of typical cells is connected in cascade (Fig. XI-3), it isfound that two of the contacts in both the left-hand and right-hand cells are superfluous.All that we need do to complete the synthesis is to take the circuit output from the sec-ond output terminal of the last cell. The resultant contact network transmits a groundif and only if there are two or more operated relays in the circuit.Because the flow table specification of an iterative circuit synthesis problem doesnot mention the physical nature of the binary variables, it holds, also, for example, foran electronic realization. A typical cell for one possible analogous vacuum-tube circuit-65-X i: I OUTPUT:STATES: O 02 2 0(C)x=I ~ ~ Ix b(b)The flow table andFig. XI-2typical cell for an illustrative iterative circuit.x x x',xx x, _ _ T Xn , X-I T CIRCUITOUTPUTX X Xi Xn-1 XnCIRCUITINPUTS OFig. XI-3The derived relay circuit.I T T (I)(2) P *(2)Fig. XI-4The typical cell of an analogous vacuum-tube circuit.-66-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)is shown in Fig. XI-4. For this circuit, X. has the value one or zero, depending onwhether the input voltage X i is high or low. The symbol T refers to a triode or to twotriodes in parallel with a common plate-load resistor, and the symbol P to a pentode.We indicated that the flow table is an implement in the synthesis of a wide varietyof switching circuits; its use is valid whether the variables be ordered in space or intime. The complexity of the description for an iterative circuit does not depend uponthe number of variables any more than the complexity of description for the synchro-nous sequential circuit depends upon the length of time it is allowed to operate. Finally,the methods that have already been developed for the simplification and manipulation offlow tables (3) have been shown to have a double usefulness.D. A. HuffmanReferencesI. D. A. Huffman, Proceedings of Symposium on Information Networks, BrooklynPolytechnic Institute, April 1954.2. W. Keister, A. Ritchie, and S. Washburn, The Design of Switching Circuits (D.Van Nostrand Company, Inc., New York, 1951), pp. 56-58.3. D. A. Huffman, Technical Report 274, Research Laboratory of Electronics, M.I.T.,Jan. 10, 1954.B. SEQUENTIAL SWITCHING SYSTEMS WITH CLOSED MEMORYI. IntroductionBefore discussing this problem it is necessary to define some of the basic conceptsof switching circuit theory.Consider a switching system with several input and output terminals. Each terminalhas associated with it a binary (two-valued) variable which may be physically repre-sented by the presence or absence of a conduction path to ground, the presence orabsence of a voltage pulse, one of two voltage levels, and so on. The possible statesof these variables are generally designated as either 0 or 1.The input state applied to a system is defined as the combination of values of thebinary variables at the input terminals. The output state produced by the system issimilarly related to the output terminals. (The terms system input or system outputwill occasionally be used.)A switching circuit is referred to as "combinational" if the output state at any timeis uniquely determined by the existing input state. (Ideally, no delay is involved.)Detailed studies of the principles of combinational switching circuits will be found inreferences 1 and 2.A sequential switching circuit is characterized by the fact that its present output-67-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)state is, in general, a function of both the present input and the past inputs. This factnecessitates the use of a memory (usually composed of 2-state elements) whose functionit is to provide information concerning previous input states. The memory state of asystem is the condition of all its memory elements. (A memury composed of m binaryelements can be in any one of 2 m states.)For simplicity, the following discussion is restricted to synchronous, or clocked,systems where all inputs, outputs, and other actions occur in cadence with a sequenceof reference pulses. This restriction is not of fundamental importance, since asynchro-nous systems can be treated in a similar manner.Figure XI-5 depicts the signal flow in a synchronous sequential switching circuit.It is a functional diagram; the various parts do not correspond to physical entities.The operation of the system described by the diagram is as follows. When a clockpulse arrives, the combinational circuit accepts at its input terminals the system inputand the memory state. Some of its output terminals provide the system output; otherslead to the memory input and specify the new memory state, which can be selected arbi-trarily. Since there is a delay associated with the memory input, the memory statecannot change while the clock pulse is on.A thorough general treatment of sequential switching circuits was pre-sented byD. A. Huffman (3). One of the principal results of his paper is the flow table, a com-pact method of specifying, completely, any sequential problem. A companion resultis a direct, though not unique, method for synthesizing a sequential circuit from anyflow table.2. Open and Closed MemoriesThe memories considered specifically by Huffman have the property that the statesof all of their elements can be displayed simultaneously to a combinational circuit; fur-thermore, the states of any number of elements can be altered at one time. Thesecharacteristics make possible the two flow lines------------------ in Fig. XI-5 which connect the combinationalCOMBINATIONALCIRCUIT circuit with the memory. Such memories willCLOCK PULSEOUTPUT STATE be designated as "open," since their states areINPUT STATEat all times fully open to inspection and controlPRESENT NEXTMEMORY MEMORY by a combinational circuit. Open memoriesSTATE STATEcan be constructed with relays, vacuum tubes,transistors, or other devices.L ------------- There are also memories without the fea-Fig. XI-5 ture described above. That is, at any partic-Flow diagram of a synchronous ular instant, the states of only a limited numbersequential switching circuit. of elements, or cells, can be made known to-68-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)the combinational circuit. In addition, the states of only a limited number of elementscan be altered during any one time interval. Such memories, which include electro-static storage tubes and magnetic core matrices, will be referred to as "closed," toindicate that at any particular time most of their elements are closed to inspection orcontrol. This discussion will be limited to those memories in which, at each instant(excluding transient intervals) a single cell is being "scanned," and only the state of thiscell can be determined or changed.The terms "reading" and "writing" are used to indicate the operations of inspectingor changing, respectively, the contents of a cell.There are at least two general ways in which closed memories can be, and are being,used. First, they may serve simply as data storage units. Binary coded sequences maybe written into the cells, and then, at a later time, portions may be read out intosmaller open memories so that they can serve as inputs to combinational circuits.Although this is a very important function, it is of little significance from the point ofview of this study. The second, and more interesting, use of closed memories is insequential switching circuits, where they can sometimes serve as replacements foropen-memory elements.It can be seen from the preceding definitions that closed memories are less flexiblethan open memories. However, considerations of such matters as size and cost mili-tate against the exclusive use of open memories when a large memory (say thousandsof elements) is required.3. A Synchronous Sequential Switching System with Closed MemoryConsider the system described by Fig. XI-6. The number of states of the openmemory is equal to the number of elements of the closed memory. The selector, whichis controlled by the open memory, is a combinational circuit that selects the cell to beCOMBI----------------------------NATIONAL --I COMBINTION~LCLOSED o r.LrVMEMORYL ----- -- -- - - --- - --------------Fig. XI-6Synchronous sequential switching system with closed memory.-69-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)scanned. For each state of the open memory there is one element of the closed memory.When a clock pulse arrives, the combinational circuit inspects the system inputstate, the open memory state, and the state of the scanned cell. On the basis of thisinformation, it fixes the system output, orders a one or a zero to be written in thescanned cell, and specifies the next open memory state (which is equivalent to choosingthe next closed element to be scanned). A delay inherent in the open memory preventsits output from changing before the writing operation is over.Note that the system has an over-all memory of mZm states, where m is the numberof closed cells. (The closed memory has 2m states, and at any time any one of the mcells may be scanned.)Since the identity of the cell to be scanned during a time interval is dependent uponthe output of the combinational circuit during the previous interval, a time translationis necessary, thus requiring the use of the open memory of Fig. XI-6. Because thesize of this memory is logarithmically related to the size of the closed memory, it ismuch smaller than the latter for most cases where this type of system would be useful.The mode of operation just described is certainly not the only one possible. Addi-tional open elements might often be profitably added, or various other models requiringless open memory could be devised. An advantage of this model is that some practicalclosed memories have inherent in their access circuitry the open memory describedhere, and no additional open elements need be added.4. Preliminary ResultsFor the sake of brevity, proofs of the following statements will be omitted.1. Sequential circuits with closed memories can be analyzed in a straightfor-ward manner by means of flow tables (3).2. There exist sequential circuits with m closed elements and n open elementswhich require at least m + n open elements when no closed memory is used. In otherwords, in these cases, the closed memory is equivalent to an open memory of the samesize.3. There exist sequential circuits with n open elements which cannot be realizedwith fewer than n - 1 open elements, regardless of how much closed memory is used. Inthese cases closed memories are worthless.Problems falling into the two preceding categories represent an extremely smallfraction of the aggregate of all possible sequential problems; most cases will be of anintermediate kind. The classification of any particular flow table remains to be investi-gated, along with the problem of devising formal synthesis techniques.4. The following interesting, though apparently inconsequential, fact has beenproved concerning cycles in closed memory systems with no inputs: While it is alwayspossible to design an open memory system that cycles through all of its memory states,-70-(XI. PROCESSING AND TRANSMISSION OF INFORMATION)this can never be done with a closed memory system.5. Several diagrammatic representations of possible memory state transitionsin closed element systems have been devised. (One of these consists of a hypercube withclipped corners.)S. H. UngerReferencesC. E. Shannon,S. H. Caldwell,D. A. Huffman,Jan. 10, 1954.Trans. AIEE 57,Notes for CourseTechnical Report713-723 (1938).6.567 (Switching Circuits), M.I.T.274, Research Laboratory of Electronics, M.I.T.,-71-

http://dspace.mit.edu/bitstream/1721.1/51254/1/RLE_QPR_036_XI.pdf

Processing and Transmission of Information

Abstract

Similar works

Full text

Available Versions

DSpace@MIT

DSpace@MIT