Contains reports on three research projects

Bose, A. G.

Fano, R. M.

Feinstein, A.

Lee, Y. W.

DSpace@MIT

X. COMMUNICATION RESEARCHProf. R. M. Fano A. Adolph R. M. LernerProf. Y. W. Lee A. Bose S. MurogaProf. P. Elias J. Capon J. C. StoddardDr. F. A. Muller A. Feinstein W. A. YoungbloodA. INFORMATION THEORY1. IntroductionInformation theory indicates that the performance of existing communication systemscould be greatly improved by suitably processing the signal before transmission. Inpractice, two major problems confront us when we attempt to realize any substantialimprovement. The first problem involves the identification of the data to be transmittedand the measurement of the corresponding amount of information. The second problem,often referred to as coding, concerns the representation of the data in a form mostsuitable for transmission through the available communication channel.The main difficulty connected with the first problem results from the fact that thecommunication system to be designed may be but a link in a larger system in which theultimate receiver, at least, is a human brain. This is the case, for instance, in teleph-ony and television. Since our knowledge of the human part of the communication systemis still very poor, we are forced to employ a cut-and-try approach to the problemtogether with a measuring technique of the null type. Thus we assume that the signal tobe transmitted belongs to some particular subclass of time function, and then we try totransmit it through a channel of limited capacity by making use of the characteristicsof the assumed subclass. If the linear receiver can extract the desired informationfrom the output of the channel of limited capacity as well as from the original signal,we can state that the signal belongs to the assumed subclass and that the average rateat which information needs to be transmitted does not exceed the capacity of the channelemployed. This approach is being used in connection with speech and picture trans-mission.In more conventional terms, the goal of this research program is the reduction ofthe required frequency band; in fact, reducing the bandwidth of a channel results, fora given signal-to-noise ratio, in a proportionate reduction of the channel capacity. Itshould be stressed that our present primary goal is the determination of upper boundsto the required channel capacities rather than the design of complete transmissionsystems; no attention is being given, for the moment, to the problems presented by thespace separation of transmitter and receiver, nor to those involving equipment com-plexity.In the case of speech communication, the program amounts to ways and means ofimproving the performance of vocoders and similar devices. Better techniques arebeing investigated for the analysis and synthesis of frequency spectra. We are also-38-(X. COMMUNICATION RESEARCH)completing a device that should constitute an appreciable improvement over the speechcompression system of the chopper type that was developed at the University of Illinois(G. Fairbanks et al.: Methods for Time or Frequency Compression-Expansion ofSpeech, Convention Record of the I. R. E., 1953 National Convention, p. 120). In thenew device, the chopping is synchronous with the pitch, so that the natural character-istics of the voice should be preserved even for large compression ratios.The work on picture transmission has been resumed along new lines. Previous workin this Laboratory, as well as elsewhere, has indicated that quantization of the lightintensity is not a promising approach. It appears, instead, that the derivative (possiblein two dimensions) of the light intensity can be quantized to a relatively small numberof levels without introducing appreciable picture degradation. The present programcalls for the determination of the minimum number of levels and for the measurementof their probability distribution. There are reasons to believe that such results willindicate the possibility of reducing the channel bandwidth by an appreciable amount.Work is in progress on the second problem mentioned above: coding. The funda-mental theorem of information theory states that if the data to be transmitted areproperly coded in blocks of increasing size (this operation involves a correspondinglyincreasing delay), it is possible to transmit information at any rate not exceeding thechannel capacity with a vanishingly small probability of error. We present in the nextsection a new proof to this theorem that is more satisfactory, in some respects, thanthe one previously available. It is hoped, in addition, that the new approach will yielduseful upper bounds to the probability of error for finite coding delay.The derivation of a general coding procedure that realizes, in practice, the behaviorpredicted by the new theorem is the second objective of the theoretical part of theresearch program. Such a coding procedure would be of particular practical importancein channels disturbed by nonadditive noise in which the probability of error cannot bedecreased at will by increasing the transmitter power. Progress in this area is expectedto be rather slow because of the extreme difficulty of the problem.R. M. Fano2. A New Proof of Shannon's Theorem for Noisy ChannelsA new proof of Shannon's theorem for noisy channels has been obtained. It yieldsthe further result that the equivocation, as well as the probability of error, vanishes.The main ideas in the proof are summarized below.Let us consider a communication channel defined by a transmitter alphabetx l , x 2 . . . . . xm , a receiver alphabet yl', y2 ... , Ym. Shannon's theorem states that ifwe consider sequences of n symbols it is possible to select from all such sequencesof x-symbols a number N such that if only these sequences are allowed to be trans-mitted, the probability that the receiver will make an error in identifying the transmitted-39-(X. COMMUNICATION RESEARCH)sequences will approach zero as n approaches infinity, as long as N satisfies theinequality log2 N < nC. The constant C represents the channel capacity per symbol;thus this inequality states that the rate of transmission of information must be smallerthan the channel capacity.The basic idea in the new proof may be explained, approximately, as follows. Letus select a sequence X 1 of x-symbols and associate to it a set Sl of sequences Y ofy-symbols in such a way that the sum of the corresponding conditional probabilities forthe specified X 1 satisfies the inequalityY P(Y/X) > 1 - eS1where e is a small constant that will be interpreted later as the probability of error.Next, let us select in succession other sequences Xi of x-symbols, together with thecorresponding sets Si of Y sequences, in such a way that the sets Si do not overlap.The proof amounts, roughly speaking, to showing that if after selecting N sequences ofx-symbols no additional sequences of x-symbols can be found that will meet the statedconditions, then N is sufficiently large to prove Shannon's theorem. More precisely,the probability of error e can be made to approach zero as n approaches infinity, whilekeeping the ratio (log 2 N)/n constant and as close as desired to the channel capacity.The basic result about the magnitude of N consists of the inequality-n(C-E 1 -E 2 ) -2nE 2 1 N2 a n(H(x)- -E 2)NZ > (e-a) 2 - 62 ) - n(H(x)_E)j (1)2where a may be taken as any positive constant smaller than e; 61 and 62, E 1 and E2,are small constants that can be made to vanish simultaneously when n approaches infin-ity; and H(x) represents the entropy of the x-alphabet appearing in the usual computationof the channel capacity. It can be shown further from this inequality that for a given[(log 2 N)/n < C] the probability of error vanishes sufficiently fast with increasing n sothat not only the ratio of the equivocation to the information transmitted vanishes butalso the equivocation itself. This last statement constitutes a new result more powerfulthan the original statement of Shannon's theorem. There are good hopes, also, thatinequality 1 will yield a useful upper bound to the probability of error for finite valuesof n.A. Feinstein-40-(X. COMMUNICATION RESEARCH)B. SECOND-ORDER CORRELATIONThe second-order autocorrelation function of a random or periodic function fl(t) isdefined asTlll(T1,T 2) =lim 2T 1  fl(t) fl(t + l(t + 1 2+ ) dtT-.ooWork is being carried on toward the experimental evaluation of this function. The blockdiagram of the experimental setup is shown in Fig. X-1.f(t)DIGITALCORRELATORDELAYINCORPORATEDIN THECORRELATORFig. X-1Setup for the experimental evaluation of second-order correlation functions.At present the chief source of difficulty in the measurements is the multiplier. Aduplicate of the multiplier designed by Mr. J. Miller, of M. I. T., has been constructedand is being tested. It is desired to have satisfactory frequency characteristics from0-50 kc/sec with a minimum drift in the multiplier.A. Bose, Y. W. LeeC. WIENER'S METHOD OF NONLINEAR SYSTEM CHARACTERIZATIONWork is being started on a method suggested by Dr. Norbert Wiener for the charac-terization of nonlinear systems. The method is confined to those nonlinear systems inwhich the remote past becomes less and less relevant to the behavior of the system aswe push it back in time.The object is to define for such a nonlinear system an operator that can be experi-mentally evaluated for the given system and that once evaluated, permits synthesis ofthe system. The probe for the investigation of the system and the evaluation of theoperator is, as suggested by Dr. Wiener, gaussian noise. The method of characteriza-tion consists of expanding the output of the system in a series of Hermite polynomialsof the Laguerre coefficients of the gaussian noise input.A. Bose, Y. W. Lee-41-

Communication Research

http://dspace.mit.edu/bitstream/1721.1/51153/1/RLE_QPR_032_X.pdf

Communication Research

Abstract

Similar works

Full text

Available Versions

DSpace@MIT