Contains reports on one research project.National Science FoundationEastman Kodak Company (Grant

Chomsky, A. Noam

DSpace@MIT

XIII. MECHANICAL TRANSLATIONProf. V. H. Yngve Dr. J. R. Applegate Dr. B. SheftsProf. A. N. Chomsky Eva Maria RitterA. ON THE LIMITS OF FINITE-STATE DESCRIPTIONIn the Quarterly Progress Report of April 15, 1956, two types of grammars weredescribed formally: finite-state grammars with no independent memory that producesentences word by word, and [Z, F] grammars which can be represented as slightly lesselementary finite-state processes and which impose phrase structure on the generatedsentences rather than produce them from "left-to-right." A theorem was stated to theeffect that every language describable in terms of a finite-state grammar (every finite-state language) is describable in terms of a system of phrase structure (is a terminallanguage) but not conversely. The natural question to raise is whether or not there areexistent languages that fall outside the range of finite-state description, but within therange of phrase-structure grammars. Further investigation has shown that certainsyntactic properties of English exclude it from the set of finite-state languages, but notfrom the set of terminal languages.Suppose that A represents the alphabet of language L, and thatS = a a ... "an (ai E A) is a sentence of L.Definition 1. S has an (i, j)-dependency with respect to L if and only if(i) 1< i<j < n(ii) there are bi , bj EA so chosen that S I is not a sentence of L and S2 is a1thsentence of L, where S 1 is formed by replacing the i symbol (ai) of S bythbi , and S2 is formed by replacing the j symbol (a ) of S I by bj.Definition 2. D = {(al, 1) .... (a m , Pm)} is a dependency set for S in L if andonly if(i) for 1 < i < m, S has an (a i , Pi)-dependency with respect to L(ii) for each i, j, ai < P(iii) for i * j, ai. a. and pi # j"If S contains an m-termed dependency set, then at least 2m states are necessaryin the finite-state grammar that generates the language L that contains S. Hence, anecessary condition on finite-state languages is that there must be a finite upper limitto the size of their dependency sets. With this condition in mind, we can easily con-struct many nonfinite-state languages. For example, let L 1 be the language containingthe "sentences" aa, bb, abba, baab, aabbaa..., and, in general, all "mirror image"sentences consisting of a string X of a's and b's followed by X read from back to front,and only these. Then, for any m, we can find a dependency set Dm = {(1, 2m), (2, 2m-1),... ,(m, m+l)}, so that L 1 is not a finite-state language.*This work was supported in part by the National Science Foundation; and in part bya grant from the Eastman Kodak Company.(XIII. MECHANICAL TRANSLATION)Turning now to the English language, we find that there are infinite sets of sentenceswith just the mirror-image properties of L 1 . For example, let S 1, S2, S , ... , bedeclarative sentences. Then the following are all English sentences:(1) (i) If S 1, then S2'(ii) Either S3 , or S4.(iii) The man who said that S5 , is arriving today.These sentences have dependencies between "if" and "then," "either" and "or," "man"and "is." But we can choose S1, S3 , and S 5 in (1) as (li), (lii), or (liii) themselves.Proceeding to construct sentences in this way, we arrive at sentences with dependencysets of more than any fixed number of terms, just as in the case of L 1 . English istherefore not a finite-state language.Note that L 1 is a terminal language. It has the [Z, F] grammar with Z = {Z} andF = {Z - aZa, Z - bZb, Z - aa, Z - bb}. Hence, the argument that we have just givendoes not show that English is not a terminal language, since the sentences we have dis-cussed could be given a [Z, F] grammar in the same way as L 1 . The question of theliteral possibility or impossibility of a phrase-structure description of English thereforeremains open, even though there is considerable evidence that more powerful methodsare required if English is to be described effectively.A. N. Chomsky

Mechanical Translation

Contains a report on a research project.National Science Foundatio

XIV. MECHANICAL TRANSLATIONProf. V. H. Yngve Dr. J. R. Applegate Dr. B. SheftsProf. A. N. Chomsky Eva Maria RitterA. THE RANGE OF ADEQUACY OF VARIOUS TYPES OF GRAMMARSBy a language we mean a set (finite or infinite) of strings (i.e., sequences of sym-bols), each finite in length and constructed from a finite set of symbols. By a grammarof the language L we mean a device of some sort for generating all and only the stringsbelonging to L. The principal concern of linguistic theory is to develop a precise andgeneral characterization of linguistic structure and to specify the form of grammars insuch a way that the underlying structure of each generated string can be recovered fromgrammars of the proper type and simple and "revealing" grammars can be provided fornatural languages. No matter how we develop linguistic theory, we shall surely requirethat any grammar associated with the theory be finite in length, hence that the set ofthese grammars be countable. Therefore, any proposed linguistic theory will fail toapply to uncountably many languages. In the last analysis, the adequacy of any lin-guistic theory is determined by the empirical consequences of applying it to actualnatural languages, but we can gain a certain insight into the relative adequacy of variousconceptions of linguistic structure by studying the languages to which these conceptions(and the grammars associated with them) literally cannot apply.The most direct way of describing a language - in a sense it corresponds to theminimum linguistic theory - is by means of a finite-state grammar of the following type.We have a finite set of states labelled So,..I.S . Transition from state S to state Sis marked by production of one of the symbols aijk (1 < k < qij); but for certain pairs(Si, S ) there may be no transition from Si to S . A string Z belongs to the describedlanguage if and only if it is produced by a sequence of states S b ,...,S b , where1 mb = b = 0, and where for 1 < i < m, there is transition between Sb. and Sb . If a lan-1 m 1 i+1guage L is described by some such grammar, we say that L is a finite-state language.Customarily, syntactic description of natural languages is presented in terms ofnotions of phrase structure and immediate constituent analysis. When these notionsare constructed carefully (at least under one interpretation), it can be shown that thesystem of phrase structure underlying a given language can be reconstructed fromgrammars of the following form and that every language describable in terms of phrasestructure can be described by these grammars. We have a finite set Z of initial strings1'... ' n (one will suffice, in fact) and a finite set F of instruction formulas X1 -0Y1'... , X m- Ym, where Xi and Y.i are strings, Yi being formed from Xi by replacementof a single symbol of X i by some string of symbols. We say that a string Z' is directly,This work was supported in part by the National Science Foundation.(XIV. MECHANICAL TRANSLATION)derivable from a string Z in terms of F if Z = aXb, Z' = aYb, and X - Y is an instruc-tion formula of F. Given Z and F, a derivation of the string Z is a sequence ofstrings with some ZEE as its initial term, Z as its final term, and the i t h term directlyderivable from the i-i s t term for each i > 1. In this case, Z is said to be derivablefrom [E, F]. A set of strings that is derivable from some system I , F] we call aderivable language.Given a system [E, F], there may be certain derivable strings from which no furtherstrings are derivable. Any string of this kind we call a terminal string with respect to1[, F]. Any set of strings that is a terminal set for some system [Z, F], we callaterminal language.In the interesting cases, there will be a set S of symbols with these properties:(a) every terminal string is made up of symbols of S; (b) if a is in S then there areno strings a, b, X, Y, p such that X = aab, Y = apb, and X - Y is an instruction formulaof F. In this case, the terminal strings with respect to [, F] can be taken as the lan-guage described by the grammar, and the derivable nonterminal strings give informationabout the phrase structure of the language. Given a derivation of a terminal string Z,we can reconstruct the phrase structure of Z. (Actually, there are certain other con-ditions that [Z, F] must meet if it is to determine a system of phrase structure, but weshall not elaborate here.)The sets of finite- state languages, derivable languages, and terminal languages arerelated in the following way:Theorem 1. (i) Every derivable language is a terminal language, but not conversely.(ii) Every finite-state language is a terminal language, but not con-versely.(iii) There are derivable, nonfinite-state languages, and finite-state,nonderivable languages.To avoid certain inconsequential considerations, we take a language L to be finite-state, derivable, or terminal, if and only if the language L 1 is finite-state, derivable,or terminal, respectively, where L 1 is formed from L by replacing each string X ofL by JXJ, where J is a symbol that is never developed in an instruction formula of F(i.e., JES, as S is defined above).It can be shown that grammars of the form , F] can be greatly simplified if weorder the instruction formulas and define certain of them as obligatory, and if we thenrequire that in constructing a derivation we must proceed through the sequence ofinstruction formulas, applying each obligatory one and perhaps certain nonobligatoryones, and beginning again at the beginning of the sequence after reaching the end. Thiscondition imposes no restriction on derivability. A natural minimum requirement forgrammars would seem to be that the grammar must generate a certain fairly large setof utterances in a reasonable amount of time so that we can actually investigate the(XIV. MECHANICAL TRANSLATION)generative adequacy of the grammar. Suppose we define a proper linear grammar as asystem 1[, Q], where Q is a sequence of instruction formulas Xi - Yi (i < m), and foreach X. (i < m) there is some j such that Xi - Y. is an obligatory instruction formula.This guarantees that every time we run through the grammar each nonterminated deri-vation must be advanced at least one step; that is, we cannot indefinitely run throughthe grammar vacuously. To increase the generative power of these grammars, we permiteach rule to be applied an indefinite number of times to a given string whenever the rulein question is selected for application.We say that a set of derivations is properly producible if it is produced by someproper linear grammar. A set of derivations is said to be producible if it is producedby some system [i, F], where F is a set of instruction formulas, as before. The twotypes of producibility are incomparable; that is, we haveTheorem 2. There are producible sets of derivations that are not properly produc-ible, and properly producible sets that are not producible.The import of Theorem 1 is that description in terms of phrase structure is essen-tially more powerful than description without higher levels. It may be, in fact, thatevery natural language can be regarded as a finite state language - hence a terminallanguage. However, when we actually attempt to construct grammars of the specifiedkinds for natural languages, we find that this description, though perhaps possible, isso complex that it is practically useless. Investigating the situation more closely, wefind that some of the complexity is due to the presence of a large but finite number ofconditions on utterances (e.g., parallelism of constructions) which, if extended to aninfinite set, would take these languages literally out of the range of grammatical descrip-tion of some specified kind. Investigation of the actual descriptive limits of certaintypes of grammars can thus offer hints about the usefulness of these grammars evenwithin their limits. Thus the extra generative power of systems of the form [1, F] overfinite-state grammars is reflected in the considerable gain in simplicity of descriptionachieved by moving to such higher levels as phrase structure. Similarly, we note thatderivable languages can also be constructed in terms of a rather limited type of finite-state process. Given a system [, F], we consider a process with the states S o ... ,S qwith transition from Si to S marked by production of the string Aij. Let Aoi = f fori n (where =11 {l'' ' ' n}) Suppose that F contains the instruction formulasX. - Y. (1 < i < m). Then the state of the process when the string A has just been pro-1 1duced is determined by that subset of X 1 , . . . , Xm whose members are substrings ofA, and we move to the next state by replacing X. by Yi in A for some X. in this subset.With the production of a terminal string, we return to S . The set of strings Aij derivedin this way is a derivable language, and every derivable language is given by some proc-ess of this kind. The extra power of proper linear grammars (as stated in Theorem 2)results from the fact that they transcend this limitation and take into account the(XIV. MECHANICAL TRANSLATION)"history of derivation" of a string (not merely the shape of the string itself) in carryingthrough a derivation. This limitation proves to have extremely undesirable conse-quences in the actual construction of grammars. However, as matters now stand, theextra power of proper linear grammars is actually a disadvantage, since a nonproducibleset of derivations does not correspond to a system of phrase structure. This suggeststhat essentially new conceptions of linguistic structure are necessary, along with moreextensive methods for generating sentences from given sentences and for taking intoaccount the history of derivation (constituent structure) of the given sentences.The fact that not all producible sets of derivations are properly producible opensup new possibilities for testing the adequacy of description in terms of phrase structure,since the requirement of proper linearity seems a reasonable one for grammars. Todemonstrate this requires too lengthy a presentation; but it seems to be true that thesimplest set of derivations for English sentences is not properly producible, althoughthe simplest set of derivations for a certain independently interesting subset of sen-tences is properly producible. This fact adds further weight to the suggestion that anew, higher level of linguistic structure is required so that those sentences that arenot derived by means of the proper linear grammar can be constructed from those thatare so derived by means of transformations that take into account the constituent struc-ture of the terminal strings.N. Chomsky

http://dspace.mit.edu/bitstream/1721.1/51946/1/RLE_QPR_041_XIV.pdf

Mechanical Translation

Abstract

Similar works

Full text

Available Versions

DSpace@MIT

DSpace@MIT