Buch A, Pietsch C. Measuring Syntactic Priming in Dialog Corpora. In:  Proceedings of the Conference on Linguistic Evidence. Empirical, theoretical and computational perspectives. Tübingen: Eberhard-Karls-Universität;  2010

Buch, Armin

Pietsch, Christian 

English

Publications at Bielefeld University

Measuring Syntactic Priming in Dialog CorporaArmin Buch and Christian PietschUniversita¨t Tu¨bingen and Universita¨t Bielefeldarmin.buch@uni-tuebingen.de,cpietsch+le@uni-bielefeld.deAbstractWe devise a simple, distribution-based measure of priming between linguisticcategories. Priming is found in tree banks of dialog corpora, both for context-free production rules and for Combinatory Categorial Grammar categories. Itis stronger for task-oriented dialogs, and stronger in lexical categories than insyntactic categories.1 IntroductionPriming1 is the phenomenon by which a recently encountered event (or seen item)is recognized more quickly or more likely to be repeated. Presumably this is due toactivation levels in the brain. Production and perception prime each other, as theyaccess the same or closely related representations. Direct priming only acts on surfacesimilarity. The according effect for learned relations is called associative or semanticpriming. Only the latter is affected by age, amnesia etc., so it is an effect of memory(Tulving and Schacter, 1990). One expects it to decay exponentially over time.In language, hearing or speaking a word facilitates the processing of similar soundingwords. The so-called semantic priming arises between semantically similar concepts,between syntactic categories etc. Priming has been attested for in single constructionssuch as the English dative alternation (Bock and Griffin, 2000). Any identifiable unitin a linguistic structure could be subject to priming. If it is, this supports the linguistictheory which proposed that structure. “[R]epeatable structures are evidence for theunits of linguistic cognition” (Reitter, 2008, sec. 1.2). Priming of syntactic categorieshas been found by Reitter et al. (2006a,b).1This research has been supported by the EC 277 “Cognitive Interaction Technology” of the DFG atBielefeld University. We would like to thank David Reitter and Julia Hockenmaier for kindly providingus with their corpora, and our adviser Gerhard Ja¨ger for creating and guiding this project.12 Experiments2.1 PreliminariesClassical priming experiments such as Bock and Griffin (2000) study a single, theory-neutral alternation in controlled experiments. In contrast, we study the distribution ofeach category in large annotated syntactic corpora. Every sub-structure of an annota-tion is a possible category. We follow Reitter et al. (2006a,b) in using CombinatoryCategorial Grammar (CCG) categories and context-free production rules. The lattercould be extended to subtrees as in data-oriented parsing (Bod, 1998).CCG assumes that there are many equivalent derivations for a given sentence analysis:the same lexical categories, but different modes of combination. Among these, thenormal form derivation is the one along the lines of constituent bracketing, which ismostly right-branching for languages such as English. The incremental derivation isas left-branching as possible; see Reitter et al. (2006a) for details.We use the same data as Reitter et al. (2006a,b): The Switchboard corpus, annotatedwith context-free rule expansions (sw-CFG) and with CCG categories (sw-CCG-Iand sw-CCG-N for incremental and normal form derivations, respectively; and theMapTask corpus with CFG annotation (mp-CFG). We also look at lexical priming(sw-words).2.2 A simple measure of primingPriming as mental activation of representations cannot be directly measured (yet). Incorpus studies, we observe the distribution of a category. The null hypothesis is arandom distribution, described as a Poisson process. For this, the (temporal) distancesbetween adjacent occurrences are exponentially distributed (p(x) = λ0e−λ0x), whereλ0 equals the frequency of the category.We fit an exponential curve with decay parameter λ to the actual distribution of dis-tances. If there is priming, shorter distances should be more frequent than longerdistances (λ > λ0). The ratio r = λ/λ0 can be interpreted as priming strength.2.3 Single CategoriesFig. 1 shows the estimated density function, a random distribution (dotted line), andthe fitted, much steeper exponential (dashed line), for the expansion VP→ VB S.Across all corpora, estimated parameters λ are always larger than λ0. Rare categoriesshow more priming with r up to 2.3, and close to 1 for the very common expansionS→ NP VP (0.34 occurrences per second). Exponential decay fits well, with standarddeviation around 0.005.2DistanceDensity0 50 100 150 200 250 3000.0000.0050.0100.0150.020Figure 1: Distribution of pairwise dis-tances of VP→ VB S.DistanceDensity0 1 2 3 4 5 60.00.51.01.52.0Figure 2: Average of normalized distribu-tions in MapTask.The exponential decay supports the suggestion that priming is an effect of (short-term)memory. While frequent categories have generally less room for skewed distributions,there is still something more to be explained about the effect of frequency. Besidesthat, our results once more confirm the existence and pervasiveness of priming.2.4 Corpus averagesMeasuring the overall priming in a corpus allows to compare several settings: differentlinguistic frameworks (CCG vs. CFG), spoken vs. written language, conversational(Switchboard) vs. task-oriented (MapTask). We normalize all categories for frequency(s.t. λ0 = 1) and take the average.Corpus decay parameter λ standard errorsw-CFG 1.1589 0.0044sw-CCG-I 1.0523 0.0054sw-CCG-N 1.0364 0.0051mp-CFG 1.4666 0.0049sw-words 1.2521 0.0113Standard errors are low, we have thus a good estimate of the actual distribution ofdistances. Yet fig. 2 suggests an even more extreme distribution. This might be a resultof cumulating activation: short distances trigger more short distances.We see strong lexical priming (1.25). Task-oriented dialog outranks conversationaldialog (Reitter et al., 2006b; Pickering and Garrod, 2004). CCG annotation showscomparably little priming. Results by Reitter et al. (2006a) stated that it is significant,but that the difference is not.33 ConclusionWe have devised a notably simple priming measure. A single parameter λ per category(or per corpus) suffices, modeling the distribution of distances. Experiments show it tobe larger than its expected value, which is the category’s frequency. The effect appearsto be larger for rare categories. Interpreting the fitted λ as a frequency is somewhatparadoxical: Primed categories appear more frequent than they actually are.So far we have viewed categories as mutually exclusive. This does not take into ac-count priming of similar categories. Adding pairwise similarities to the model couldimprove it. A simple example is stemming or lemmatization: A word also primes allinflected forms.The more priming a category shows, the more it can be taken as psycholinguisticallyvalid. We plan to use priming to evaluate grammar formalisms and their proposed cat-egories. As pointed out by Reitter (et al.), this is a novel approach to inform linguistictheory (about linguistic competence) by performance data.ReferencesBock, K. and Z. M. Griffin (2000). The persistence of structural priming: Transientactivation or implicit learning? Journal of Experimental Psychology, 129(2):177–192.Bod, R. (1998). Beyond Grammar: An Experience-Based Theory of Language. CSLILecture Notes. CSLI Publications, Stanford. ISBN 157586150X.Pickering, M. J. and S. Garrod (2004). Toward a mechanistic psychology of dialogue.Behavioral and Brain Sciences, 27(02):169–190.Reitter, D. (2008). Context Effects in Language Production: Models of Syntactic Prim-ing in Dialogue Corpora. Ph.D. thesis, University of Edinburgh.Reitter, D., J. Hockenmaier, and F. Keller (2006a). Priming effects in CombinatoryCategorial Grammar. In Proceedings of the 2006 Conference on Empirical Methodsin Natural Language Processing (EMNLP), pp. 308–316. Sydney, Australia.Reitter, D., J. D. Moore, and F. Keller (2006b). Priming of syntactic rules in task-oriented dialogue and spontaneous conversation. In Proceedings of the 28th An-nual Conference of the Cognitive Science Society (CogSci), pp. 685–690. Vancouver,Canada.Tulving, E. and D. L. Schacter (1990). Priming and human memory systems. Science,247(4940):301–306.4

(2006a). Priming effects in Combinatory Categorial Grammar.

(2006b). Priming of syntactic rules in taskoriented dialogue and spontaneous conversation.

Beyond Grammar: An Experience-Based Theory of Language.

Context Effects in Language Production: Models of SyntacticPriming in Dialogue Corpora.

Priming and human memory systems.

The persistence of structural priming: Transient activation or implicit learning?

Toward a mechanistic psychology of dialogue.

Measuring Syntactic Priming in Dialog Corpora

http://pub.uni-bielefeld.de/publication/1785187

Measuring Syntactic Priming in Dialog Corpora

Abstract

Similar works

Full text

Available Versions

Publications at Bielefeld University