The overt structure of wh-questions varies across and within languages. How does a child learn the number of wh-question types that are present in her language and the surface properties of each type? We propose a non-parametric Bayesian model of this aspect of language acquisition, focusing on discrete morphosyntactic properties of questions such as displacement and continuous prosodic properties such as wh-word duration, and apply it to data based on child-directed speech in English and French. The model successfully infers that English has fewer wh-question types than French, identifies the properties of the main question types in each language, and achieves reasonable classification accuracy on naturalistic test utterances. Non-parametric Bayesian inference is a promising method for addressing cross-linguistic and language-internal syntactic variation

Nguyen, An

Wilson, Colin

English

ScholarWorks@UMass Amherst

Proceedings of the Society for Computation in Linguistics Volume 4 Article 47 2021 Learning the surface structure of wh-questions in English and French with a non-parametric Bayesian model An Nguyen Johns Hopkins University, anguye60@jhu.edu Colin Wilson colin.wilson@jhu.edu Follow this and additional works at: https://scholarworks.umass.edu/scil  Part of the Computational Linguistics Commons Recommended Citation Nguyen, An and Wilson, Colin (2021) "Learning the surface structure of wh-questions in English and French with a non-parametric Bayesian model," Proceedings of the Society for Computation in Linguistics: Vol. 4 , Article 47. DOI: https://doi.org/10.7275/kp97-zx82 Available at: https://scholarworks.umass.edu/scil/vol4/iss1/47 This Abstract is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Proceedings of the Society for Computation in Linguistics by an authorized editor of ScholarWorks@UMass Amherst. For more information, please contact scholarworks@library.umass.edu. Learning the surface structure of wh-questions in Englishand French with a non-parametric Bayesian modelAn Nguyen, Colin WilsonJohns Hopkins University, Department of Cognitive Sciencean.nguyen@jhu.edu, colin.wilson@jhu.eduIntroduction. There is considerable variation inthe surface structure of wh-questions (e.g. Soare,2007). Across languages, questions can differ withrespect to overt displacement of wh-phrases (i.e.,ex-situ vs. in-situ questions), inversion of subjectsand auxiliaries or verbs, presence of a Q-marker,and prosodic properties such as pitch accent. Thesedifferences present a problem for language acqui-sition: How does a child learn the grammaticalsurface structure of wh-questions in her native lan-guage(s), given that within a language there can betypically multiple wh-question types, each with itsown characteristic surface properties?In this paper, we propose a model for learningthe number of wh-question types in a given lan-guage and the properties of each type. The ideapursued here is that a child learns the number ofwh-question types in her language and their sur-face structures by clustering observed utterances.Because the child cannot know a priori how manytypes are present, our model specifically employsnon-parametric clustering. This allows the numberof types to be learned from the input data ratherthan specified beforehand. We focus here on overtproperties such as displacement, morphologicalmarking, and prosody.The learning problem. In English, thereare at least three types of wh-question avail-able to children in child-directed speech (CDS),namely fronted information-seeking questions, in-situ probe questions, and in-situ echo questions(Nguyen and Legendre, 2020). Fronted questionsand probe questions are both information-seekingand have similar prosody: the wh-word typicallyhas flat/falling fundamental frequency (F0) and re-ceives no stress accent (Reinhardt, 2019). Echoquestions, on the other hand, are typically usedto ask for repetition of a previous utterance ratherthan to request new information. Echo questionsprosody is distinctive: the wh-word typically hashigh rising pitch and receives heavy stress (cuedin part by longer duration) (Artstein, 2012; Chengand Rooryck, 2002).Children learning English must determine thenumber of wh-question variants present in the lan-guage and the characteristics of each type. Theymust learn that while fronted questions and probequestions have different morphosyntactic proper-ties, they are both information-seeking and can beused in similar contexts. They must also learn thatdepending on the prosody, an in-situ question cancall for a new-information response (probe ques-tions) or a repetition/clarification (echo questions).The model. We adopt a non-parametric ap-proach because, unlike classical parametric finitemixture models, it does not force the learner tocommit to the existence of a particular number ofclusters (question types) in advance of analyzingthe input data. For purposes of implementation, weplace an upper bound of K = 10 on the numberof wh-question types that the model can learn, andexamine only overt morphosyntactic and prosodicproperties. The morphosyntactic properties are dis-crete variables that can take on two values (1 =presence and 0 = absence). They include the po-sition of the wh-word, the inversion status of theauxiliary, and the presence of a Q-marker. Theprosodic properties consist of two continuous vari-ables: the duration and F0 contour of the wh-word.The non-parametric model proposed here is techni-cally a truncated Dirichlet Process Mixture Model(e.g. Gershman and Blei, 2012), as specified below.The probability that the ith question utterance,represented as three binary morphopsyntactic vari-ables and two continuous prosodic variables, be-longs to question type k is given by Bayes’ Rule:p(k|yi) =p(yi|k) wk∑Kk=1 p(yi|k′) wk′403Proceedings of the Society for Computation in Linguistics (SCiL) 2021, pages 403-405.Held on-line February 14-19, 2021Cluster probabilitiesα ∼ Gamma(1, 1)v` | α ∼ Beta(1, α) for ` = 1, . . . ,K − 1α ∼ Gamma(1, 1)w1 = v1wk = vkk−1∏`=21− v` for k = 2, . . . ,K − 1wK =K−1∏`=11− v`Parameters of each clusterpkj ∼ Beta(1, 1) for j = 1, . . . , Jµk` ∼ Normal(M`, S`) for ` = 1, . . . , Llog σk` ∼ Normal(4, 2)Distribution of observationsp(yi|w,p, µ, σ) for n = 1, . . . , N=K∑k=1wkJ∏j=1Bernoulli(yij |pkj)·L∏`=1Normal(yi`|µk`, σk`)where each p(yi|k) is a product of threeBernoulliprobabilities and two Normal densities. The mix-ture weights wK are given by a stick-breaking pro-cess (Sethuraman, 1994).Data and results. The model was trained on2000 simulated instances of the 3 questions types.88 English wh-questions were collected from fourCHILDES audio corpora, HSLLD (Dickinson andTabors, 2001), Snow (MacWhinney and Snow,1990), Van Houten (Van Houten, 1986), and Weist(Weist and Zevenbergen, 2008), for testing. Thefrequency distributions of question types in thesimulated data matched distribution in CDS. In-ference proceeded by MCMC sampling for 5000iterations with the initial 2500 samples discardedas burn-in. Trace plots indicated that all parameterssettled on stable values within the burn-in period,therefore without loss of detail we present only av-erage values over the remaining 2500 samples. Thesampling run shown in Table 1b (see appendix A)converged on three clusters, ordered in descend-ing probability, that closely approximate the actualwh-question types in the training data. (The otherclusters inferred by the model had a total proba-bility of 0.05 and are ignored here as noise.) Themodel accurately classified 97.7% of the simulatedquestion utterances on which it was trained, and86.0% of the natural CDS test utterances. The mainconfusion for test utterances was misclassificationof echo questions as probe questions (i.e., as thesomewhat more frequent type of in-situ question).Discussion. We have described a non-parametric Bayesian model that infers the num-ber of wh-variants within a language and classifiesquestion utterances accordingly. The model doesfairly well in terms of identifying the number ofclusters. The current model addresses the acquisi-tion problem posed by differences in the surfaceproperties of wh-questions across and within lan-guages. While much work remains to be done toexpand the range of variation that the model canaccommodate, to integrate the model with otherapproaches to the acquisition of wh-questions (e.g.,Pearl and Sprouse, 2013) and other aspects of syn-tax, and to synergistically combine the learningof surface properties with that of semantics andpragmatics, the present results show the promiseof applying non-parametric Bayesian methods tocross-linguistic and especially language-internalsyntactic variation.ReferencesRon Artstein. 2012. A focus semantics for echo ques-tions. In Ágnes Bende-Farkas and Arndt Riester, ed-itors, Workshop on Information Structure in Context,pages 98–107. IMS, University of Stuttgart.Lisa Cheng and Johan Rooryck. 2002. Licensing wh-insitu. Syntax, 3:1–19.David Dickinson and Patton Tabors. 2001. Beginningliteracy with language: Young children learning athome and school. Paul Brookes Publishing, Balti-more, MD.Samuel J Gershman and David M Blei. 2012. A tuto-rial on Bayesian nonparametric models. Journal ofMathematical Psychology, 56(1):1–12.Brian MacWhinney and Catherine Snow. 1990. Thechild language data exchange system: An update.Journal of Child Language, 17:457–472.An Nguyen and Géraldine Legendre. 2020. Covertmovement in English probing wh-questions. Pro-ceedings of Linguistic Society of America 2020 An-nual Meeting, 5(1):180–186.Lisa Pearl and Jon Sprouse. 2013. Syntactic islandsand learning biases: Combining experimental syntax404and computational modeling to investigate the lan-guage acquisition problem. Language Acquisition,20(1):23–68.Janina Reinhardt. 2019. Regularity and variation inFrench direct interrogatives: The morpho-syntaxand intonation of question forms in reality TV shows,audio books and teaching materials. Ph.D. thesis,University of Konstanz.Jayaram Sethuraman. 1994. A constructive definitionof Dirichlet priors. Statistica Sinica, 4:639–650.Gabriela Soare. 2007. A cross-linguistic typology ofquestion formation and the antisymmetry hypothesis.Generative Grammar in Geneva, 5:107–131.Lori Van Houten. 1986. Role of maternal input in theacquisition process: The communicative strategiesof adolescent and older mothers with their languagelearning children. Paper presented at the 11th An-nual Boston University Conference on Language De-velopment.Richard Weist and Andrea Zevenbergen. 2008. Auto-biographical memory and past time reference. Lan-guage Learning and Development, 4(4):291–308.A AppendicesTable 1. English question types based on CDS (a) and inferred by the non-parametric Bayesian model (b)(a) Type Proportion WhFront Inver InitialQ WhDur (sd) Wh∆F0 (sd)Fronted .84 1 1 0 150 (49) -6 (36)Probe .09 0 0 0 208 (85) -21 (61)Echo .07 0 0 0 254 (60) 108 (64)(b) Cluster w pWhFront pInver pInitialQ µWhDur (σ) µWh∆F0 (σ)1 ≈ Fronted 0.79 1.0 1.0 0 119 (48) 4 (35)2 ≈ Probe 0.11 0 0 0 203 (78) -8 (68)3 ≈ Echo 0.05 0.01 0.01 0.01 270 (49) 120 (58)405

Learning the surface structure of wh-questions in English and French with a non-parametric Bayesian model

https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1214&amp;context=scil

Learning the surface structure of wh-questions in English and French with a non-parametric Bayesian model

Abstract

Similar works

Full text

Available Versions

ScholarWorks@UMass Amherst