Verbing and nouning in French : toward an ecologically valid approach to sentence processing

La présente thèse utilise la technique des potentiels évoqués afin d’étudier les méchanismes neurocognitifs qui sous-tendent la compréhension de la phrase. Plus particulièrement, cette recherche vise à clarifier l’interaction entre les processus syntaxiques et sémantiques chez les locuteurs natifs et les apprenants d’une deuxième langue (L2). Le modèle “syntaxe en premier” (Friederici, 2002, 2011) prédit que les catégories syntaxiques sont analysées de façon précoce: ce stade est reflété par la composante ELAN (Early anterior negativity, Négativité antérieure gauche), qui est induite par les erreurs de catégorie syntaxique. De plus, ces erreurs semblent empêcher l’apparition de la composante N400 qui reflète les processus lexico-sémantiques. Ce phénomène est défini comme le bloquage sémantique (Friederici et al., 1999). Cependant, la plupart des études qui observent la ELAN utilisent des protocoles expérimentaux problématiques dans lesquels les différences entre les contextes qui précèdent la cible pourraient être à l’origine de résultats fallacieux expliquant à la fois l’apparente “ELAN” et l’absence de N400 (Steinhauer & Drury, 2012). La première étude rééevalue l’approche de la “syntaxe en premier” en adoptant un paradigme expériemental novateur en français qui introduit des erreurs de catégorie syntaxique et les anomalies de sémantique lexicale. Ce dessin expérimental équilibré contrôle à la fois le mot-cible (nom vs. verbe) et le contexte qui le précède. Les résultats récoltés auprès de locuteurs natifs du français québécois ont révélé un complexe N400-P600 en réponse à toutes les anomalies, en contradiction avec les prédictions du modèle de Friederici. Les effets additifs des manipulations syntaxique et sémantique sur la N400 suggèrent la détection d’une incohérence entre la racine du mot qui avait été prédite et la cible, d’une part, et l’activation lexico-sémantique, d’autre part. Les réponses individuelles se sont pas caractérisées par une dominance vers la N400 ou la P600: au contraire, une onde biphasique est présente chez la majorité des participants. Cette activation peut donc être considérée comme un index fiable des mécanismes qui sous-tendent le traitement des structures syntagmatiques. La deuxième étude se concentre sur les même processus chez les apprenants tardifs du français L2. L’hypothèse de la convergence (Green, 2003 ; Steinhauer, 2014) prédit que les apprenants d’une L2, s’ils atteignent un niveau avancé, mettent en place des processus de traitement en ligne similaires aux locuteurs natifs. Cependant, il est difficile de considérer en même temps un grand nombre de facteurs qui se rapportent à leurs compétences linguistiques, à l’exposition à la L2 et à l’âge d’acquisition. Cette étude continue d’explorer les différences inter-individuelles en modélisant les données de potentiels-évoqués avec les Forêts aléatoires, qui ont révélé que le pourcentage d’explosition au français ansi que le niveau de langue sont les prédicteurs les plus fiables pour expliquer les réponses électrophysiologiques des participants. Plus ceux-ci sont élevés, plus l’amplitude des composantes N400 et P600 augmente, ce qui confirme en partie les prédictions faites par l’hypothèse de la convergence. En conclusion, le modèle de la “syntaxe en premier” n’est pas viable et doit être remplacé. Nous suggérons un nouveau paradigme basé sur une approche prédictive, où les informations sémantiques et syntaxiques sont activées en parallèle dans un premier temps, puis intégrées via un recrutement de mécanismes contrôlés. Ces derniers sont modérés par les capacités inter-individuelles reflétées par l’exposition et la performance.The present thesis uses event-related potentials (ERPs) to investigate neurocognitve mechanisms underlying sentence comprehension. In particular, these two experiments seek to clarify the interplay between syntactic and semantic processes in native speakers and second language learners. Friederici’s (2002, 2011) “syntax-first” model predicts that syntactic categories are analyzed at the earliest stages of speech perception reflected by the ELAN (Early left anterior negativity), reported for syntactic category violations. Further, syntactic category violations seem to prevent the appearance of N400s (linked to lexical-semantic processing), a phenomenon known as “semantic blocking” (Friederici et al., 1999). However, a review article by Steinhauer and Drury (2012) argued that most ELAN studies used flawed designs, where pre-target context differences may have caused ELAN-like artifacts as well as the absence of N400s. The first study reevaluates syntax-first approaches to sentence processing by implementing a novel paradigm in French that included correct sentences, pure syntactic category violations, lexical-semantic anomalies, and combined anomalies. This balanced design systematically controlled for target word (noun vs. verb) and the context immediately preceding it. Group results from native speakers of Quebec French revealed an N400-P600 complex in response to all anomalous conditions, providing strong evidence against the syntax-first and semantic blocking hypotheses. Additive effects of syntactic category and lexical-semantic anomalies on the N400 may reflect a mismatch detection between a predicted word-stem and the actual target, in parallel with lexical-semantic retrieval. An interactive rather than additive effect on the P600 reveals that the same neurocognitive resources are recruited for syntactic and semantic integration. Analyses of individual data showed that participants did not rely on one single cognitive mechanism reflected by either the N400 or the P600 effect but on both, suggesting that the biphasic N400-P600 ERP wave can indeed be considered to be an index of phrase-structure violation processing in most individuals. The second study investigates the underlying mechanisms of phrase-structure building in late second language learners of French. The convergence hypothesis (Green, 2003; Steinhauer, 2014) predicts that second language learners can achieve native-like online- processing with sufficient proficiency. However, considering together different factors that relate to proficiency, exposure, and age of acquisition has proven challenging. This study further explores individual data modeling using a Random Forests approach. It revealed that daily usage and proficiency are the most reliable predictors in explaining the ERP responses, with N400 and P600 effects getting larger as these variables increased, partly confirming and extending the convergence hypothesis. This thesis demonstrates that the “syntax-first” model is not viable and should be replaced. A new account is suggested, based on predictive approaches, where semantic and syntactic information are first used in parallel to facilitate retrieval, and then controlled mechanisms are recruited to analyze sentences at the interface of syntax and semantics. Those mechanisms are mediated by inter-individual abilities reflected by language exposure and performance

What can co-speech gestures in aphasia tell us about the relationship between language and gesture?: A single case study of a participant with Conduction Aphasia

Cross-linguistic evidence suggests that language typology influences how people gesture when using ‘manner-of-motion’ verbs (Kita 2000; Kita & Özyürek 2003) and that this is due to ‘online’ lexical and syntactic choices made at the time of speaking (Kita, Özyürek, Allen, Brown, Furman & Ishizuka, 2007). This paper attempts to relate these findings to the co-speech iconic gesture used by an English speaker with conduction aphasia (LT) and five controls describing a Sylvester and Tweety1 cartoon. LT produced co-speech gesture which showed distinct patterns which we relate to different aspects of her language impairment, and the lexical and syntactic choices she made during her narrative

Semantic radical consistency and character transparency effects in Chinese: an ERP study

BACKGROUND: This event-related potential (ERP) study aims to investigate the representation and temporal dynamics of Chinese orthography-to-semantics mappings by simultaneously manipulating character transparency and semantic radical consistency. Character components, referred to as radicals, make up the building blocks used dur...postprin

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

From minimal dependencies to sentence contexts: neural correlates of agreement processing

289 p.Language comprehension is incremental, involving the integration of formal and conceptual information from different words, together with the need to resolve conflicting cues when unexpected information occurs. However, despite the extensive amount of findings regarding how the brain deals with these information, two essential and still open questions are 1whether the neural circuit(s) for coding syntactic and semantic information embedded in our linguistic code are the same or different, and 2whether the possible interaction(s) between these two different types of information leaves a trace in the brain response. The current thesis seeks to segregate the neuro-anatomical substrates of these processes by taking advantage of the Spanish agreement system. This system comprised those procedural mechanisms concerning the regular assignment of the number [singular, plural], person [first, second and third] and/or gender [feminine, masculine] information, associated with different sentence constituents. Experimental manipulations concerning different agreement features and the elements involved in an agreement relation, allowed us to characterize the neural network underlying agreement processing. This thesis comprised five experiments: while experiments I and II explored nominal dependencies in local as well as non-local relations, experiments III, IV and V explored subject-verb relations in a more complex sentence context. To distinguish between purely syntactic mechanisms and those where semantic and syntactic factors would interact during language comprehension, different types of agreement relations and/or agreement features were manipulated in well- and ill-formed constructions. The interaction effect between the different factors included in each experiment was always the critical comparison. In general, our results include firstly a functional dissociation between well-formed and ill-formed constructions: while ill-formed constructions recruited a bilateral distributed fronto-parietal network associated to conflict monitoring operations, not language specific, well-formed constructions recruited a left lateralized fronto-temporo-parietal network that seems to be specifically related to different aspects of phrase and sentence processing. Secondly, there was an anterior to posterior functional gradient associated to the middle and superior temporal cortex that consistently appears across experiments. Specifically, while the posterior portion of the left MTG-STG seems to be related to the storage and retrieval of lexical and morpho-syntactic information, the anterior portion of this region was related to syntactic-combinatorial building mechanisms. Critically, in the most anterior part of the left temporal cortex, corresponding with the middle and superior temporal pole, form-to-meaning mapping processes seems to be represented. Thirdly, the response of the left temporal cortex appears to be controlled by left inferior frontal regions (LIFG). Finally, left parietal regions such us the angular gyrus showed increased activation for those manipulations involving semantic factors (e.g., conceptual gender and Unagreement constructions), highlighting its crucial role in the processing of different types of semantic information (e.g., conceptual integration and semantic-discourse integration). Overall, these findings highlight the sensitivity of the agreement system to syntactic and semantic factors embedded into an agreement relation, opening new windows to the study of agreement computation and language comprehension.bcbl: basque center on cognition, brain and languag

Features and Functions: Decomposing the Neural and Cognitive Bases of Semantic Composition

In this dissertation, I present a suite of studies investigating the neural and cognitive bases of semantic composition. First, I motivate why a theory of semantic combinatorics is a fundamental desideratum of the cognitive neuroscience of language. I then introduce a possible typology of semantic composition: one which involves contrasting feature-based composition with function-based composition. Having outlined several different ways we might operationalize such a distinction, I proceed to detail two studies using univariate and multivariate fMRI measures, each examining different dichotomies along which the feature-vs.-function distinction might cleave. I demonstrate evidence that activity in the angular gyrus indexes certain kinds of function-/relation-based semantic operations and may be involved in processing event semantics. These results provide the first targeted comparison of feature- and function-based semantic composition, particularly in the brain, and delineate what proves to be a productive typology of semantic combinatorial operations. The final study investigates a different question regarding semantic composition: namely, how automatic is the interpretation of plural events, and what information does the processor use when committing to either a distributive plural event (comprising separate events) or a collective plural event (consisting of a single joint event)

From minimal dependencies to sentence contexts: neural correlates of agreement processing

289 p.Language comprehension is incremental, involving the integration of formal and conceptual information from different words, together with the need to resolve conflicting cues when unexpected information occurs. However, despite the extensive amount of findings regarding how the brain deals with these information, two essential and still open questions are 1whether the neural circuit(s) for coding syntactic and semantic information embedded in our linguistic code are the same or different, and 2whether the possible interaction(s) between these two different types of information leaves a trace in the brain response. The current thesis seeks to segregate the neuro-anatomical substrates of these processes by taking advantage of the Spanish agreement system. This system comprised those procedural mechanisms concerning the regular assignment of the number [singular, plural], person [first, second and third] and/or gender [feminine, masculine] information, associated with different sentence constituents. Experimental manipulations concerning different agreement features and the elements involved in an agreement relation, allowed us to characterize the neural network underlying agreement processing. This thesis comprised five experiments: while experiments I and II explored nominal dependencies in local as well as non-local relations, experiments III, IV and V explored subject-verb relations in a more complex sentence context. To distinguish between purely syntactic mechanisms and those where semantic and syntactic factors would interact during language comprehension, different types of agreement relations and/or agreement features were manipulated in well- and ill-formed constructions. The interaction effect between the different factors included in each experiment was always the critical comparison. In general, our results include firstly a functional dissociation between well-formed and ill-formed constructions: while ill-formed constructions recruited a bilateral distributed fronto-parietal network associated to conflict monitoring operations, not language specific, well-formed constructions recruited a left lateralized fronto-temporo-parietal network that seems to be specifically related to different aspects of phrase and sentence processing. Secondly, there was an anterior to posterior functional gradient associated to the middle and superior temporal cortex that consistently appears across experiments. Specifically, while the posterior portion of the left MTG-STG seems to be related to the storage and retrieval of lexical and morpho-syntactic information, the anterior portion of this region was related to syntactic-combinatorial building mechanisms. Critically, in the most anterior part of the left temporal cortex, corresponding with the middle and superior temporal pole, form-to-meaning mapping processes seems to be represented. Thirdly, the response of the left temporal cortex appears to be controlled by left inferior frontal regions (LIFG). Finally, left parietal regions such us the angular gyrus showed increased activation for those manipulations involving semantic factors (e.g., conceptual gender and Unagreement constructions), highlighting its crucial role in the processing of different types of semantic information (e.g., conceptual integration and semantic-discourse integration). Overall, these findings highlight the sensitivity of the agreement system to syntactic and semantic factors embedded into an agreement relation, opening new windows to the study of agreement computation and language comprehension.bcbl: basque center on cognition, brain and languag