A role for the developing lexicon in phonetic category acquisition

Infants segment words from fluent speech during the same period when they are learning phonetic categories, yet accounts of phonetic category acquisition typically ignore information about the words in which sounds appear. We use a Bayesian model to illustrate how feedback from segmented words might constrain phonetic category learning by providing information about which sounds occur together in words. Simulations demonstrate that word-level information can successfully disambiguate overlapping English vowel categories. Learning patterns in the model are shown to parallel human behavior from artificial language learning tasks. These findings point to a central role for the developing lexicon in phonetic category acquisition and provide a framework for incorporating top-down constraints into models of category learning

The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study

Carminati MN, Knoeferle P. The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study. Presented at the Architectures and Mechanisms of Language and Processing (AMLaP), Riva del Garda, Italy

Neuroeconomics: How Neuroscience Can Inform Economics

Neuroeconomics uses knowledge about brain mechanisms to inform economic analysis, and roots economics in biology. It opens up the "black box" of the brain, much as organizational economics adds detail to the theory of the firm. Neuroscientists use many tools— including brain imaging, behavior of patients with localized brain lesions, animal behavior, and recording single neuron activity. The key insight for economics is that the brain is composed of multiple systems which interact. Controlled systems ("executive function") interrupt automatic ones. Emotions and cognition both guide decisions. Just as prices and allocations emerge from the interaction of two processes—supply and demand— individual decisions can be modeled as the result of two (or more) processes interacting. Indeed, "dual-process" models of this sort are better rooted in neuroscientific fact, and more empirically accurate, than single-process models (such as utility-maximization). We discuss how brain evidence complicates standard assumptions about basic preference, to include homeostasis and other kinds of state-dependence. We also discuss applications to intertemporal choice, risk and decision making, and game theory. Intertemporal choice appears to be domain-specific and heavily influenced by emotion. The simplified ß-d of quasi-hyperbolic discounting is supported by activation in distinct regions of limbic and cortical systems. In risky decision, imaging data tentatively support the idea that gains and losses are coded separately, and that ambiguity is distinct from risk, because it activates fear and discomfort regions. (Ironically, lesion patients who do not receive fear signals in prefrontal cortex are "rationally" neutral toward ambiguity.) Game theory studies show the effect of brain regions implicated in "theory of mind", correlates of strategic skill, and effects of hormones and other biological variables. Finally, economics can contribute to neuroscience because simple rational-choice models are useful for understanding highly-evolved behavior like motor actions that earn rewards, and Bayesian integration of sensorimotor information

Typological parameters of genericity

Different languages employ different morphosyntactic devices for expressing genericity. And, of course, they also make use of different morphosyntactic and semantic or pragmatic cues which may contribute to the interpretation of a sentence as generic rather than episodic. [...] We will advance the strong hypo thesis that it is a fundamental property of lexical elements in natural language that they are neutral with respect to different modes of reference or non-reference. That is, we reject the idea that a certain use of a lexical element, e.g. a use which allows reference to particular spatio-temporally bounded objects in the world, should be linguistically prior to all other possible uses, e.g. to generic and non-specific uses. From this it follows that we do not consider generic uses as derived from non-generic uses as it is occasionally assumed in the literature. Rather, we regard these two possibilities of use as equivalent alternative uses of lexical elements. The typological differences to be noted therefore concern the formal and semantic relationship of generic and non-generic uses to each other; they do not pertain to the question of whether lexical elements are predetermined for one of these two uses. Even supposing we found a language where generic uses are always zero-marked and identical to lexical sterns, we would still not assume that lexical elements in this language primarily have a generic use from which the non-generic uses are derived. (Incidentally, none of the languages examined, not even Vietnamese, meets this criterion.

The influence of prior visual gender and action cues versus long-term knowledge in (situated) language processing

Rodríguez A. The influence of prior visual gender and action cues versus long-term knowledge in (situated) language processing. Bielefeld: Universität Bielefeld; 2018.Studies on situated language comprehension (i.e., comprehension in rich visual contexts), have shown that the comprehender makes use of different information sources in order to establish visual reference and to visually anticipate entities in a scene while understanding language (reflecting expectations on what might be mentioned next). Semantics and world-knowledge (i.e., experiential, long-term knowledge) are among these sources. For instance, when listening to a sentence like *The girl will ride...*, the comprehender will likely anticipate an object that a girl could ride, e.g., a carrousel, rather than other objects, such as a motorbike (Kamide, Altmann, & Haywood, 2003). However, following the inspection of events (featuring agents acting upon objects or patients), comprehenders have so far shown a preference to visually anticipate the agents or objects that have been seen as part of those prior events (i.e., *recent-event preference* or the preference for event-based representations; Abashidze, Carminati, & Knoeferle, 2014; Knoeferle, Carminati, Abashidze, & Essig, 2011). This preference emerged even when other plausible objects or better stereotypically fitting agents were present. Although the preference for event-based information over other sources (e.g., plausibility or stereotypicality) seems to be strong and has been accommodated in accounts of situated language comprehension (Knoeferle & Crocker, 2006, 2007), its nature when comprehenders generate expectations is still unspecified. Crucially, the preference for recent events has not been generalized from action events to other types of information in the visual and linguistic contexts. To further examine this issue, this thesis investigated the role of a particular type of information during situated language comprehension under the influence of prior events, namely, visual gender and action cues and knowledge about gender stereotypes. As many studies in the field of psycholinguistics have highlighted, gender (both a biological and a social feature of human beings) is relevant in language comprehension (e.g., grammatical gender can serve to track reference in discourse, and gender-stereotype knowledge can bias our interpretation of a sentence). However, little psycholinguistic research has examined the comprehension of gender information in a visual context. We argue that gender is worth exploring in a paradigm where prior event representations can be pitted against long-term knowledge. Not only that, inspired by experiments using mismatch designs, we wanted to see how the visual attention of the comprehender might be affected as a function of referential incongruencies (i.e., mismatches between visual events and linguistic information, e.g., Knoeferle, Urbach, & Kutas, 2014; Vissers, Kolk, Van de Meerendonk, & Chwilla, 2008; Wassenaar & Hagoort, 2007) and incongruences at the level of worldknowledge (i.e., gender stereotypes; e.g., Duffy & Keir, 2004; Kreiner, Sturt, & Garrod, 2008). By doing so, we could get insights into how both types of sources (event-based information and gender-stereotype knowledge from language) are used, i.e., whether one is more important than the other or if both are equally exploited in situated language comprehension. We conducted three eye-tracking, visual-world experiments and one EEG experiment. In all of these experiments, participants saw events taking place prior to sentence comprehension, i.e., videos of (female or male) hands acting upon objects. In the eye-tracking experiments, following the videos, a visual scene appeared with the faces of two potential agents: one male and one female1. While the agent matching the gender features from prior events (i.e., the hands) was considered as the target agent, the other potential agent, whose gender was not cued in previous events, was the competitor agent. The visual scene in Experiment 3 further included the images of two objects; one was the target object (i.e., the object that appeared in prior events), while the other was a competitor object with opposite stereotypical valence. During the presentation of this scene, an OVS sentence was presented (e.g., translation from German: ‘The cakeNP1/obj bakesV soonADV SusannaNP2/subj’). We used the non-canonical OVS word order as opposed to SVO (more commonly used in prior research, e.g., Knoeferle, Carminati, et al., 2011) precisely to examine participants’ expectations towards the agent, who was mentioned at final position. We manipulated two factors. One factor was the match between prior visual events and language: there were action-verb(-phrase) mismatches in Experiments 1 and 3, and mismatches between the gender of the hands and the final subject (i.e., the proper name) in Experiments 2 and 4. The second manipulation, present in Experiments 1 to 3, was the match between the stereotypical valence of the described actions/events in the sentence and the target agent’s gender. In the eye-tracking experiments, we measured participants’ visual attention towards the agents’ faces during sentence comprehension. In the EEG experiment, we measured ERP responses time-locked to the final, proper name region (i.e., Susanna). Participants’ task was to verify via button press whether the sentence matched the events they just saw. In line with prior research, our results support the idea that the preference for eventbased representations generalizes to another cue, i.e., gender features from the hands of an agent during prior events. Participants generally preferred to look at the target agent compared to the competitor. These results also suggest that the recent-event preference does not just rely on representations of full objects, agents and events, but also subtler (gender) features that serve to identify feature-matching targets during comprehension (i.e., faces of agents are inspected based on the gender features from hands seen in prior events). This preference is however modulated by mismatches in language, i.e., whenever the actions described or the gender implied by the final noun in the sentence were at odds with prior events, attention towards the target agent was reduced. In addition, the scene configuration of Experiment 3 gave rise to gender stereotypicality effects, which had not yet been found in prior studies using a similar design. Participants looked at the target agent (vs. the competitor) to a greater extent when the action described by the sentence stereotypically matched (vs. mismatched) them. As for the electrophysiological response towards mismatches between event-based gender cues and language, we found a biphasic ERP response, which suggests that this type of verification requires two semantically-induced stages of processing. This response had commonalities both with some effects found in strictly linguistic/discourse contexts but also with previously observed mismatch effects in picture-sentence verification studies (i.e., role relation and action mismatches; Knoeferle et al., 2014), which suggests that a similar (perhaps a single) processing mechanism might be involved in several visuolinguistic relations. In sum, our results using gender and action cues from prior events and long-term knowledge call for a more refined consideration of the different aspects involved in (situated) language comprehension. On the one hand, existing accounts need to accommodate further reconciliations/verifications of visuolinguistic relations (e.g., roles, actions, gender features, etc.). When it comes to listeners generating expectations during comprehension while inspecting the visual world, we further suggest that a weighted system (i.e., a system indexing the strength of the expectation and how different information sources contribute to it; also suggested in Münster, 2016), applies for gender of information. Not only event-based representations, but also different discrepancies between these representations and language and, depending on the concurrent visual scene configuration, long-term knowledge (e.g., pertaining to gender stereotypes), can affect weighted expectations. Biosocial aspects such as gender may be of particular interest to answer some of the open questions in how situated language comprehension works, as these aspects can be found and manipulated at different levels of communication (e.g., the comprehender, the speaker, the linguistic content, etc.)

Automatic reconstruction of itineraries from descriptive texts

Esta tesis se inscribe dentro del marco del proyecto PERDIDO donde los objetivos son la extracción y reconstrucción de itinerarios a partir de documentos textuales. Este trabajo se ha realizado en colaboración entre el laboratorio LIUPPA de l' Université de Pau et des Pays de l' Adour (France), el grupo de Sistemas de Información Avanzados (IAAA) de la Universidad de Zaragoza y el laboratorio COGIT de l' IGN (France). El objetivo de esta tesis es concebir un sistema automático que permita extraer, a partir de guías de viaje o descripciones de itinerarios, los desplazamientos, además de representarlos sobre un mapa. Se propone una aproximación para la representación automática de itinerarios descritos en lenguaje natural. Nuestra propuesta se divide en dos tareas principales. La primera pretende identificar y extraer de los textos describiendo itinerarios información como entidades espaciales y expresiones de desplazamiento o percepción. El objetivo de la segunda tarea es la reconstrucción del itinerario. Nuestra propuesta combina información local extraída gracias al procesamiento del lenguaje natural con datos extraídos de fuentes geográficas externas (por ejemplo, gazetteers). La etapa de anotación de informaciones espaciales se realiza mediante una aproximación que combina el etiquetado morfo-sintáctico y los patrones léxico-sintácticos (cascada de transductores) con el fin de anotar entidades nombradas espaciales y expresiones de desplazamiento y percepción. Una primera contribución a la primera tarea es la desambiguación de topónimos, que es un problema todavía mal resuelto dentro del reconocimiento de entidades nombradas (Named Entity Recognition - NER) y esencial en la recuperación de información geográfica. Se plantea un algoritmo no supervisado de georreferenciación basado en una técnica de clustering capaz de proponer una solución para desambiguar los topónimos los topónimos encontrados en recursos geográficos externos, y al mismo tiempo, la localización de topónimos no referenciados. Se propone un modelo de grafo genérico para la reconstrucción automática de itinerarios, donde cada nodo representa un lugar y cada arista representa un camino enlazando dos lugares. La originalidad de nuestro modelo es que además de tener en cuenta los elementos habituales (caminos y puntos del recorrido), permite representar otros elementos involucrados en la descripción de un itinerario, como por ejemplo los puntos de referencia visual. Se calcula de un árbol de recubrimiento mínimo a partir de un grafo ponderado para obtener automáticamente un itinerario bajo la forma de un grafo. Cada arista del grafo inicial se pondera mediante un método de análisis multicriterio que combina criterios cualitativos y cuantitativos. El valor de estos criterios se determina a partir de informaciones extraídas del texto e informaciones provenientes de recursos geográficos externos. Por ejemplo, se combinan las informaciones generadas por el procesamiento del lenguaje natural como las relaciones espaciales describiendo una orientación (ej: dirigirse hacia el sur) con las coordenadas geográficas de lugares encontrados dentro de los recursos para determinar el valor del criterio ``relación espacial''. Además, a partir de la definición del concepto de itinerario y de las informaciones utilizadas en la lengua para describir un itinerario, se ha modelado un lenguaje de anotación de información espacial adaptado a la descripción de desplazamientos, apoyándonos en las recomendaciones del consorcio TEI (Text Encoding and Interchange). Finalmente, se ha implementado y evaluado las diferentes etapas de nuestra aproximación sobre un corpus multilingüe de descripciones de senderos y excursiones (francés, español, italiano)

The Impact of Unfamiliar Proper Names on ESL Learners' Listening Comprehension

Vocabulary knowledge is a prerequisite to successful comprehension for native speakers and second language learners alike. Proper names, a peculiar and diverse group of lexical items, have long been the focus of discussion in general linguistics but have received practically no attention in second language vocabulary acquisition research. This study is the first attempt to assess whether proper names impact on second language learners' listening ability. First, I examine the question of how proper names can be adequately defined and discuss their semantic, structural, pragmatic and functional properties. I analyze proper names in light of the prototype theory and argue that personal, deity and pet names constitute the core of the proper name category. Names of places and enterprises occupy an intermediate position while names of events and artefacts are considered the least prototypical, i.e. peripheral members of the category. After identifying essential properties of prototypical proper names, I argue that in a spoken (as opposed to a written) text proper names cannot be considered automatically known items and place high demands on the listeners' cognitive resources. English as a second language (ESL) learners have to bring in a large amount of linguistic and encyclopaedic knowledge in order to cope with proper names in the flow of speech. I propose a 3-level model of such knowledge: recognition -> categorization -> referent properties. I then subject this model to empirical testing. The first experiment shows that among intermediate to advanced ESL learners the proper names recognition rate is around 60 percent. It is harder for ESL listeners to recognize proper names when the percentage of difficult common vocabulary in the text is high. The participants' proficiency level and the structure of a specific text were also found to affect the ability to recognize unfamiliar names. Well over a third of proper names are missed, which suggests that in real life listening, ESL learners mistake unknown common expressions for proper names and vice versa. In the second experiment, the participants' comprehension of a news story is tested under two conditions: Names Known (all proper names are familiar prior to listening) and Names Unknown (all proper names are unfamiliar). Results indicate that the presence of unfamiliar proper names hinders the intermediate to advanced proficiency learners' comprehension of a short news text as measured by immediate free recall and the ability to evaluate proper names related statements. The effect is local; it concerns comprehension of details, particularly those details that are associated with processing the proper names themselves. The Names Unknown group produced fewer details and more incorrect inferences in their recalls, scored significantly lower on the measure of proper names related comprehension, and selfreported a lower amount of comprehension. In contrast, the Names Known group produced more details and fewer incorrect inferences in their recalls, scored much higher on the measure of proper names related comprehension, and self-reported a greater degree of comprehension. The experiment also shows that participants in the Names Unknown treatment were not always able to ascertain from context what the referent of an unfamiliar proper name is, and in cases when they did, they could not extract as much information about the referent as the participants in the Names Known treatment had available. It is evidently unrealistic to expect ESL learners to determine what unfamiliar proper names refer to from context. On average, after 2-3 attempts at listening participants in the Names Unknown group were able to extract just over 40 percent of the information about the referents of unfamiliar proper names. Also participants' difficulty ratings of experimental tasks confirmed that the presence of unfamiliar proper names definitely makes the text seem harder to understand. The last experiment replicated the findings of the previous one on a larger sample. The Names Known group performed significantly better on open-ended questions and true-false-don't know statements. A substantial effect of unfamiliar proper names on the overall comprehension scores was found. Around 17 percent of the variance in the scores was accounted for by familiarity/lack of familiarity with proper names. The findings also provide some evidence in support of the claim that a name form that hints at the cognitive category its referent belongs to is less likely to adversely affect comprehension than a form that does not. Unfamiliar proper names contribute to raising the vocabulary threshold in second language listening, which should be taken into account by teachers, test-developers and other TESOL (teaching English to speakers of other languages) professionals