We investigated the extent to which the literal meanings of the words forming literally-plausible idioms (e.g., break the ice) are retrieved from memory and semantically composed during sentence comprehension. Idiom strings were embedded in highly predictable, literal and idiomatic contexts. EEG data showed that the integration of the idiom\u2019s conventional meaning occurred at the end of the expression and affected the Post-N400 Positivity. Time-Frequency Representation of the EEG clarified that prior to integrating the idiom\u2019s meaning, an increase in power of higher gamma frequency band was associated only with literal processing. We argue that idioms comprehension proceeds in discrete steps: idiom recognition hampers semantic composition processes so that readers wait until the end of the expression to integrate the idiom meaning with the representation of the sentence

Cacciari, Cristina

Canal, Paolo

Molinaro, Nicola

Pesciarelli, Francesca

Vespignani, Francesco

We investigated the extent to which the literal meanings of the words forming literally-plausible idioms (e.g., break the ice) are retrieved from memory and semantically composed during sentence comprehension. Idiom strings were embedded in highly predictable, literal and idiomatic contexts. EEG data showed that the integration of the idiom’s conventional meaning occurred at the end of the expression and affected the Post-N400 Positivity. Time-Frequency Representation of the EEG clarified that prior to integrating the idiom’s meaning, an increase in power of higher gamma frequency band was associated only with literal processing. We argue that idioms comprehension proceeds in discrete steps: idiom recognition hampers semantic composition processes so that readers wait until the end of the expression to integrate the idiom meaning with the representation of the sentence

PESCIARELLI, Francesca

CACCIARI, Cristina

Archivio istituzionale della ricerca - Università di Modena e Reggio Emilia

Copyright © by the paper’s authors. Copying permitted for private and academic purposes. In Vito Pirrelli, Claudia Marzi, Marcello Ferro (eds.): Word Structure and Word Usage. Proceedings of the NetWordS Final Conference, Pisa, March 30-April 1, 2015, published at http://ceur-ws.org    Electrophysiological correlates of idiom comprehension: semantic composition does not follow lexical retrieval Paolo Canala,b, Francesca Pesciarellia, Francesco Vespignanic, Nicola Molinarod,e & Cristina Cacciaria  a Department of Biomedical Sciences, Università degli Studi di Modena e Reggio Emilia, Italy  b NEtS Center for Neurocognition Epistemology and Theorethical Syntax, IUSS, Pavia, Italy e Ikerbasque, Basque Foundation for Science, Bilbao, 48001, Spain  c Department of Cognition and Formation Sciences, Università degli Studi di Trento, Italy  d BCBL, Basque center on Cognition, Brain and Language, Donostia/San Sebastian, Spain  1 Introduction Idiomatic expressions, such as break the ice, are pervasive in everyday communication. They are frequently co-occurring sequences of words with a conventional meaning that is not derived from word-by-word semantic composition, but rather can be retrieved as such from semantic memory. Idioms are often read faster compared to literal sentences [e.g., Siyanova-Chanturia et al., 2011] and also lexical decision times are faster on id-iom related words than on literal related targets [e.g., Cacciari & Tabossi, 1988]. Recent EEG data further suggest that semantic composition processes of idiomatic constituents might be not fully engaged during comprehension [Rommers et al, 2013]. Finally brain-imaging studies re-ported stronger and more widespread activation of the language network when reading idioms compared to non-idiomatic sentences [Zempleni et al., 2007; Lauro et al., 2008; Boulenger et al., 2009], suggesting that idiom comprehension might involve more cognitive resources. From these fragmented results, it is not clear yet how idiomatic semantic processing differs from literal semantic processing and this might be due to the paradoxical nature of idioms [e.g., Libben & Ti-tone, 2008], which seem to be at the same time amenable of direct memory retrieval and word-by-word compositional analysis. The two main questions of the present re-search thus concern two aspects of idiom comprehension: one relates to how the meaning of the whole is retrieved and integrated in the sentence representation; the second relates to what happens to word-by-word semantic composition of the literal meanings of the expression: is it carried out or suspended? To answer these questions we used EEG measures (with the analysis of Event-Related Potentials and oscillatory dynamics of Time-Frequency representations) because of their temporal precision [e.g., Luck, 2014], and because of the possibility of disentangling between memory gling between memory retrieval and semantic integration processes [e.g., Hoecks & Brower, 2014].  2 The present Study We carried out two Experiments in which short and literally plausible idioms (e.g., break the ice), i.e. having a literal well-formed meaning and a conventional meaning, were embedded in literal or idiomatic contexts. Notably, materials were designed in such way that the sentential context would constrain expectations on the up-coming target words to a similar extent across conditions. By doing so we minimized the im-pact of differential sentence constraints, known to elicit N400 effects, and we carried out a com-parison between sentences that were semanti-cally well-formed and for which contextual ex-pectations on upcoming words were always ful-filled. Experiment 1 used EEG measures as de-pendant variable to investigate the time course of idioms comprehension and was followed up by Experiment 2 in which a cross modal priming paradigm was implemented, in order to confirm the activation of the literal meaning of the idio-matic constituents in both types of contexts.  On the basis of the previous ERP literature we hypothesized that meaning retrieval processes would affect the N400 component [e.g., Feder-meier, 2007]: more demanding retrieval proc-esses should be associated to larger N400 effects. The debate about the role of the N400 in seman-tic integration vs. retrieval mechanisms [see se-mantic unification processes in Hagoort & Van Berkum, 2007] makes it hard to exclude that the N400 component is not associated with the se-mantic integration of the meaning of the whole; however, given the available evidence on figura-tive language processing, we could also expect an effect on later occurring positivities, previ-ously associated with metaphor (Late Positive Complex, LPC) [e.g., Coulson & Van Petten, 2002; Lai et al., 2009] or irony (P600) [Regel et 98al., 2010] processing, or semantic pragmatic reanalysis (frontal Post-N400 Positivity) [e.g., Van Berkum et al., 2009; Molinaro et al., 2012]. Another result that has been previously reported in the ERP literature of idioms processing is the finding of an involvement of the P300 compo-nent. The P300 is generally associated with cog-nitive mechanisms of context update [Donchin & Coles, 1988] or context closure [Verleger, 1988]: Vespignani et al. (2010) found that the brain’s electrical response to the correct idiom constitu-ent was different if recorded before or after the idiom recognition point (RP, e.g., prendere il toro per leRP … corna -- take the bull by theRP … horns). The match to the correct idiom word was associated with an N400 reduction before recog-nition, but the electrophysiological response led to a P300 effect after the recognition of the id-iom. Such effect would mirror a qualitative change in readers’ expectations about upcoming words, after the expression has been recognized. We also expected to replicate Rommers et al (2013) results in the time-frequency domain of the EEG. The authors observed a power increase in the upper gamma frequency band after the presentation of the expected target words in lit-eral but not in idiomatic contexts, supporting the hypothesis that semantic unification mechanisms are less engaged in idioms comprehension. 3 Method 3.1 Participants 380 students at Università degli studi di Modena e Reggio Emilia participated to the study set up to norm the experimental materials. 32 different students took part in Experiment 1. 42 students volunteered in experiment 2. 3.2 Materials Experiment 1 materials were 90 idiomatic ex-pressions of similar structure (VP+NP idioms) embedded in sentences. Idioms were selected for being highly Familiar and correctly paraphrased. Three sentential contexts for each expression were created so that the last word of the expres-sion was highly predictable in the three contexts (above 85% cloze probability). ERPs were time-locked to the presentation of the first word of the expression (W1), and epochs comprising W1, W2 and W3 were extracted from the EEG. In Experiment 2 a subset of 44 idioms was used.  1a) La maestra aveva notato che Nicola dis-turbava i compagni, ma la prima volta chiuse un occhio e continuò la lezione.  (The teacher saw Nick was bothering his desk mate but for the first time she closed an eye (turned a blind eye) and kept on teaching.)  1b) Alla visita oculistica Enrico, prima di leg-gere le lettere indicate sulla lavagna lumi-nosa, chiuse un occhio per valutare la mio-pia.  (At the Ophthalmological visit, before starting to read the letters on the panel aloud Henry closed an eye in order to evaluate his nearsightedness.)  1c) Giovanni ha rotto gli occhiali durante la rissa perché ha preso un pugno in un occhio e gli sono caduti a terra. (Jack broke his glasses during the fight because got a punch in his eye and fell on the ground.)  3.3 Procedure In Experiment 1 sentences were presented word-by-word at the centre of the screen (SOA=600ms). In Experiment 2, contexts sen-tences were auditorily presented via headphones until the last word of the expression. Targets that could be related or unrelated to the literal mean-ing of the last word of the expression were visu-ally presented at the offset of the audio file. 4 Results Fig.1 Grand Average ERPs from a pool of 7 fron-tal electrodes (AF3, AF4, F3, FZ, F4, FC1, FC2) in which frontal PNP effects are usually reported (negative voltage is plotted upwards). Idiomatic condition (solid line), Literal condition (dashed line) and Control condition (dotted line) are com-pared at the onset of the last word of the idiomatic 99expression (e.g., ice).  Experiment 1 showed that: - No N400 differences emerged between literal and idiomatic context, during the processing of the three constituent words. - Differences between Idiomatic vs. Literal, and Idiomatic vs. Control conditions emerged during the presentation of the last word of the expression (e.g., ice), and oc-curred in the 400 to 600 ms time interval. - Consistently with Rommers et al (2013) study, the Time-Frequency analysis of the EEG revealed power differences in the higher gamma frequency band (60-80Hz) between expressions embedded in literal vs. idiomatic contexts: no power increase was associated with the idiomatic condi-tion.   Experiment 2 showed that: - Target words related to the literal meaning of the idiomatic constituents obtained faster lexical decision times with respect to unrelated targets, regardless of type of context. 5 Discussion Concerning the question related to how the meaning of the whole idiom is integrated in the sentence representation, our results suggest that integration mechanisms occur only upon presen-tation of the last constituent word, when the idiomatic expression has very likely been recog-nized. On the last constituent, ERP differences between idiomatic and literal contexts emerged between 400 and 600 ms in frontal electrodes. The timing and scalp distribution of the effect suggest that it affected a positive component (the frontal Post-N400 Positivity) occurring soon af-ter the peak of the N400. These results could be accommodated elaborating the framework pro-posed by the Retrieval-Integration hypothesis [Hoecks & Brower, 2014], which holds that se-mantic - pragmatic integration processes are re-flected in P600 like positivities. One possible interpretation is that the observed frontal positive shift might be part of a larger family of positive components reflecting the engagement of a se-mantic/pragmatic wrap-up mechanism that is performed at end of the expression to assign a full interpretation to the incoming input.  Concerning the second experimental question related to the composition of individual constitu-ent words we argue that Experiment 2 showed that the literal meaning of the last word of the expression was at least accessed, and confirms other evidence supporting the idea that readers process the literal meaning of idiomatic constitu-ents (Boulenger, Shtyrov & Pulvermüller, 2012). Moreover, the lack of N400 differences across conditions and word positions, suggests that lexi-cal retrieval processes similarly occurred in literal and idiomatic contexts. However, the analysis of the frequency domain replicated Rommers et al’s findings of a larger power in-crease in the high gamma frequency band for literal compared to idiomatic contexts, which, consistently with their interpretation, could sig-nal that word-by-word composition mechanisms are less engaged in idioms comprehension.  Conclusions When presented with idiomatic expressions readers retrieve the literal meaning of the con-stituent words. However, word-by-word seman-tic composition mechanisms are idling, and, only at the end of the expression, a seman-tic/pragmatic wrap-up of the idiom is carried out to update the sentence representation. 100Reference Siyanova-Chanturia, A., Conklin, K., & Schmitt, N. (2011). Adding more fuel to the fire: An eye-tracking study of idiom processing by na-tive and non-native speakers. Second Lan-guage Research, 27(2), 251–272.  Cacciari, C., & Tabossi, P. (1988). The compre-hension of idioms. Journal of Memory and Language, 27(6), 668–683.  Zempleni, M.-Z., Haverkort, M., Renken, R., & A. Stowe, L. (2007). Evidence for bilateral in-volvement in idiom comprehension: An fMRI study. NeuroImage, 34(3), 1280–1291.  Lauro, L. J. R., Tettamanti, M., Cappa, S. F., & Papagno, C. (2008). Idiom Comprehension: A Prefrontal Task? Cerebral Cortex, 18(1), 162–170.  Boulenger, V., Hauk, O., & Pulvermüller, F. (2009). Grasping Ideas with the Motor Sys-tem: Semantic Somatotopy in Idiom Compre-hension. Cerebral Cortex, 19(8), 1905–1914.  Rommers, J., Dijkstra, T., & Bastiaansen, M. (2012). Context-dependent Semantic Process-ing in the Human Brain: Evidence from Idiom Comprehension. Journal of Cognitive Neuro-science, 25(5), 762–776.  Libben, M. R., & Titone, D. A. (2008). The multidetermined nature of idiom processing. Memory & Cognition, 36(6),1103–1121.  Luck, S. J. (2014). An Introduction to the Event-Related Potential Technique. MIT Press. Hoeks, J. C. J. and Brouwer, H. (2014). Electrophysiological Research on Conversation and Discourse Processing. In: Holtgraves, T. (Ed.), Oxford Handbook of Language and Social Psychology, pp. 365-386. New York: Oxford University Press. Hagoort, P., & Berkum, J. van. (2007). Beyond the sentence given. Philosophical Transac-tions of the Royal Society B: Biological Sci-ences, 362(1481), 801–811.  Federmeier, K. D. (2007). Thinking ahead: The role and roots of prediction in language com-prehension. Psychophysiology, 44(4), 491–505. Coulson, S., & Petten, C. V. (2002). Conceptual integration and metaphor: An event-related potential study. Memory & Cognition, 30(6), 958–968.  Lai, V. T., Curran, T., & Menn, L. (2009). Com-prehending conventional and novel metaphors: An ERP study. Brain Research, 1284, 145–155.  Regel, S., Gunter, T. C., & Friederici, A. D. (2010). Isn’t It Ironic? An Electrophysiologi-cal Exploration of Figurative Language Proc-essing. Journal of Cognitive Neuroscience, 23(2), 277–293.  Berkum, J. J. A. V., Holleman, B., Nieuwland, M., Otten, M., & Murre, J. (2009). Right or Wrong? The Brain’s Fast Response to Morally Objectionable Statements. Psychological Sci-ence, 20(9), 1092–1099.  Molinaro, N., Carreiras, M., & Duñabeitia, J. A. (2012). Semantic combinatorial processing of non-anomalous expressions. NeuroImage, 59(4), 3488–3501.  Donchin, E., & Coles, M. G. H. (1988). Is the P300 component a manifestation of context updating? Behavioral and Brain Sciences, 11(03), 357–374. Verleger, R. (1988).Event-related potentials and cognition: A critique of the context-updating hypothesis and an alternative interpretation of the P300.Behavioral and Brain Sciences,11, 343–427. Vespignani, F., Canal, P., Molinaro, N., Fonda, S., & Cacciari, C. (2009). Predictive Mecha-nisms in Idiom Comprehension. Journal of Cognitive Neuroscience, 22(8), 1682–1700.  Boulenger, V., Shtyrov, Y., & Pulvermüller, F. (2012). When do you grasp the idea? MEG evidence for instantaneous idiom understand-ing. NeuroImage, 59(4), 3502–3513.   101

Electrophysiological correlates of idiom comprehension: semantic composition does not follow lexical retrieval

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Modena e Reggio Emilia

Archivio istituzionale della ricerca - Università di Padova