A speech envelope landmark for syllable encoding in human superior temporal gyrus.

The most salient acoustic features in speech are the modulations in its intensity, captured by the amplitude envelope. Perceptually, the envelope is necessary for speech comprehension. Yet, the neural computations that represent the envelope and their linguistic implications are heavily debated. We used high-density intracranial recordings, while participants listened to speech, to determine how the envelope is represented in human speech cortical areas on the superior temporal gyrus (STG). We found that a well-defined zone in middle STG detects acoustic onset edges (local maxima in the envelope rate of change). Acoustic analyses demonstrated that timing of acoustic onset edges cues syllabic nucleus onsets, while their slope cues syllabic stress. Synthesized amplitude-modulated tone stimuli showed that steeper slopes elicited greater responses, confirming cortical encoding of amplitude change, not absolute amplitude. Overall, STG encoding of the timing and magnitude of acoustic onset edges underlies the perception of speech temporal structure

Low foreign language proficiency reduces optimism about the personal future

Optimistic estimates about the personal future constitute one of the best-described and most-debated decision biases related to emotion. Nevertheless, it has been difficult to isolate manipulations that reduce optimistic estimates. Eliciting estimates in a foreign language is a promising candidate manipulation because foreign language use alters decision biases in scenarios with emotional components. Consequently, we tested whether foreign language use reduces optimistic estimates. In a laboratory experiment, participants (n = 45) estimated their probability of experiencing life events either in their native language or a foreign language, in which they were highly proficient. We found no differences in these estimates or in the updating of these estimates after receiving feedback about the population baseline probability. Importantly, three online experiments with large sample sizes (ns = 706, 530, and 473) showed that using a foreign language with low proficiency reduced comparative optimism. Participants in the online experiments had diverse proficiency levels and were matched on a variety of control metrics. Fine-grained analyses indicated that low proficiency weakens the coupling between probability estimates and rated arousal. Overall, our findings suggest that an important decision bias can be reduced when using a foreign language with low proficiency

The framing effect in a monetary gambling task is robust in minimally verbal language switching contexts

Decision-making biases, in particular the framing effect, can be altered in foreign language settings (foreign language effect) and following switching between languages (the language switching effect on framing). Recently, it has been suggested that the framing effect is only affected by foreign language use if the task is presented in a rich textual form. Here, we assess whether an elaborate verbal task is also a prerequisite for the language switching effect on framing. We employed a financial gambling task that induces a robust framing effect but is less verbal than the classical framing paradigms (e.g., the Asian disease problem). We conducted an online experiment (n = 485), where we orthogonally manipulated language use and language switching between trials. The results showed no effects of foreign language use or language switching throughout the experiment. This online result was confirmed in a laboratory experiment (n = 27). Overall, we find that language switching does not reduce the framing effect in a paradigm with little verbal content and thus that language switching effects seem contingent on the amount of verbal processing required

Second Language Use Facilitates Implicit Emotion Regulation via Content Labeling

Previous studies reported that negative stimuli induced less affect in bilinguals when stimuli were presented in bilinguals’ second, weaker language (L2) than when they were presented in their native language (L1). This effect of L2 use was attributed to increased emotional distance as well as to increased levels of cognitive control during L2 use. Here we investigated how explicit (cognitive reappraisal, i.e., reinterpreting the meaning of the emotional stimulus to alter its emotional impact) and implicit (content labeling, i.e., categorizing the content of the image; and emotion labeling, i.e., naming the emotion induced by the emotional stimulus) emotion regulation strategies are altered in an L2 (English) context in German native speakers with medium to high proficiency in their L2. While previous studies used linguistic stimuli, such as words, to induce affect, here we used images to test whether reduced affect could also be observed for non-linguistic stimuli when presented in an L2 context. We hypothesized that the previously implicated increase in emotional distance and cognitive control in an L2 would result in an L2 advantage in emotion regulation (i.e., leading to less negative emotions compared to an L1 context), by strengthening the effect of linguistic re-evaluation on the evoked emotions. Using a classic emotion regulation paradigm, we examined changes in subjective emotional state ratings during reappraisal, emotion labeling and content labeling in a L1 and L2 context. We found that the strength of evoked affective responses did not depend on the language context in which an image was presented. Crucially, content labeling in L2 was more effective than in L1, whereas emotion labeling did not differ between languages. Overall, evoked responses were regulated most effectively through explicit emotion regulation (reappraisal) in L1 and L2 context. These results demonstrate an L2 advantage effect for emotion regulation through content labeling and suggest that L2 context alters sub- processes implicated in content labeling but not emotion labeling

A Diffusion Model Analysis

Effects of stimulus length on reaction times (RTs) in the lexical decision task are the topic of extensive research. While slower RTs are consistently found for longer pseudo-words, a finding coined the word length effect (WLE), some studies found no effects for words, and yet others reported faster RTs for longer words. Moreover, the WLE depends on the orthographic transparency of a language, with larger effects in more transparent orthographies. Here we investigate processes underlying the WLE in lexical decision in German-English bilinguals using a diffusion model (DM) analysis, which we compared to a linear regression approach. In the DM analysis, RT-accuracy distributions are characterized using parameters that reflect latent sub-processes, in particular evidence accumulation and decision-independent perceptual encoding, instead of typical parameters such as mean RT and accuracy. The regression approach showed a decrease in RTs with length for pseudo-words, but no length effect for words. However, DM analysis revealed that the null effect for words resulted from opposing effects of length on perceptual encoding and rate of evidence accumulation. Perceptual encoding times increased with length for words and pseudo-words, whereas the rate of evidence accumulation increased with length for real words but decreased for pseudo-words. A comparison between DM parameters in German and English suggested that orthographic transparency affects perceptual encoding, whereas effects of length on evidence accumulation are likely to reflect contextual information and the increase in available perceptual evidence with length. These opposing effects may account for the inconsistent findings on WLEs

Kognitive und neuronale Mechanismen bilingualer Entscheidungsprozesse: Von visueller Wortverarbeitung zu Entscheidungen unter Risiko

Bilingual individuals read in their second language (L2) and frequently make decisions based on information perceived in their L2. Compared to monolinguals, visual word recognition in bilinguals bears additional aspects that may also affect later stages of processing. Their integration into neurocognitive models of single word reading and decision making still leaves many open questions. In particular, bilingual word recognition entails the ability to recognize the language membership of a word, and to map letter strings to lexical representations and phonology of the appropriate language, whereby the underlying mapping rules can be different and even contradictory across languages. Furthermore, following these perceptual aspects of word recognition, the perceived information is often used as a basis for higher level, non-perceptual processes, such as decision making under risk. A prominent example for the effects of linguistic framing on decision making is the stronger preference for risky choices when negative outcomes are emphasized than when positive consequences are explicitly stated (the framing effect). The effects of foreign language use on processes influencing the framing effect, most prominently affective processing and cognitive control, has been postulated, but direct evidence for these effects is scarce. This dissertation's research investigates cognitive and neuronal aspects of bilingual visual word recognition and decision making, using the case of bilingual individuals with native language German (L1) and second language English (L2). Study 1 isolated the effects of word and pseudoword length (i.e. number of letters) on the dynamics of sublexical encoding and lexical access in German and English, using a diffusion model of reaction times (RTs) in lexical decisions. Study 2 investigated whether language membership decisions and naming in a language-ambiguous context are influenced by continuous sublexical (bigram frequencies) and lexical (orthographic neighborhood size) similarity of a letter string to German and English words. This was extended to an investigation of the neuronal correlates of language similarity statistics and language membership representations in Study 3, using functional magnetic resonance imaging (fMRI). Study 4 investigated whether a reduced framing effect during foreign language use is due to reduced affect or increased cognitive control. Study 1 showed that sublexical encoding of words and pseudowords during lexical decision takes longer for longer stimuli, whereas lexical access is accelerated for words but slowed for pseudowords. Sublexical length effects in English were larger than previously found in native speakers of English, whereas the effects of length on lexical access in English were similar to those previously reported for English. These findings suggest that sublexical but not lexical processing remains tuned to the orthographic structure of the L1. In Studies 2 and 3, graded sublexical and lexical language similarities to each of the two languages biased language membership decisions. Moreover, sublexical processing and phonological encoding were faster for letter strings with an L1-typical sublexical structure. fMRI data of Study 3 revealed that brain activity in left ventral occipito-temporal cortex, previously associated with processing of letter strings, was positively correlated with sublexical similarity L1. The level of ambiguity in lexical similarity to L1 and L2 determined activation in bilateral angular gyri, providing direct evidence for language-unselective lexical access in a region previously associated with lexical access in the L1. Brain activations in the left supramarginal gyrus and temporo-parietal junction, previously associated with phonological encoding, were positively correlated with lexical similarity to L1. This finding suggests that phonological representations of the L1 but not L2 are automatically activated during language decisions on visual input. Finally, multivariate fMRI analyses revealed that language membership information is contained in distributed activation patterns throughout the visual word processing network, as well as in additional right parietal areas. Study 4 showed that in a consistent language setting framing effects are comparable in native and foreign languages and at all foreign language proficiency levels. However, it also revealed that the framing effect is reduced following a language switch, not only into a foreign language but also into the L1. This finding is best interpreted in terms of increased cognitive control following language switching, rather than as altered affective processing during foreign language use, as was claimed in previous studies. Based on these findings I propose to extend neurocognitive models of bilingual visual word recognition in several aspects. First, sublexical mapping of orthography should prioritize the orthographic structure of the L1. Second, the summed activation propagated from sublexical representations to the lexicon should increase with stimulus length, which would lead to faster lexical access for longer words. Third, lexical representations should be clustered by language, to constitute a basis for language membership information within the core visual word processing network. Fourth, input to language membership representations should stem from all sublexical and lexical representations, weighted by their typicality for each language. Finally, our findings imply that the input language affects non-linguistic decision processes through a modulation of cognitive control levels and not affective processing. This link needs to be included in integrated models of perceptual linguistic and higher-level processing.Zweisprachige Individuen können sowohl in ihrer Zweitsprache (L2) lesen, als auch Entscheidungen treffen, die auf in der L2 gelesenen Informationen beruhen. Im Vergleich zur monolingualen visuellen Wortverarbeitung erfordert das Lesen in einer zweiten Sprache zusätzliche Verarbeitungsschritte, die auch nicht-linguistische Kognition beeinflussen können. Die Integration dieser in Modelle der monolingualen Wortverarbeitung und Entscheidungsfindung birgt einige offene Fragen. So erfordert die bilinguale Wortverarbeitung, dass die Sprachzugehörigkeit eines Wortes erkannt wird. Außerdem müssen Buchstabenfolgen lexikalisch und phonologisch in beiden Sprachen verarbeitet werden. Dabei können sich die zugrundeliegenden Abbildungsregeln zwischen den Sprachen unterscheiden und sich sogar gegenseitig widersprechen. Diese perzeptuellen Aspekte der Wortverarbeitung dienen oftmals als Basis für weitere übergeordnete Entscheidungsprozesse, wie z.B. Risikoentscheidungen. Ein wichtiges Beispiel für sprachliche Effekte auf Risikoentscheidungen ist der Framingeffekt. Dieser beschreibt, dass die Bereitschaft ein Risiko einzugehen höher ist, wenn die negativen Konsequenzen von Entscheidungsalternativen explizit beschrieben werden, als wenn positive Konsequenzen betont werden. Der Einfluss des Fremdsprachengebrauchs auf tragende Prozesse des Framingeffekts, insbesondere auf die affektive Verarbeitung und kognitive Kontrolle, wurde zwar desöfteren postuliert, jedoch bis dato nur unzureichend empirisch bestätigt. Die vorliegende Dissertation untersuchte kognitive und neuronale Aspekte bilingualer visueller Wortverarbeitung und Entscheidungsfindung am Beispiel von Zweisprachigkeit mit Deutsch als Muttersprache (L1) und Englisch als Zweitsprache (L2). In der ersten Studie verglich ich die Effekte von Wort -und Pseudowortlänge auf die sublexikalische Enkodierung und den lexikalischen Zugriff im Deutschen und Englischen. Dazu wurden Reaktionszeiten in einer lexikalischen Entscheidungsaufgabe mit Hilfe eines Diffusionsmodells modelliert. In der zweiten Studie untersuchte ich inwiefern kontinuierliche sublexikalische und lexikalische Ähnlichkeit von Buchstabenfolgen zu deutschen und englischen Wörtern die Sprachzuordnung und Wortbenennung in einem sprachlich ambigen Kontext beeinflusst. In der dritten Studie wurden die neuronalen Korrelate von Sprachähnlichkeit und -zugehörigkeit mithilfe funktioneller Magnetresonanztomographie (fMRT) betrachtet. Schließlich untersuchte ich in der vierten Studie, ob ein verminderter Framingeffekt in einer Fremdsprache auf reduzierten Affekt oder aber erhöhte kognitive Kontrolle zurückgeführt werden kann. Studie 1 zeigte, dass die sublexikalische Enkodierung von Wörtern und Pseudowörtern mit zunehmender Länge länger andauert. Hingegen war der lexikalische Zugriff für längere Wortstimuli schneller, für längere Pseudoworte jedoch langsamer. Desweiteren waren sublexikalische Längeneffekte im Englischen größer als im Deutschen. Dieser Befund steht im Gegensatz zu in der Vergangenheit berichteten größeren muttersprachlichen Längeneffekten im Deutschen als in Englischen. Er legt daher nahe, dass die bilinguale sublexikalische Verarbeitung auf die orthographische Struktur der Muttersprache ausgerichtet ist. In Studien 2 und 3 wurde beobachtet, dass die Sprachzuordnung von Pseudoworten durch ihre sublexikalische und lexikalische Ähnlichkeit zu jeder der Sprachen beeinflusst wird. Zudem wurde sublexikalische und phonologische Enkodierung durch sublexikalische Ähnlichkeit zur L1 beschleunigt. fMRT-Daten aus Studie 3 zeigten eine positive Korrelation zwischen Hirnaktivierungen im linken okzipito-temporalen Kortex, der mit der Verarbeitung von Buchstabenfolgen assoziiert wird, und der sublexikalischen Ähnlichkeit der Stimuli zur L1. Darüber hinaus waren die bilateralen Angulargyri umso aktiver, je ambiger die lexikalische Ähnlichkeit der Stimuli zu beiden Sprachen war. Dieser Befund bietet eine direkte Evidenz für sprachunabhängigen lexikalischen Zugriff in einer Region, die bisher mit lexikalischen Zugriff in monolingualen Studien assoziiert wurde. Aktivierungen im linken supramarginalen Gyrus und im temporo-parietalen Bereich, zwei Regionen denen in phonologischer Enkodierung eine wichtige Rolle zukommt, waren mit der lexikalischen Ähnlichkeit zur L1 positiv korreliert. Dieser Befund legt nahe, dass visuell basierte Sprachentscheidungen eine automatische Aktivierung der L1-Phonologie nach sich ziehen. Aus multivariaten fMRT- Analysen wurde ersichtlich, dass Sprachzugehörigkeit sowohl in verteilten Aktivierungsmustern im gesamten Sprachnetzwerk, als auch in weiteren rechtsparietalen Regionen enkodiert wird. Studie 4 zeigte, dass der Framingeffekt von der Sprache, in der eine Risikoentscheidung präsentiert wurde, nicht beeinflusst wird, solange die Risikoentscheidung in einen einheitlichen Sprachkontext eingebettet ist. Desweiteren war der Framingeffekt in einer Fremdsprache vom Niveau der Sprachkenntnisse unabhängig. Allerdings reduzierte ein unmittelbarer Sprachwechsel den Framingeffekt, und zwar unabhängig davon, ob von der Muttersprache in eine Fremdsprache oder andersherum gewechselt wurde. Wir interpretieren diesen Befund als das Ergebnis von erhöhter kognitiver Kontrolle beim Sprachwechsel, und nicht – wie in früheren Studien vorgeschlagen – als modifizierte affektive Verarbeitung beim Gebrauch einer Fremdsprache. Basierend auf diesen Ergebnissen diskutiere ich eine Erweiterung neurokognitiver Modelle zweisprachiger visueller Wortbearbeitung. In diesem sollte die sublexikalische orthographische Enkodierung die Struktur der L1 priorisieren. Desweiteren sollte die Gesamtaktivierung, die von sublexikalischen Repräsentationen an das Lexikon propagiert wird, mit Wortlänge ansteigen und daher zu einem schnelleren lexikalischen Zugriff führen. Zudem könnten sich lexikalische Repräsentationen nach Sprache gruppieren, um eine Basis für Sprachrepräsentationen innerhalb des Wortverarbeitungsnetzwerkes zu ermöglichen. Außerdem sollten modelltheoretisch alle sublexikalischen und lexikalischen Repräsentationen mit denen der Sprachzugehörigkeit verbunden sein, wobei die Verbindungsstärke mit Sprachähnlichkeit gewichtet werden sollte. Schließlich sollte der Effekt von sprachbedingt erhöhter kognitiver Kontrolle und Entscheidungen unter Risiko in Modellen zur Integration visueller Wortverarbeitung und nicht-linguistischer Entscheidungsprozesse hergestellt werden