Developmental constraints on learning artificial grammars with fixed, flexible and free word order

Human learning, although highly flexible and efficient, is constrained in ways that facilitate or impede the acquisition of certain systems of information. Some such constraints, active during infancy and childhood, have been proposed to account for the apparent ease with which typically developing children acquire language. In a series of experiments, we investigated the role of developmental constraints on learning artificial grammars with a distinction between shorter and relatively frequent words (‘function words,’ F-words) and longer and less frequent words (‘content words,’ C-words). We constructed 4 finite-state grammars, in which the order of F-words, relative to C-words, was either fixed (F-words always occupied the same positions in a string), flexible (every F-word always followed a C-word), or free. We exposed adults (N = 84) and kindergarten children (N = 100) to strings from each of these artificial grammars, and we assessed their ability to recognize strings with the same structure, but a different vocabulary. Adults were better at recognizing strings when regularities were available (i.e., fixed and flexible order grammars), while children were better at recognizing strings from the grammars consistent with the attested distribution of function and content words in natural languages (i.e., flexible and free order grammars). These results provide evidence for a link between developmental constraints on learning and linguistic typology

Task-dependent fractal patterns of information processing in working memory

We applied detrended fluctuation analysis, power spectral density, and eigenanalysis of detrended cross-correlations to investigate fMRI data representing a diurnal variation of working memory in four visual tasks: two verbal and two nonverbal. We show that the degree of fractal scaling is regionally dependent on the engagement in cognitive tasks. A particularly apparent difference was found between memorisation in verbal and nonverbal tasks. Furthermore, the detrended cross-correlations between brain areas were predominantly indicative of differences between resting state and other tasks, between memorisation and retrieval, and between verbal and nonverbal tasks. The fractal and spectral analyses presented in our study are consistent with previous research related to visuospatial and verbal information processing, working memory (encoding and retrieval), and executive functions, but they were found to be more sensitive than Pearson correlations and showed the potential to obtain other subtler results. We conclude that regionally dependent cognitive task engagement can be distinguished based on the fractal characteristics of BOLD signals and their detrended cross-correlation structure

Constraints on Language Learning : behavioral and neurocognitive studies with adults and children

This thesis will contribute to a body of experimental work addressing the question of whether language learning plays a role in certain fundamental design properties of natural languages. Methodologically, this thesis seeks to extend the artificial language learning paradigm, investigating whether learners are sensitive to the constraints embodied by key properties of languages. For example, we will explore whether communicative pressure influences the final outcome of language learning, namely how the structures that are acquired by individuals are transmitted to downstream generations. We will also explore how basic language learning constraints operate in different age groups and, importantly, cross-linguistically. Next to the behavioral experiments focusing on learning and its outcomes, we will look at preliminary electrophysiological correlates of basic compositional processing in the early stages of learning a miniature artificial language using electroencephalography (EEG). In this general introduction I will briefly discuss some of the relevant concepts and methods which will be used in three studies that constitute this thesis

FluentSigners-50: A signer independent benchmark dataset for sign language processing

This paper presents a new large-scale signer independent dataset for Kazakh-Russian Sign Language (KRSL) for the purposes of Sign Language Processing. We envision it to serve as a new benchmark dataset for performance evaluations of Continuous Sign Language Recognition (CSLR) and Translation (CSLT) tasks. The proposed FluentSigners-50 dataset consists of 173 sentences performed by 50 KRSL signers resulting in 43,250 video samples. Dataset contributors recorded videos in real-life settings on a wide variety of backgrounds using various devices such as smartphones and web cameras. Therefore, distance to the camera, camera angles and aspect ratio, video quality, and frame rates varied for each dataset contributor. Additionally, the proposed dataset contains a high degree of linguistic and inter-signer variability and thus is a better training set for recognizing a real-life sign language. FluentSigners-50 baseline is established using two state-of-the-art methods, Stochastic CSLR and TSPNet. To this end, we carefully prepared three benchmark train-test splits for models’ evaluations in terms of: signer independence, age independence, and unseen sentences. FluentSigners-50 is publicly available at https://krslproject.github.io/FluentSigners-50/publishedVersio

Multimodal imaging of brain reorganization in hearing late learners of sign language

The neural plasticity underlying language learning is a process rather than a single event. However, the dynamics of training - induced brain reorganization have rarely been examined, especially using a multimodal magnetic resonance imaging approach, which allows us to study the relationship between functional and structural changes. We focus on sign language acquisition in hearing adults who underwent an 8‐month long course and five neuroimaging sessions. We assessed what neural changes occurred as participants learned a new language in a different modality - as reflected by task‐based activity, connectivity changes, and co‐occurring structural alterations. Major changes in the activity pattern appeared after just 3 months of learning, as indicated by increases in activation within the modality‐independent perisylvian language network, together with increased activation in modality‐dependent parieto‐occipital, visuospatial and motion‐sensitive regions. Despite further learning, no alterations in activation were detected during the following months. However, enhanced coupling between left‐lateralized occipital and inferior frontal regions was observed as the proficiency increased. Furthermore, an increase in gray matter volume was detected in the left inferior frontal gyrus which peaked at the end of learning. Overall, these results showed complexity and temporal distinctiveness of various aspects of brain reorganization associated with learning of new language in different sensory modality

Cognitive and Neurobiological Degeneration of the Mental Lexicon in Primary Progressive Aphasia

The ease with which we use the thousands of words in our vocabulary stands in stark contrast to our difficulty establishing how they are organized in our mind and brain. The breakdown of language due to cortical atrophy in primary progressive aphasia (PPA) creates conditions to study this organization at a cognitive and neurobiological level in that the three variants of this disease, namely non-fluent, logopenic, and semantic PPA, each bear their own signature of language-specific decline and cortical atrophy. As the impaired regions in each variant are linked to different lexical and semantic attributes of words, lexical decision performance of individuals with the distinct variants can reveal the conceptual and neural architecture of the lexicon through an anatomical-behavioral relationship. This dissertation investigated which lexical and semantic factors influence the structural degeneration of word processing in individuals with each variant of PPA through three studies that focused on the role of general semantic knowledge, psycholinguistic variables, and sensory-perceptual features, respectively. In Study 1, 41 individuals with PPA (13 non-fluent, 14 logopenic, and 14 semantic) as well as healthy controls (N = 25) performed a lexical decision task that consisted of 355 real words, carefully controlled on a broad range of psycholinguistic and semantic variables, and 175 pseudowords matched with the real words on the psycholinguistic variables. Two additional non-verbal semantic tasks (Pyramids and Palm Trees test and Over-regular Object Test) were administered to assess semantic ability and its relation with lexical decision performance. Results showed that—contrary to diagnostic expectations for the PPA variants—all three groups of individuals with PPA scored below the performance of matched control participants. The lexical-decision performance across all individuals with PPA correlated with semantic ability, but this correlation was not significant when separately analyzed per diagnosis. These findings suggest that semantic ability plays an active role in word recognition, but is not essential to lexical-semantic processing. In Study 2, the performance of the same participants was analyzed on a selected subset of the 355 words to examine the differential influence of the psycholinguistic factors lexical frequency, age of acquisition, and neighborhood density on lexical-semantic processing across the three diagnostic groups. The results demonstrated that lexical frequency has the largest influence on lexical-semantic processing, but that independent of that, age of acquisition and neighborhood density also play a role. The effect of these two variables becomes more salient dependent on the variant of PPA, accordant to the patterns of atrophy. That is, individuals with non-fluent and logopenic PPA experienced a neighborhood density effect consistent with atrophy in the inferior frontal and temporoparietal cortices, associated with lexical analysis and word form processing. By contrast, individuals with semantic PPA experienced an age of acquisition effect consistent with atrophy in the anterior temporal lobe which has been associated with semantic processing in previous literature. These findings suggest that the degeneration of lexical-semantic processing is affected by lexical factors—which relate to language-specific brain regions—in line with a hierarchical mental lexicon structure, such that a selective deficit at one of the levels of the mental lexicon results in distinctively expressed effects among psycholinguistic variables. Study 3 employed voxel-based morphometry (VBM) to identify the association between cortical volume—measured through T1-weighted magnetic resonance images (MRI)—and lexical decision performance related to sensory-perceptual features in 37 of the individuals with PPA and 17 of the controls on a second subset of the 355 words. Results showed that at both behavioral and neurobiological levels, semantic sensory-perceptual features of words (a strong association with, e.g., sound or action) influence lexical decision performance across all three groups with PPA. The results highlight the roles of the right hemisphere, the cerebellum, and the anterior temporal lobe in processing various sensory-perceptual features of concepts. The anterior temporal lobe has been proposed to be a semantic hub which processes various sensory-perceptual features (‘spokes’) into a conceptual representation in the hub-and-spoke model. The current results confirm this hub-role of the anterior temporal lobe, as well as the link of the ‘spokes’ to sensory-perceptual brain regions, as proposed by the hypothesis of embodied cognition. Most importantly, the results suggest that the intensity of semantic processing in the anterior temporal lobe is regulated by the degree of association with sensory-perceptual information. The current research presents novel evidence that lexical-semantic processing is influenced by a combination of lexical and semantic factors at both conceptual and neurobiological levels, which can become impaired in different ways in individuals with PPA based on a set of anatomical-behavioral relationships. In particular, this dissertation broke new ground in demonstrating that the intensity of semantic processing in the anterior temporal lobe depends on the degree of sensory-perceptual information of concepts, supporting both the hub-and-spoke model and the hypothesis of embodied cognition. As well, this dissertation established the independent effects of lexical frequency from age of acquisition and neighborhood density and their roles in lexical-semantic decline in PPA, supporting the theory of hierarchical distinctions between lexemes and their conceptual representations in the mental lexicon

Directional adposition use in English, Swedish and Finnish

Directional adpositions such as to the left of describe where a Figure is in relation to a Ground. English and Swedish directional adpositions refer to the location of a Figure in relation to a Ground, whether both are static or in motion. In contrast, the Finnish directional adpositions edellä (in front of) and jäljessä (behind) solely describe the location of a moving Figure in relation to a moving Ground (Nikanne, 2003). When using directional adpositions, a frame of reference must be assumed for interpreting the meaning of directional adpositions. For example, the meaning of to the left of in English can be based on a relative (speaker or listener based) reference frame or an intrinsic (object based) reference frame (Levinson, 1996). When a Figure and a Ground are both in motion, it is possible for a Figure to be described as being behind or in front of the Ground, even if neither have intrinsic features. As shown by Walker (in preparation), there are good reasons to assume that in the latter case a motion based reference frame is involved. This means that if Finnish speakers would use edellä (in front of) and jäljessä (behind) more frequently in situations where both the Figure and Ground are in motion, a difference in reference frame use between Finnish on one hand and English and Swedish on the other could be expected. We asked native English, Swedish and Finnish speakers’ to select adpositions from a language specific list to describe the location of a Figure relative to a Ground when both were shown to be moving on a computer screen. We were interested in any differences between Finnish, English and Swedish speakers. All languages showed a predominant use of directional spatial adpositions referring to the lexical concepts TO THE LEFT OF, TO THE RIGHT OF, ABOVE and BELOW. There were no differences between the languages in directional adpositions use or reference frame use, including reference frame use based on motion. We conclude that despite differences in the grammars of the languages involved, and potential differences in reference frame system use, the three languages investigated encode Figure location in relation to Ground location in a similar way when both are in motion. Levinson, S. C. (1996). Frames of reference and Molyneux’s question: Crosslingiuistic evidence. In P. Bloom, M.A. Peterson, L. Nadel & M.F. Garrett (Eds.) Language and Space (pp.109-170). Massachusetts: MIT Press. Nikanne, U. (2003). How Finnish postpositions see the axis system. In E. van der Zee & J. Slack (Eds.), Representing direction in language and space. Oxford, UK: Oxford University Press. Walker, C. (in preparation). Motion encoding in language, the use of spatial locatives in a motion context. Unpublished doctoral dissertation, University of Lincoln, Lincoln. United Kingdo

Improving the translation environment for professional translators

When using computer-aided translation systems in a typical, professional translation workflow, there are several stages at which there is room for improvement. The SCATE (Smart Computer-Aided Translation Environment) project investigated several of these aspects, both from a human-computer interaction point of view, as well as from a purely technological side. This paper describes the SCATE research with respect to improved fuzzy matching, parallel treebanks, the integration of translation memories with machine translation, quality estimation, terminology extraction from comparable texts, the use of speech recognition in the translation process, and human computer interaction and interface design for the professional translation environment. For each of these topics, we describe the experiments we performed and the conclusions drawn, providing an overview of the highlights of the entire SCATE project