Tracking Eye Movements as a Window on Language Processing: The Visual World Paradigm

This entry overviews the pioneering experimental studies exploiting eye movement data to investigate language processing in real time. After examining how vision and language were found to be closely related, herein focus the discussion on the evolution of eye-tracking methodologies to investigate children’s language development. To conclude, herein provide some insights about the use of eye-tracking technology for research purposes, focusing on data collection and data analysis

Using normative modelling to detect disease progression in mild cognitive impairment and Alzheimer’s disease in a cross-sectional multi-cohort study

Abstract Normative modelling is an emerging method for quantifying how individuals deviate from the healthy populational pattern. Several machine learning models have been implemented to develop normative models to investigate brain disorders, including regression, support vector machines and Gaussian process models. With the advance of deep learning technology, the use of deep neural networks has also been proposed. In this study, we assessed normative models based on deep autoencoders using structural neuroimaging data from patients with Alzheimer’s disease (n = 206) and mild cognitive impairment (n = 354). We first trained the autoencoder on an independent dataset (UK Biobank dataset) with 11,034 healthy controls. Then, we estimated how each patient deviated from this norm and established which brain regions were associated to this deviation. Finally, we compared the performance of our normative model against traditional classifiers. As expected, we found that patients exhibited deviations according to the severity of their clinical condition. The model identified medial temporal regions, including the hippocampus, and the ventricular system as critical regions for the calculation of the deviation score. Overall, the normative model had comparable cross-cohort generalizability to traditional classifiers. To promote open science, we are making all scripts and the trained models available to the wider research community

Normative values of the topological metrics of the structural connectome: a multi-site reproducibility study across the italian neuroscience network

Purpose: The use of topological metrics to derive quantitative descriptors from structural connectomes is receiving increasing attention but deserves specific studies to investigate their reproducibility and variability in the clinical context. This work exploits the harmonization of diffusion-weighted acquisition for neuroimaging data performed by the Italian Neuroscience and Neurorehabilitation Network initiative to obtain normative values of topological metrics and to investigate their reproducibility and variability across centers. Methods: Different topological metrics, at global and local level, were calculated on multishell diffusion-weighted data acquired at high-field (e.g. 3 T) Magnetic Resonance Imaging scanners in 13 different centers, following the harmonization of the acquisition protocol, on young and healthy adults. A "traveling brains" dataset acquired on a subgroup of subjects at 3 different centers was also analyzed as reference data. All data were processed following a common processing pipeline that includes data pre-processing, tractography, generation of structural connectomes and calculation of graph-based metrics. The results were evaluated both with statistical analysis of variability and consistency among sites with the traveling brains range. In addition, inter-site reproducibility was assessed in terms of intra-class correlation variability. Results: The results show an inter-center and inter-subject variability of <10%, except for "clustering coefficient" (variability of 30%). Statistical analysis identifies significant differences among sites, as expected given the wide range of scanners' hardware. Conclusions: The results show low variability of connectivity topological metrics across sites running a harmonised protocol

Quantitative MRI Harmonization to Maximize Clinical Impact: The RIN-Neuroimaging Network

Neuroimaging studies often lack reproducibility, one of the cardinal features of the scientific method. Multisite collaboration initiatives increase sample size and limit methodological flexibility, therefore providing the foundation for increased statistical power and generalizable results. However, multisite collaborative initiatives are inherently limited by hardware, software, and pulse and sequence design heterogeneities of both clinical and preclinical MRI scanners and the lack of benchmark for acquisition protocols, data analysis, and data sharing. We present the overarching vision that yielded to the constitution of RIN-Neuroimaging Network, a national consortium dedicated to identifying disease and subject-specific in-vivo neuroimaging biomarkers of diverse neurological and neuropsychiatric conditions. This ambitious goal needs efforts toward increasing the diagnostic and prognostic power of advanced MRI data. To this aim, 23 Italian Scientific Institutes of Hospitalization and Care (IRCCS), with technological and clinical specialization in the neurological and neuroimaging field, have gathered together. Each IRCCS is equipped with high- or ultra-high field MRI scanners (i.e., ≥3T) for clinical or preclinical research or has established expertise in MRI data analysis and infrastructure. The actions of this Network were defined across several work packages (WP). A clinical work package (WP1) defined the guidelines for a minimum standard clinical qualitative MRI assessment for the main neurological diseases. Two neuroimaging technical work packages (WP2 and WP3, for clinical and preclinical scanners) established Standard Operative Procedures for quality controls on phantoms as well as advanced harmonized quantitative MRI protocols for studying the brain of healthy human participants and wild type mice. Under FAIR principles, a web-based e-infrastructure to store and share data across sites was also implemented (WP4). Finally, the RIN translated all these efforts into a large-scale multimodal data collection in patients and animal models with dementia (i.e., case study). The RIN-Neuroimaging Network can maximize the impact of public investments in research and clinical practice acquiring data across institutes and pathologies with high-quality and highly-consistent acquisition protocols, optimizing the analysis pipeline and data sharing procedures

Development of a prediction model of conversion to Alzheimer’s disease in people with mild cognitive impairment: the statistical analysis plan of the INTERCEPTOR project

Background In recent years, signifcant eforts have been directed towards the research and development of disease-modifying therapies for dementia. These drugs focus on prodromal (mild cognitive impairment, MCI) and/ or early stages of Alzheimer’s disease (AD). Literature evidence indicates that a considerable proportion of individuals with MCI do not progress to dementia. Identifying individuals at higher risk of developing dementia is essential for appropriate management, including the prescription of new disease-modifying therapies expected to become available in clinical practice in the near future. Methods The ongoing INTERCEPTOR study is a multicenter, longitudinal, interventional, non-therapeutic cohort study designed to enroll 500 individuals with MCI aged 50–85 years. The primary aim is to identify a biomarker or a set of biomarkers able to accurately predict the conversion from MCI to AD dementia within 3 years of follow-up. The biomarkers investigated in this study are neuropsychological tests (mini-mental state examination (MMSE) and delayed free recall), brain glucose metabolism ([18F]FDG-PET), MRI volumetry of the hippocampus, EEG brain connectivity, cerebrospinal fuid (CSF) markers (p-tau, t-tau, Aβ1-42, Aβ1-42/1–40 ratio, Aβ1-42/p-Tau ratio) and APOE genotype. The baseline visit includes a full cognitive and neuropsychological evaluation, as well as the collection of clinical and socio-demographic information. Prognostic models will be developed using Cox regression, incorporating individual characteristics and biomarkers through stepwise selection. Model performance will be evaluated in terms of discrimination and calibration and subjected to internal validation using the bootstrapping procedure. The fnal model will be visually represented as a nomogram. Discussion This paper contains a detailed description of the statistical analysis plan to ensure the reproducibility and transparency of the analysis. The prognostic model developed in this study aims to identify the populatio

Differential diagnosis of neurodegenerative dementias with the explainable MRI based machine learning algorithm MUQUBIA

Biomarker-based differential diagnosis of the most common forms of dementia is becoming increasingly important. Machine learning (ML) may be able to address this challenge. The aim of this study was to develop and interpret a ML algorithm capable of differentiating Alzheimer's dementia, frontotemporal dementia, dementia with Lewy bodies and cognitively normal control subjects based on sociodemographic, clinical, and magnetic resonance imaging (MRI) variables. 506 subjects from 5 databases were included. MRI images were processed with FreeSurfer, LPA, and TRACULA to obtain brain volumes and thicknesses, white matter lesions and diffusion metrics. MRI metrics were used in conjunction with clinical and demographic data to perform differential diagnosis based on a Support Vector Machine model called MUQUBIA (Multimodal Quantification of Brain whIte matter biomArkers). Age, gender, Clinical Dementia Rating (CDR) Dementia Staging Instrument, and 19 imaging features formed the best set of discriminative features. The predictive model performed with an overall Area Under the Curve of 98%, high overall precision (88%), recall (88%), and F1 scores (88%) in the test group, and good Label Ranking Average Precision score (0.95) in a subset of neuropathologically assessed patients. The results of MUQUBIA were explained by the SHapley Additive exPlanations (SHAP) method. The MUQUBIA algorithm successfully classified various dementias with good performance using cost-effective clinical and MRI information, and with independent validation, has the potential to assist physicians in their clinical diagnosis

How children acquire adjectives: Evidence from three eye-tracking studies on Italian

While the literature on early language acquisition has mainly focused on nouns and verbs, studies on adjectives are comparatively scarce and have thus far provided contrasting evidence on the timing and mode of acquisition of adjectival meaning in children younger than 4. This thesis presents three eye-tracking studies exploring the online processing of adjectives by children (2;4 – 5;3 years) in comparison to adult controls, providing insights on how linguistic information and visual context interact during real-time comprehension. Experiment 1 investigated potential differences in the interpretation of three classes of adjectives, intersective (e.g., red), relative (e.g., big) and absolute (e.g., full). 38 Italian monolingual children (2;4 – 5;3) were tested in a visual-world task, where they listened to noun-adjective combinations in a four-picture scenario. Results showed that children as young as 28 months were slower than adults when interpreting noun-adjective combinations, while their looking pattern in the interpretation process was essentially the same. Furthermore, the computation of intersective adjectives was faster than that of absolute and, especially, relative adjectives, showing that children are sensitive to the different ways in which each adjective class is interpreted within different contexts. In Experiment 2 the complexity of Experiment 1 was reduced by lowering the processing load associated with both the different semantic classes of adjectives and the four-picture scenario. 28 Italian-native children (2;4 – 5;2) were presented with a two-picture display while listening to nouns combined with color-adjectives. In Experiment 2, the visual conditions varied according to the informativeness of the noun or the adjective with respect to the target referent. Results showed evidence that adjective processing develops over time. When the computation of the noun was insufficient and the integration with the adjective was necessary to resolve reference, the youngest children, unlike 3- and 4-year-olds, failed to interpret adjectives correctly and, consequently, task resolution. Experiment 3 investigated children’s incremental processing of prenominal adjectives and their ability to predict the following noun based on the lexical meaning of the adjective. 39 Italian children (2;4 – 5;3) were tested in the online 3 interpretation of Italian predicative yes/no questions (e.g., È morbido il cuscino?, lit. ‘Is soft the pillow?’) while looking at two pictures on the screen. Here, the informativeness of the adjective was manipulated. Results showed that, when informative (e.g., soft, upon looking at a pillow and a bone), the adjective was processed incrementally, i.e., before the noun was heard, indicating that children as young as 28 months are able to predict the upcoming noun based on adjective meaning. Furthermore, children were successful independently of whether the interpretation of the adjective required world knowledge (e.g., being soft for a pillow) or the exploration of the visual scene (e.g., being open for a window). Taken together, the three studies provided compelling evidence of a continuous process in children’s development of sophisticated, adult-like processing skills. By means of eye-tracking, we were able to reveal that the overall difference between children and adults is mostly attributable to toddlers younger that 36 months of age, whose processing skills are still limited when it comes to the meaning computation of noun-adjective combinations. However, from the age of 3, children’s processing abilities improve rapidly and, within a few months, they become successful parsers

How children acquire adjectives : Evidence from three eye-tracking studies on Italian

While the literature on early language acquisition has mainly focused on nouns and verbs, studies on adjectives are comparatively scarce and have thus far provided contrasting evidence on the timing and mode of acquisition of adjectival meaning in children younger than 4. This thesis presents three eye-tracking studies exploring the online processing of adjectives by children (2;4 – 5;3 years) in comparison to adult controls, providing insights on how linguistic information and visual context interact during real-time comprehension. Experiment 1 investigated potential differences in the interpretation of three classes of adjectives, intersective (e.g., red), relative (e.g., big) and absolute (e.g., full). 38 Italian monolingual children (2;4 – 5;3) were tested in a visual-world task, where they listened to noun-adjective combinations in a four-picture scenario. Results showed that children as young as 28 months were slower than adults when interpreting noun-adjective combinations, while their looking pattern in the interpretation process was essentially the same. Furthermore, the computation of intersective adjectives was faster than that of absolute and, especially, relative adjectives, showing that children are sensitive to the different ways in which each adjective class is interpreted within different contexts. In Experiment 2 the complexity of Experiment 1 was reduced by lowering the processing load associated with both the different semantic classes of adjectives and the four-picture scenario. 28 Italian-native children (2;4 – 5;2) were presented with a two-picture display while listening to nouns combined with color-adjectives. In Experiment 2, the visual conditions varied according to the informativeness of the noun or the adjective with respect to the target referent. Results showed evidence that adjective processing develops over time. When the computation of the noun was insufficient and the integration with the adjective was necessary to resolve reference, the youngest children, unlike 3- and 4-year-olds, failed to interpret adjectives correctly and, consequently, task resolution. Experiment 3 investigated children’s incremental processing of prenominal adjectives and their ability to predict the following noun based on the lexical meaning of the adjective. 39 Italian children (2;4 – 5;3) were tested in the online interpretation of Italian predicative yes/no questions (e.g., È morbido il cuscino?, lit. ‘Is soft the pillow?’) while looking at two pictures on the screen. Here, the informativeness of the adjective was manipulated. Results showed that, when informative (e.g., soft, upon looking at a pillow and a bone), the adjective was processed incrementally, i.e., before the noun was heard, indicating that children as young as 28 months are able to predict the upcoming noun based on adjective meaning. Furthermore, children were successful independently of whether the interpretation of the adjective required world knowledge (e.g., being soft for a pillow) or the exploration of the visual scene (e.g., being open for a window). Taken together, the three studies provided compelling evidence of a continuous process in children’s development of sophisticated, adult-like processing skills. By means of eye-tracking, we were able to reveal that the overall difference between children and adults is mostly attributable to toddlers younger that 36 months of age, whose processing skills are still limited when it comes to the meaning computation of noun-adjective combinations. However, from the age of 3, children’s processing abilities improve rapidly and, within a few months, they become successful parsers.publishe

Processing adjectives in development: Evidence from eye-tracking

Combining adjective meaning with the modified noun is particularly challenging for children under three years. Previous research suggests that in processing noun-adjective phrases children may over-rely on noun information, delaying or omitting adjective interpretation. However, the question of whether this difficulty is modulated by semantic differences among (subsective) adjectives is underinvestigated.A visual-world experiment explores how Italian-learning children (N=38, 2;4-5;3) process noun-adjective phrases and whether their processing strategies adapt based on the adjective class. Our investigation substantiates the proficient integration of noun and adjective semantics by children. Nevertheless, alligning with previous research, a notable asymmetry is evident in the interpretation of nouns and adjectives, the latter being integrated more slowly. Remarkably, by testing toddlers across a wide age range, we observe a developmental trajectory in processing, supporting a continuity approach to children's development. Moreover, we reveal that children exhibit sensitivity to the distinct interpretations associated with each subsective adjective

Experimental evidence for the interpretation of definite plural articles as markers of genericity : How Italian can help

In the Romance languages, definite plural articles (e.g., le rane ‘the frogs’) are generally ambiguous between a generic and a specific interpretation, and speakers must reconstruct the intended interpretation through the linguistic or extra-linguistic context. Following the “polar bear” paradigm implemented in Czypionka & Kupisch (2019)’s investigation on German, the goal of the present study is to check the suitability of their test on article semantics, by establishing to what extent native speakers of Italian interpret ambiguous definite plural DPs as generic or specific in the presence of a nonlinguistic picture context. We present judgment and reaction time data monitoring the preferred reading of sentences introduced by different kinds of noun phrases (e.g., Le rane/Queste rane/Le rane di solito sono verdi/gialle ‘The/These/Usually frogs are green/yellow’), while looking at pictures showing prototypical or non-prototypical properties (e.g., green vs. yellow frogs). Our results show that both possible interpretations of definite plural articles are routinely considered in Italian, despite the presence of a picture with specific referents, validating the “polar bear” paradigm as a suitable test of article semantics.publishe