Morphology in the Age of Pre-trained Language Models

The field of natural language processing (NLP) has adopted deep learning methods in the past 15 years. Nowadays the state-of-the-art in most NLP tasks is some kind of neural model, often the fine-tuned version of a pre-trained language model. The efficacy of these models is demonstrated on various English benchmarks and increasingly, other monolingual and multimultilingual benchmarks. In this dissertation I explore the application of deep learning models on low level tasks, particularly morphosyntactic tasks in multiple languages. The first part of this dissertation (Chapters 3 and 4) explores the application of deep learning models for classical morphosyntactic tasks such as morphological analysis and generation in dozens of languages with special focus on Hungarian. The second part of this dissertation (Chapters 5 to 8) deals with pre-trained language models, mostly models from the BERT family. I include some experiments on GPT-4o and GPT-4o-mini. These models show excellent performance on various tasks in English and some high density languages. However, their evaluation in medium and low density languages is lacking. I present a methodology for generating morphosyntactic benchmarks in arbitrary languages and I analyze multiple BERT-like models in detail. My main tool for analysis is the probing methodology which I extend the with perturbations, the systematic removal of certain information from the sentence. I use Shapley values to further refine my analysis

A Machine-Learning-Based Analysis of Resting Electroencephalogram Signals to Identify Latent Schizotypal and Bipolar Development in Healthy University Students

Background: Early and accurate diagnosis is crucial for effective prevention and treatment of severe mental illnesses, such as schizophrenia and bipolar disorder. However, identifying these conditions in their early stages remains a significant challenge. Our goal was to develop a method capable of detecting latent disease liability in healthy volunteers. Methods: Using questionnaires examining affective temperament and schizotypal traits among voluntary, healthy university students (N = 710), we created three groups. These were a group characterized by an emphasis on positive schizotypal traits (N = 20), a group showing cyclothymic temperament traits (N = 17), and a control group showing no susceptibility in either direction (N = 21). We performed a resting-state EEG examination as part of a complex psychological, electrophysiological, psychophysiological, and laboratory battery, and we developed feature-selection machine-learning methods to differentiate the low-risk groups. Results: Both low-risk groups could be reliably (with 90% accuracy) separated from the control group. Conclusions: Models applied to the data allowed us to differentiate between healthy university students with latent schizotypal or bipolar tendencies. Our research may improve the sensitivity and specificity of risk-state identification, leading to more effective and safer secondary prevention strategies for individuals in the prodromal phases of these disorders