Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

Background: Advances in machine learning (ML) technology have opened new avenues for detection and monitoring of cognitive decline. In this study, a multimodal approach to Alzheimer's dementia detection based on the patient's spontaneous speech is presented. This approach was tested on a standard, publicly available Alzheimer's speech dataset for comparability. The data comprise voice samples from 156 participants (1:1 ratio of Alzheimer's to control), matched by age and gender. Materials and Methods: A recently developed Active Data Representation (ADR) technique for voice processing was employed as a framework for fusion of acoustic and textual features at sentence and word level. Temporal aspects of textual features were investigated in conjunction with acoustic features in order to shed light on the temporal interplay between paralinguistic (acoustic) and linguistic (textual) aspects of Alzheimer's speech. Combinations between several configurations of ADR features and more traditional bag-of-n-grams approaches were used in an ensemble of classifiers built and evaluated on a standardised dataset containing recorded speech of scene descriptions and textual transcripts. Results: Employing only semantic bag-of-n-grams features, an accuracy of 89.58% was achieved in distinguishing between Alzheimer's patients and healthy controls. Adding temporal and structural information by combining bag-of-n-grams features with ADR audio/textual features, the accuracy could be improved to 91.67% on the test set. An accuracy of 93.75% was achieved through late fusion of the three best feature configurations, which corresponds to a 4.7% improvement over the best result reported in the literature for this dataset. Conclusion: The proposed combination of ADR audio and textual features is capable of successfully modelling temporal aspects of the data. The machine learning approach toward dementia detection achieves best performance when ADR features are combined with strong semantic bag-of-n-grams features. This combination leads to state-of-the-art performance on the AD classification task

The Linguistic and Cultural Aspects of Neuropsychological Assessment in People with Dementia

Objectives: Given the public health crisis that Alzheimer's disease (AD) has become (Naylor et al., 2012),neuropsychological assessment tools that provide timely and accurate identification of cognitive decline in older adults have gained increasing focus in the scientific literature. Accurate evaluation of cognitive function and early identification of cognitive changes are paramount to understanding the disease course of AD and improving effective treatments and patients' quality of life. To this end, language offers a cognitive neuropsychological approach to identifying cognitive decline in the early stages of AD. Moreover, it represents a multi-dimensional variable that may influence the neuropsychological test performance of older adults due to its potential contribution to cognitive reserve. Therefore, the present thesis aims at combining two aspects of language to explore its potential in the early detection of AD and its association with neuropsychological test performance in older adults and cross-cultural neuropsychology. Study 1 assessed the currently available studies to explore whether discourse processing, particularly macro-structural discourse comprehension, offers a novel approach to neuropsychological testing in distinguishing normal cognitive aging from AD pathology-related decline. Study 2 evaluated the results of the studies that examined the impact of bilingualism on neuropsychological test performance in monolingual and bilingual older adults to inform the neuropsychological evaluation of these groups in clinical practice. Study 3 investigated the influence of bilingualism and its associated factors, namely, cultural background and acculturation, on cognitive screening tests in three clinically diagnosed AD patient groups to identify a cross-culturally/linguistically appropriate measure of cognition. Method: Data of Study 1 and Study 2 were based on the original research studies published in English investigating discourse comprehension and bilingualism in healthy older adults, individuals with mild cognitive impairment (MCI), and AD. A literature search focusing on these topics with participant groups aged 60 years and over was conducted in PubMed, Web of Science, and PsycINFO databases. Study 1 included eight articles consisting of studies only with cross sectional designs. Study 2 was comprised of twenty-seven articles, of which sixteen articles had cross-sectional designs. On the other hand, Study 3 was original research based on a cross sectional design targeting culturally/linguistically diverse patients diagnosed with AD. Specifically, the study sample consisted of Turkish immigrant (n=21) and monolingual, non-immigrant German (n=20) and Turkish (n=24) patients with AD. All participants were administered the Mini-Mental State Examination (MMSE), Rowland Universal Dementia Assessment Scale (RUDAS), a dementia severity rating scale, and a self-report measure of depression. Additionally, self-report measures of bilingualism and acculturation were conducted with Turkish-immigrant participants with AD. Results: Study 1 revealed that people with AD and MCI have significant deficits in discourse comprehension, which are not observed in cognitively normal older adults of any age. On five of six discourse comprehension measures, groups with AD were significantly worse than healthy older adults, with one measure yielding mixed findings. Furthermore, compared to the cognitively healthy groups, individuals with MCI showed significant performance deficits in discourse comprehension measures similar to those with AD. Study 2 indicated better performance for bilingual older adults on executive function tests when compared to their monolingual counterparts. On the other hand, bilinguals were found to perform poorer than monolinguals on tests assessing the language domain. However, these findings did not remain robust when the impact of bilingualism on test performance was investigated longitudinally. Lastly, Study 3 provided further evidence on the linguistic and educational bias of the MMSE when employed in culturally and linguistically diverse individuals with AD. Bilingualism was linked to better performance on the MMSE in the Turkish immigrant group. German patients with AD obtained higher scores on this test than the other two groups. Furthermore, RUDAS was shown to be a better alternative for assessing global cognition in German and Turkish individuals with AD. Conclusion: The macro-structural discourse comprehension assessment paradigm has shown promising results in identifying the preclinical stages of AD. Further research on this paradigm may help develop a diagnostic tool with a clinical value that can be utilized for differential diagnosis, predicting conversion from MCI to dementia in research and clinical settings. On the other hand, another aspect of linguistic skills, namely, the evaluation of research on the link between bilingualism and neuropsychological test performance, did not provide definitive answers to the question of bilingual advantages and disadvantages addressed in the second study due to methodological challenges in the field. However, it identified a comprehensive and critical list of clinically and empirically relevant bilingualism-associated variables which may guide future research and neuropsychological practice. In light of the Study 2 findings, Study 3 filled an important gap in the literature by exploring cultural, demographic, and immigration related factors that may influence neuropsychological testing experiences in Germany. The study findings may help the field of cross-cultural neuropsychology serve culturally and linguistically diverse populations more efficiently. Overall, the present thesis contributed to the literature by highlighting the importance and potential of linguistic abilities in the clinical diagnosis and neuropsychological evaluation of individuals with dementia

Executive function & semantic memory impairments in Alzheimer’s disease — investigating the decline of executive function and semantic memory in Alzheimer’s disease through computer-supported qualitative analysis of semantic verbal fluency and its applications in clinical decision support

Alzheimer’s Disease (AD) has a huge impact on an ever-aging society in highly developed industrialized countries such as the EU member states: according to the World Alzheimer’s Association the number one risk factor for AD is age. AD patients suffer from neurodegenerative processes driving cognitive decline which eventually results in the loss of patients’ ability of independent living. Episodic memory impairment is the most prominent cognitive symptom of AD in its clinical stage. In addition, also executive function and semantic memory impairments significantly affect activities of daily living and are discussed as important cognitive symptoms during prodromal as well as acute clinical stages of AD. Most of the research on semantic memory impairments in AD draws evidence from the Semantic Verbal Fluency (SVF) task which evidentially also places high demands on the executive function level. At the same time, the SVF is one of the most-applied routine assessments in clinical neuropsychology especially in the diagnosis of AD. Therefore, the SVF is a prime task to study semantic memory and executive function impairment side-by-side and draw conclusions about their parallel or successive impairments across the clinical trajectory of AD. To effectively investigate semantic memory and executive function processes in the SVF, novel computational measures have been proposed that tap into data-driven semantic as well as temporal metrics scoring an SVF performance on the item-level. With a better and more differentiated understanding of AD-related executive function and semantic memory impairments in the SVF, the SVF can grow from a well-established screening into a more precise diagnostic tool for early AD. As the SVF is one of the most-applied easy-to-use and low-burden neurocognitive assessments in AD, such advancements have a direct impact on clinical practice as well. For the last decades huge efforts have been put on the discovery of disease-modifying compounds responding to specific AD biomarker-related cognitive decline characteristics. However, as most pharmaceutical trials failed, the focus has shifted towards population-wide early screening with cost-effective and scalable cognitive tests representing an effective mid-term strategy. Computer-supported SVF analysis responds to this demand. This thesis pursues a two-fold objective: (1) improve our understanding of the progressive executive function and semantic memory impairments and their interplay in clinical AD as measured by the SVF and (2) harness those insights for applied early and specific AD screening. To achieve both objectives, this thesis comprises work on subjects from different clinical stages of AD (Healthy Aging, amnestic Mild Cognitive Impairment—aMCI, and AD dementia) and in different languages (German & French). All results are based on SVF speech data generated either as a one-time assessment or a repeated within-participant testing. From these SVF speech samples, qualitative markers are extracted with different amount of computational support (ranging from manual processing of speech to fully automated evaluation). The results indicate, that semantic memory is structurally affected from an early clinical—amnestic Mild Cognitive Impairment (aMCI)—stage on and is even more affected in the later acute dementia stage. The semantic memory impairment in AD is particularly worsened through the patients’ inability to compensate by engaging executive functions. Hence, over the course of the disease, hampered executive functioning and therefore the inability to compensate for corrupt semantic memory structures might be the main driver of later-stage AD patients’ notably poor cognitive performance. These insights generated on the SVF alone are only made possible through computer-supported qualitative analysis on an item-per-item level which leads the way towards potential applications in clinical decision support. The more fine-grained qualitative analysis of the SVF is clinically valuable for AD diagnosis and screening but very time-consuming if performed manually. This thesis shows though that automatic analysis pipelines can reliably and validly generate this diagnostic information from the SVF. Automatic transcription of speech plus automatic extraction of the novel qualitative SVF features result in clinical interpretation comparable to manual transcripts and improved diagnostic decision support simulated through machine learning classification experiments. This indicates that the computer-supported SVF could ultimately be used for cost-effective fully automated early clinical AD screening. This thesis advances current AD research in a two-fold manner. First it improves the understanding of the decline of executive function and semantic memory in AD as measured through computational qualitative analysis of the SVF. Secondly, this thesis embeds these theoretical advances into practical clinical decision support concepts that help screen population-wide and cost-effective for early-stage AD.Die Alzheimer-Krankheit (AD) stellt eine enorme Herausforderung für die immer älter werdende Gesellschaft in hochentwickelten Industrieländern wie den EU-Mitgliedsstaaten dar. Nach Angaben der World Alzheimer's Association ist der größte Risikofaktor für AD das Alter. Alzheimer-Patienten leiden unter neurodegenerativen Prozessen, die kognitiven Abbau verursachen und schließlich dazu führen, dass Patienten nicht länger selbstbestimmt leben können. Die Beeinträchtigung des episodischen Gedächtnisses ist das prominenteste kognitive Symptom von AD im klinischen Stadium. Darüber hinaus führen auch Störungen der Exekutivfunktionen sowie der semantischen Gedächtnisleistung zu erheblichen Einschränkungen bei Aktivitäten des täglichen Lebens und werden als wichtige kognitive Symptome sowohl im Prodromal- als auch im akuten klinischen Stadium von AD diskutiert. Der Großteil der Forschung zu semantischen Gedächtnisbeeinträchtigungen bei AD stützt sich auf Ergebnisse aus dem Semantic Verbal Fluency Tests (SVF), der auch die Exekutivfunktionen stark fordert. In der Praxis ist die SVF eines der am häufigsten eingesetzten Routine- Assessments in der klinischen Neuropsychologie, insbesondere bei der Diagnose von AD. Daher ist die SVF eine erstklassige Aufgabe, um die Beeinträchtigung des semantischen Gedächtnisses und der exekutiven Funktionen Seite an Seite zu untersuchen und Rückschlüsse auf ihre parallelen oder sukzessiven Beeinträchtigungen im klinischen Verlauf von AD zu ziehen. Um semantische Gedächtnis- und Exekutivfunktionsprozesse in der SVF effektiv zu untersuchen, wurden jüngst neuartige computergestützte Verfahren vorgeschlagen, die sowohl datengetriebene semantische als auch temporäre Maße nutzen, die eine SVF-Leistung auf Item-Ebene bewerten. Mit einem besseren und differenzierteren Verständnis von ADbedingten Beeinträchtigungen der Exekutivfunktionen und des semantischen Gedächtnisses in der SVF kann sich die SVF von einem gut etablierten Screening zu einem präziseren Diagnoseinstrument für frühe AD entwickeln. Da die SVF eines der am häufigsten angewandten, einfach zu handhabenden und wenig belastenden neurokognitiven Assessments bei AD ist, haben solche Fortschritte auch einen direkten Einfluss auf die klinische Praxis. In den letzten Jahrzehnten wurden enorme Anstrengungen unternommen, um krankheitsmodifizierende Substanzen zu finden, die auf spezifische, mit AD-Biomarkern verbundene Merkmale des kognitiven Abbaus reagieren. Da jedoch die meisten pharmazeutischen Studien in jüngster Vergangenheit fehlgeschlagen sind, wird heute als mittelfristige Strategie bevölkerungsweite Früherkennung mit kostengünstigen und skalierbaren kognitiven Tests gefordert. Die computergestützte SVF-Analyse ist eine Antwort auf diese Forderung. Diese Arbeit verfolgt deshalb zwei Ziele: (1) Verbesserung des Verständnisses der fortschreitenden Beeinträchtigungen der Exekutivfunktionen und des semantischen Gedächtnisses und ihres Zusammenspiels bei klinischer AD, gemessen durch die SVF, und (2) Nutzung dieser Erkenntnisse für angewandte AD-Früherkennung. Um beide Ziele zu erreichen, umfasst diese Thesis Forschung mit Probanden aus verschiedenen klinischen AD Stadien (gesundes Altern, amnestisches Mild Cognitive Impairment-aMCI, und AD-Demenz) und in verschiedenen Sprachen (Deutsch & Französisch). Alle Ergebnisse basieren auf SVF Sprachdaten, erhoben im Querschnittdesign oder als wiederholte Testung in einem Längsschnittdesign. Aus diesen SVF-Sprachproben werden mit unterschiedlicher rechnerischer Unterstützung qualitative Marker extrahiert (von manueller Verarbeitung der Sprache bis hin zu vollautomatischer Auswertung). Die Ergebnisse zeigen, dass das semantische Gedächtnis bereits im frühen aMCI Stadium strukturell beeinträchtigt ist und im späteren akuten Demenzstadium noch stärker betroffen ist. Die strukturelle Beeinträchtigung des semantischen Gedächtnisses bei Alzheimer wird insbesondere dadurch verschlimmert, dass die Patienten nicht in der Lage sind, dies durch den Einsatz exekutiver Funktionen zu kompensieren. Daher könnten im Verlauf der Erkrankung eingeschränkte Exekutivfunktionen und damit die Unfähigkeit, degenerierte semantische Gedächtnisstrukturen zu kompensieren, die Hauptursache für die auffallend schlechten kognitiven Leistungen von AD-Patienten im Akutstadium sein. Diese Erkenntnisse basierend auf der SVF alleine werden erst durch die computergestützte qualitative Analyse auf Item-per-Item-Ebene möglich und weisen den Weg zu möglichen Anwendungen in der klinischen Entscheidungsunterstützung. Die feinkörnigere qualitative Analyse der SVF ist klinisch wertvoll für die AD-Diagnose und das Screening, aber sehr zeitaufwändig, wenn sie manuell durchgeführt wird. Diese Arbeit zeigt jedoch, dass automatische Analysepipelines diese diagnostischen Informationen zuverlässig und valide aus der SVF generieren können. Die automatische Transkription von Sprache plus die automatische Extraktion der neuartigen qualitativen SVF-Merkmale führen zu einer klinischen Interpretation, die mit manuellen Analysen vergleichbar ist. Diese Verarbeitung führt auch zu einer verbesserten diagnostischen Entscheidungsunterstützung, die durch Klassifikationsexperimente mit maschinellem Lernen simuliert wurde. Dies deutet darauf hin, dass die computergestützte SVF letztendlich für ein kostengünstiges vollautomatisches klinisches AD-Frühscreening eingesetzt werden könnte. Diese Arbeit bringt die aktuelle AD-Forschung auf zweifache Weise voran. Erstens verbessert sie unser Verständnis der kognitiven Einschränkungen im Bereich der Exekutivfunktionen und des semantischen Gedächtnisses bei AD, gemessen durch die computergestützte qualitative Analyse der SVF. Zweitens bettet diese Arbeit diese theoretischen Fortschritte in ein praktisches Konzept zur klinischen Entscheidungsunterstützung ein, das zukünftig ein bevölkerungsweites und kosteneffektives Screening für AD im Frühstadium ermöglichen könnte

Marqueurs discursifs de neurodégénérescence liée à la pathologie Alzheimer

La maladie d’Alzheimer (MA) et les aphasies progressives primaires (APP) s’accompagnent de perturbations du langage expressif parfois subtiles, mais précoces dans l’évolution de ces maladies neurodégénératives. Considérés dans une approche automatisée, ces changements pourraient constituer des marqueurs de dégénérescence identifiés de façon non invasive et peu onéreuse. À ce titre, ils font l’objet d’études visant à automatiser leur utilisation clinique. Cependant, l’intégration des marqueurs langagiers à une approche diagnostique centrée sur les biomarqueurs reste à faire. À cette fin, la présente thèse a deux objectifs. D’abord, recenser systématiquement les marqueurs du discours qui distinguent le mieux les personnes avec une MA de témoins en santé. Ensuite, appliquer une approche automatisée et à un large éventail de marqueurs de discours pour identifier, dans un groupe hétérogène de patients avec une APP, lesquels ont une pathologie Alzheimer sous-jacente. Afin de mettre en contexte ces deux objectifs, nous proposons une introduction générale comprenant les éléments suivants : la pathophysiologie de la MA et des APP, le rôle croissant des biomarqueurs dans la prise de décision clinique dans les maladies neurodégénératives, les études pionnières du discours en neurodégénérescence, ainsi que de récentes études computationnelles sur les marqueurs de discours dans la MA et les APP. Nos résultats font émerger un patron multidimensionnel (acoustique, lexical, syntaxique, sémantique et pragmatique) de changements langagiers qui distinguent les personnes avec une MA de témoins en santé, avec une prépondérance des marqueurs lexicosémantiques. Dans le groupe de patients avec une APP avec une imagerie amyloïde positive ou négative, nous mesurons ensuite le pouvoir de classification d’un court échantillon de discours et montrons qu’il peut être avantageusement comparé à d’autres biomarqueurs. Nous discutons du patron spécifique de marqueurs discriminants pour ce sous-groupe de patients, notamment l’importance des marqueurs psycholinguistiques pour prédire le résultat de l’imagerie amyloïde à partir du discours.Alzheimer’s disease (AD) and primary progressive aphasias (PPA) feature changes in expressive language that appear early in the course of the disease. Within an automated analysis framework, these language changes could offer a non-invasive and inexpensive alternative to the collection of biomarkers which are not readily available in most settings. Current research is thus focused on the automated analysis of language data for clinical use. The usefulness of connected speech (CS) markers has not yet been established in a diagnostic perspective focused on biomarkers. To this aim, the present thesis contains two phases. First, we systematically review the CS markers that best differentiate persons with AD from healthy controls. Second, we automatically extract a wide array of CS markers in a heterogenous group of PPA patients by combining expert knowledge and the latest natural language processing software. A machine-learning classification approach identifies PPA patients for the presence of underlying AD pathology. The most discriminant CS features are identified. To integrate the two phases of the thesis, we provide a general introduction with the following sections: the pathophysiology of AD and PPAs, the growing importance of biomarkers in clinical decision-making for neurodegenerative diseases, the seminal studies of CS in neurodegenerative diseases, and the latest computational studies of CS markers in AD and PPA. Our results bring forth a multidimensional pattern (acoustic, lexical, syntactic, semantic, pragmatic) of language changes that distinguish people with AD from healthy controls, with an emphasis on lexical-semantic features. In the group of PPA patients with either positive or negative amyloid imaging, we then describe the classificatory power of a short sample of CS and show that it compares favorably to other biomarkers. We discuss the specific pattern of discriminant markers for this subgroup of patients, in particular the role of psycholinguistics