Multimodal Inductive Transfer Learning for Detection of Alzheimer's Dementia and its Severity

Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADReSS challenge dataset, which is a subject-independent and balanced dataset matched for age and gender to mitigate biases, and is available through DementiaBank. Our system achieves state-of-the-art test accuracy, precision, recall, and F1-score of 83.3% each for AD classification, and state-of-the-art test root mean squared error (RMSE) of 4.60 for MMSE score regression. To the best of our knowledge, the system further achieves state-of-the-art AD classification accuracy of 88.0% when evaluated on the full benchmark DementiaBank Pitt database. Our work highlights the applicability and transferability of spontaneous speech to produce a robust inductive transfer learning model, and demonstrates generalizability through a task-agnostic feature-space. The source code is available at https://github.com/wazeerzulfikar/alzheimers-dementiaComment: To appear in INTERSPEECH 202

Uncertainty-Aware Ensembling in Multi-Modal AI and its Applications in Digital Health for Neurodegenerative Disorders

Common neurodegenerative disorders such as Alzheimer's dementia and Parkinson's disease are increasingly recognised as leading causes of death and disability with debilitating symptoms such as progressive cognitive decline, communication breakdown, motor dysfunction and accompanying psychiatric disorders. However, factors such as unavailability of efficient and cost-effective assessments for conclusive diagnosis, time-consuming test protocols, poor prognostic capabilities, and inadequate treatment options with accompanying side effects are all barriers to progress in providing faster and more effective intervention to individuals living with these life-altering disorders. In this thesis, we take a step towards using digital health and machine learning to improve diagnostic and prognostic capabilities and to address remote care via telemedicine in Alzheimer's dementia and Parkinson's disease. Our goal is to provide more cost-effective, non-invasive, and scalable technologies for risk stratification of Alzheimer's dementia using speech. We also aim to monitor drug response and disease progression for Parkinson's disease via telemedicine, allowing real time symptom tracking through wearables alongside a patient's treatment status, which will help facilitate remote care and dynamic and adaptive treatment plans. In addition to addressing the challenges in diagnosis and treatment of neurodegenerative disorders, we further propose a novel uncertainty aware boosting technique for multi-modal ensembling and evaluate it on healthcare tasks related to Alzheimer's dementia and Parkinson's disease. This presents manifold benefits, such as reducing the overall entropy of the system, making it more robust to heteroscedasticity, and improving calibration of each of the modalities along with high quality prediction intervals.S.M

AttentivU: Evaluating the Feasibility of Biofeedback Glasses to Monitor and Improve Attention

Our everyday work is becoming increasingly complex and cognitively demanding. What we pay attention to during our day influences how effectively our brain prepares itself for action, and how much effort we apply to a task. To address this issue we present AttentivU -a system that uses wearable electroencephalography (EEG) to measure the attention of a person in realtime. When the user's attention level is low, the system provides real-time, subtle, haptic or audio feedback to nudge the person to become attentive again. We tested a first version of the system, which uses an EEG headband on 48 adults over several sessions in both a lab and classroom setting. The results show that the biofeedback redirects the attention of the participants to the task at hand and improves their performance on comprehension tests. We next tested the same approach in the form of glasses on 6 adults in a lab setting, as the glasses form factor may be more acceptable in the long run. We conclude with a discussion of an improved third version of AttentivU, currently under development, which combines a custom-made solution of the glasses form-factor with built-in electrooculography (EOG) and EEG electrodes as well as auditory feedback. Keyword: Physiological sensing; EEG; EOG; Feedback; Glasse

Multimodal inductive transfer learning for detection of Alzheimer's dementia and its severity

Copyright © 2020 ISCA Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADReSS challenge dataset, which is a subject-independent and balanced dataset matched for age and gender to mitigate biases, and is available through DementiaBank. Our system achieves state-of-the-art test accuracy, precision, recall, and F1-score of 83.3% each for AD classification, and state-of-the-art test root mean squared error (RMSE) of 4.60 for MMSE score regression. To the best of our knowledge, the system further achieves state-of-the-art AD classification accuracy of 88.0% when evaluated on the full benchmark DementiaBank Pitt database. Our work highlights the applicability and transferability of spontaneous speech to produce a robust inductive transfer learning model, and demonstrates generalizability through a task-agnostic feature-space. The source code is available at https://github.com/wazeerzulfikar/alzheimers-dementia

Robustness to Missing Features using Hierarchical Clustering with Split Neural Networks (Student Abstract)

The problem of missing data has been persistent for a long time and poses a major obstacle in machine learning and statistical data analysis. Past works in this field have tried using various data imputation techniques to fill in the missing data, or training neural networks (NNs) with the missing data. In this work, we propose a simple yet effective approach that clusters similar input features together using hierarchical clustering and then trains proportionately split neural networks with a joint loss. We evaluate this approach on a series of benchmark datasets and show promising improvements even with simple imputation techniques. We attribute this to learning through clusters of similar features in our model architecture.</jats:p