Computational Language Assessment in patients with speech, language, and communication impairments

Speech, language, and communication symptoms enable the early detection, diagnosis, treatment planning, and monitoring of neurocognitive disease progression. Nevertheless, traditional manual neurologic assessment, the speech and language evaluation standard, is time-consuming and resource-intensive for clinicians. We argue that Computational Language Assessment (C.L.A.) is an improvement over conventional manual neurological assessment. Using machine learning, natural language processing, and signal processing, C.L.A. provides a neuro-cognitive evaluation of speech, language, and communication in elderly and high-risk individuals for dementia. ii. facilitates the diagnosis, prognosis, and therapy efficacy in at-risk and language-impaired populations; and iii. allows easier extensibility to assess patients from a wide range of languages. Also, C.L.A. employs Artificial Intelligence models to inform theory on the relationship between language symptoms and their neural bases. It significantly advances our ability to optimize the prevention and treatment of elderly individuals with communication disorders, allowing them to age gracefully with social engagement.Comment: 36 pages, 2 figures, to be submite

Conversational affective social robots for ageing and dementia support

Socially assistive robots (SAR) hold significant potential to assist older adults and people with dementia in human engagement and clinical contexts by supporting mental health and independence at home. While SAR research has recently experienced prolific growth, long-term trust, clinical translation and patient benefit remain immature. Affective human-robot interactions are unresolved and the deployment of robots with conversational abilities is fundamental for robustness and humanrobot engagement. In this paper, we review the state of the art within the past two decades, design trends, and current applications of conversational affective SAR for ageing and dementia support. A horizon scanning of AI voice technology for healthcare, including ubiquitous smart speakers, is further introduced to address current gaps inhibiting home use. We discuss the role of user-centred approaches in the design of voice systems, including the capacity to handle communication breakdowns for effective use by target populations. We summarise the state of development in interactions using speech and natural language processing, which forms a baseline for longitudinal health monitoring and cognitive assessment. Drawing from this foundation, we identify open challenges and propose future directions to advance conversational affective social robots for: 1) user engagement, 2) deployment in real-world settings, and 3) clinical translation

Relationships between cognitive status, speech impairment and communicative participation in Parkinson’s disease

Aim: To assess the relationships between cognitive status, speech impairment and communicative participation in Parkinson’s disease. Introduction: Speech and communication difficulties, as well as cognitive impairment, are prevalent in Parkinson’s. The contributions of cognitive impairment and acoustic speech characteristics remain equivocal. Relationships between Impairment and Participation levels of the International Classification of Functioning, Disability and Health (ICF) have not been thoroughly investigated. Methods: 45 people with Parkinson’s and 29 familiar controls performed read, mood and conversational speech tasks as part of a multimethod investigation. Data analysis formed three main parts. Depression, cognition and communication were assessed using questionnaires. Phonetic analysis was used to produce an acoustic characterisation of speech. Listener assessment was used to assess conveyance of emotion and intelligibility. Qualitative Content Analysis was used to provide a participant’s insight into speech and communicative difficulties associated with Parkinson’s disease. Results: Cognitive status was significantly associated with certain read speech acoustic characteristics, emotional conveyance and communicative participation. No association was found with intelligibility or conversational speech acoustic characteristics. The only acoustic speech characteristics that predicted intelligibility were intensity and pause in the read speech condition. The contribution of intelligibility to communicative participation was modest. People with Parkinson’s disease reported a range of psychosocial, cognitive and physical factors affecting their speech and communication. Conclusions: I provide evidence for a role for cognitive status in emotional conveyance and communicative participation, but not necessarily general speech production, in Parkinson’s disease. I demonstrate that there may not be a strong relationship between ICF Impairment level speech measures and functional measures of communication. I also highlight the distinction between measures of communication at the ICF Activity and Participation levels. This study demonstrates that reduced participation in everyday communication in Parkinson’s disease appears to result from a complex interplay of physical, cognitive and psychosocial factors. Further research is required to apply these findings to contribute to future advances in speech and language therapy for Parkinson’s disease

Evaluating a technology-based reminiscence program on engagement and affect in respite aged care : time travelling with technology

With an aging population, there is greater focus in ensuring that aged care facilities are delivering high quality care. This is particularly important with the increase in aged related diseases, such as those that result in dementia. Previous research makes clear the value of meaningful activity, socialisation and engagement for wellbeing and quality of life for older adults. Reminiscence therapy (RT), is as well-established a non-pharmacological intervention, used to increase engagement in older adults. It actively involves stimulating conversation through discussion of past events and experiences. The theory behind RT is founded in person-centered care and meaningful activity. Person-centered care focuses on the needs of an individual and has an emphasis on interpersonal relationships. Through personal life events, autobiographical memories are recalled, which assist in creating a meaningful experience and connecting a person to their identity. Additionally, technological developments (such as sharing video/images) offer possible new methods for increased engagement in the RT approach. However, there is controversy in existing research as to the benefits of RT and there is limited understanding of the effect of RT when driven by digital technology. The aim of this thesis is to build on and refine previous research by conceptualizing and quantifying older adult engagement. It explores this through investigating the impact of an experimental framework Time Travelling with Technology (TTT) on the engagement of older adults in respite aged care. TTT is a dynamic, interactive and immersive, technology driven RT program, that enables older adults to travel to locations of their past and novel places of interest. The research objective of this thesis is to investigate the effect of technology driven group RT on older adult engagement. More specifically, the focus is primarily on characterising engagement of older adults in residential care. This will be achieved through a multi-dimensional approach to measuring behavioural markers as proxies of the concept of engagement. The dependent variables include facial movement, lexical use and prosodic patterns of speech. Potential covarying factors, such as cognitive capacity, will additionally be considered to further explain such relationships. The central research question addressed in this thesis is: To what extent does technology delivered through TTT impact the engagement of older adults in respite aged care

The linguistic and cognitive mechanisms underlying language tests in healthy adults : a principal component analysis

Pour un processus d’évaluation linguistique plus précis et rapide, il est important d’identifier les mécanismes cognitifs qui soutiennent des tâches langagières couramment utilisées. Une façon de mieux comprendre ses mécanismes est d’explorer la variance partagée entre les tâches linguistiques en utilisant l’analyse factorielle exploratoire. Peu d’études ont employé cette méthode pour étudier ces mécanismes dans le fonctionnement normal du langage. Par conséquent, notre objectif principal est d’explorer comment un ensemble de tâches linguistiques se regroupent afin d’étudier les mécanismes cognitifs sous-jacents de ses tâches. Nous avons évalué 201 participants en bonne santé âgés entre 18 et 75 ans (moyenne=45,29, écart-type= 15,06) et avec une scolarité entre 5 et 23 ans (moyenne=11,10, écart-type=4,68), parmi ceux-ci, 62,87% étaient des femmes. Nous avons employé deux batteries linguistiques : le Protocole d’examen linguistique de l’aphasie Montréal-Toulouse et Protocole Montréal d’Évaluation de la Communication – version abrégé. Utilisant l’analyse en composantes principales avec une rotation Direct-oblimin, nous avons découvert quatre composantes du langage : la sémantique picturale (tâches de compréhension orale, dénomination orale et dénomination écrite), l'exécutif linguistique (tâches d’évocation lexicale - critères sémantique, orthographique et libre), le transcodage et la sémantique (tâches de lecture, dictée et de jugement sémantique) et la pragmatique (tâches d'interprétation d'actes de parole indirecte et d'interprétation de métaphores). Ces quatre composantes expliquent 59,64 % de la variance totale. Deuxièmement, nous avons vérifié l'association entre ces composantes et deux mesures des fonctions exécutives dans un sous-ensemble de 33 participants. La performance de la flexibilité cognitive a été évaluée en soustrayant le - temps A au temps B du Trail Making Test et celle de la mémoire de travail en prenant le total des réponses correctes au test du n-back. La composante exécutive linguistique était associée à une meilleure flexibilité cognitive (r=-0,355) et la composante transcodage et sémantique à une meilleure performance de mémoire de travail (r=.0,397). Nos résultats confirment l’hétérogénéité des processus sous-jacent aux tâches langagières et leur relation intrinsèque avec d'autres composantes cognitives, tels que les fonctions exécutives.To a more accurate and time-efficient language assessment process, it is important to identify the cognitive mechanisms that sustain commonly used language tasks. One way to do so is to explore the shared variance across language tasks using the technique of principal components analysis. Few studies applied this technique to investigate these mechanisms in normal language functioning. Therefore, our main goal is to explore how a set of language tasks are going to group to investigate the underlying cognitive mechanisms of commonly used tasks. We assessed 201 healthy participants aged between 18 and 75 years old (mean = 45.29, SD = 15.06) and with a formal education between 5 and 23 years (mean = 11.10, SD =4.68), of these 62.87% were female. We used two language batteries: the Montreal-Toulouse language assessment battery and the Montreal Communication Evaluation Battery – brief version. Using a Principal Component Analysis with a Direct-oblimin rotation, we discovered four language components: pictorial semantics (auditory comprehension, naming and writing naming tasks), language-executive (unconstrained, semantic, and phonological verbal fluency tasks), transcoding and semantics (reading, dictation, and semantic judgment tasks), and pragmatics (indirect speech acts interpretation and metaphors interpretation tasks). These four components explained 59.64% of the total variance. Secondarily, we sought to verify the association between these components with two executive measures in a subset of 33 participants. Cognitive flexibility was assessed by the time B-time A score of the Trail Making Test and working memory by the total of correct answers on the n-back test. The language-executive component was associated with a better cognitive flexibility score (r=-.355) and the transcoding and semantics one with a better working memory performance (r=.397). Our findings confirm the heterogeneity process underlying language tasks and their intrinsic relationship to other cognitive components, such as executive functions

Big Data analytics to assess personality based on voice analysis

Trabajo Fin de Grado en Ingeniería de Tecnologías y Servicios de TelecomunicaciónWhen humans speak, the produced series of acoustic signs do not encode only the linguistic message they wish to communicate, but also several other types of information about themselves and their states that show glimpses of their personalities and can be apprehended by judgers. As there is nowadays a trend to film job candidate’s interviews, the aim of this Thesis is to explore possible correlations between speech features extracted from interviews and personality characteristics established by experts, and to try to predict in a candidate the Big Five personality traits: Conscientiousness, Agreeableness, Neuroticism, Openness to Experience and Extraversion. The features were extracted from a genuine database of 44 women video recordings acquired in 2020, and 78 in 2019 and before from a previous study. Even though many significant correlations were found for each years’ dataset, lots of them were proven to be inconsistent through both studies. Only extraversion, and openness in a more limited way, showed a good number of clear correlations. Essentially, extraversion has been found to be related to the variation in the slope of the pitch (usually at the end of sentences), which indicates that a more "singing" voice could be associated with a higher score. In addition, spectral entropy and roll-off measurements have also been found to indicate that larger changes in the spectrum (which may also be related to more "singing" voices) could be associated with greater extraversion too. Regarding predictive modelling algorithms, aimed to estimate personality traits from the speech features obtained for the study, results were observed to be very limited in terms of accuracy and RMSE, and also through scatter plots for regression models and confusion matrixes for classification evaluation. Nevertheless, various results encourage to believe that there are some predicting capabilities, and extraversion and openness also ended up being the most predictable personality traits. Better outcomes were achieved when predictions were performed based on one specific feature instead of all of them or a reduced group, as it was the case for openness when estimated through linear and logistic regression based on time over 90% of the variation range of the deltas from the entropy of the spectrum module. Extraversion too, as it correlates well with features relating variation in F0 decreasing slope and variations in the spectrum. For the predictions, several machine learning algorithms have been used, such as linear regression, logistic regression and random forests

Using fMRI to investigate speech-stream segregation and auditory attention in healthy adults and patients with memory complaints

Poor memory for recent conversations is the commonest presenting symptom in patients attending a cognitive neurology clinic. They also frequently have greater difficulty following and remembering conversations in the presence of background noise and/or unattended speech. While the ability to participate in and recall conversations depends on several cognitive functions (language-processing, attention, episodic and working memory), without the ability to perform auditory scene analysis, and more specifically speech-stream segregation, recall of verbal information will be impaired as a consequence of poor initial registration, over and above impaired encoding and subsequent retrieval. This thesis investigated auditory attention and speech-stream segregation in healthy participants (‘controls’) and patients presenting with ‘poor memory’, particularly a complaint of difficulty remembering recent verbal information. Although this resulted in the recruitment of many patients with possible or probable Alzheimer’s disease, it also included patients with mild cognitive impairment (MCI) of uncertain aetiology and a few with depression. Functional MRI data revealed brain activity involved in attention, working memory and speech-stream segregation as participants attended to a speaker in the absence and presence of background speech. The study on controls demonstrated that the right anterior insula, adjacent frontal operculum, left planum temporale and precuneus were more active when the attended speaker was partially masked by unattended speech. Analyses also revealed a central role for a right hemisphere system for successful attentive listening, a system that was not modulated by administration of a central cholinesterase inhibitor. Therefore, this study identified non-auditory higher-order regions in speech-stream segregation, and the demands on a right hemisphere system during attentive listening. Administration of a central cholinesterase inhibitor did not identify any benefit in the present patient group. However, my research has identified systems that might be therapeutic targets when attempting to modulate auditory attention and speech-stream segregation in patients with neurodegenerative disease.Open Acces

The Potential Impact of Undiagnosed Hearing Loss on the Diagnosis of Dementia

Hearing loss and dementia are conditions that impact similar populations. Many adults do not seek audiologic care for their hearing loss and thus are seen in the primary care physician’s office with an undiagnosed hearing loss. This study sought to determine the impact of undiagnosed hearing loss and thus decreased audibility on the items of the Mini Mental State Examination (MMSE) commonly used to diagnose dementia. Many physicians use the MMSE along with self-report of cognitive decline to diagnose dementia. Previous studies have suggested that self-report of cognitive decline is impacted by hearing loss. This study suggests that a decrease in audibility that would be associated with an undiagnosed hearing loss significantly impacts performance on the MMSE. Physicians should be cautious when using the MMSE and self-report of cognitive impairment to diagnose dementia without accounting for hearing loss as both may be significantly impacted by undiagnosed hearing loss

Language loss in bilingual speakers with Alzheimer's disease

PhD ThesisThis study investigated the changes in language and cognition in five bilingual speakers with Alzheimer's Disease over a period of twelve months. The pattern and rate of loss in English was compared to that of Afrikaans. The bilingual behaviour of language mixing was also investigated, as was the interaction between deteriorating cognitive skills and language functions. Data was collected at three time points (0 - 6 - 12 months) employing a battery of neuropsychological and language tests, and conversation analysis. It was predicted that where both languages were automatised to a similar extent, a similar pattern, severity and rate of loss would be evident across languages. This hypothesis was supported by results. It was also predicted that in cases where one language was less automatised than the other, the less automatised language (i.e. the language learnt later in life (L2) anchor the less proficient language) would be more severely impaired and would deteriorate at a faster rate than the fully automatised language (Li). Results revealed that while L2 was more impaired than Li for some speakers, for others, languages were similarly impaired/spared. These discrepancies were attributed to the fact that tests were not sensitive to inter-language differences near floor or ceiling. Results did not strongly support the second prediction that L2 would deteriorate at a faster rate. Ambiguous findings could be artefacts of the time window of examination, insensitive assessment tasks, and the heterogeneous nature of the population. With regards to language mixing behaviour, code switching mainly affected L2 interactions even though the extent of switching varied across speakers. The amount of language mixing increased for two participants over the year. With regards to a possible interaction between language and cognition, complex language tasks appeared to be more compromised by deteriorating neuropsychological support than less complex tasks, but the extent of this interaction varied across languages and across speakers. Finally, the overall profile of results suggested that a language learnt later in life will never become fully automatised, even if high levels of L2 proficiency had been attained in adulthood.Overseas Research Students Awar