An Auditory-Perceptual Rating of Connected Speech in Aphasia

Purpose: The goal of this study was to develop a novel tool for connected speech analysis in aphasia, so that spoken output can be characterized in a data-driven and explanatory manner. Method: We designed a multidimensional rating scheme called the Auditory-Perceptual Rating of Connected Speech in Aphasia (APROCSA), in which 27 common features were each rated on a 5-point scale. Three researchers and twelve student clinicians rated 24 connected speech samples from the AphasiaBank database. Results: Ratings conducted by both researchers and student clinicians demonstrated good-to-excellent reliability and strong concurrent validity with AphasiaBank measures derived from transcriptions, clinical measures, and subscores from the Western Aphasia Battery (WAB). Factor analysis revealed that four underlying factors—Paraphasia, Logopenia, Agrammatism, and Motor speech—accounted for 79% of the variance in the connected speech profiles. Examination of individual patient scores showed considerable diversity of factor scores among patients of any given aphasia subtype. Conclusions: The APROCSA proved to be a reliable, valid, and efficient tool for research or clinical purposes. The preliminary findings of the factor analysis suggest a parcellation of non-fluency into three distinct profiles—Logopenia, Agrammatism, and Motor speech—which may occur in conjunction with other non-fluent profiles or with the fluent profil

Computer Adaptive Testing for the Assessment of Anomia Severity

Anomia assessment is a fundamental component of clinical practice and research inquiries involving individuals with aphasia, and confrontation naming tasks are among the most commonly used tools for quantifying anomia severity. While currently available confrontation naming tests possess many ideal properties, they are ultimately limited by the overarching psychometric framework they were developed within. Here, we discuss the challenges inherent to confrontation naming tests and present a modern alternative to test development called item response theory (IRT). Key concepts of IRT approaches are reviewed in relation to their relevance to aphasiology, highlighting the ability of IRT to create flexible and efficient tests that yield precise measurements of anomia severity. Empirical evidence from our research group on the application of IRT methods to a commonly used confrontation naming test is discussed, along with future avenues for test development

Refining Semantic Similarity of Paraphasias Using a Contextual Language Model

Purpose: ParAlg (Paraphasia Algorithms) is a software that automatically categorizes a person with aphasia\u27s naming error (paraphasia) in relation to its intended target on a picture-naming test. These classifications (based on lexicality as well as semantic, phonological, and morphological similarity to the target) are important for characterizing an individual\u27s word-finding deficits or anomia. In this study, we applied a modern language model called BERT (Bidirectional Encoder Representations from Transformers) as a semantic classifier and evaluated its performance against ParAlg\u27s original word2vec model. Method: We used a set of 11,999 paraphasias produced during the Philadelphia Naming Test. We trained ParAlg with word2vec or BERT and compared their performance to humans. Finally, we evaluated BERT\u27s performance in terms of word-sense selection and conducted an item-level discrepancy analysis to identify which aspects of semantic similarity are most challenging to classify. Results: Compared with word2vec, BERT qualitatively reduced word-sense issues and quantitatively reduced semantic classification errors by almost half. A large percentage of errors were attributable to semantic ambiguity. Of the possible semantic similarity subtypes, responses that were associated with or category coordinates of the intended target were most likely to be misclassified by both models and humans alike. Conclusions: BERT outperforms word2vec as a semantic classifier, partially due to its superior handling of polysemy. This work is an important step for further establishing ParAlg as an accurate assessment tool

Paralg: A Paraphasia Algorithm for Multinomial Classification of Picture Naming Errors

A preliminary version of a paraphasia classification algorithm (henceforth called ParAlg) has previously been shown to be a viable method for coding picture naming errors. The purpose of this study is to present an updated version of ParAlg, which uses multinomial classification, and comprehensively evaluate its performance when using two different forms of transcribed input

An Open Dataset of Connected Speech in Aphasia with Consensus Ratings of Auditory-Perceptual Features

Auditory-perceptual rating of connected speech in aphasia (APROCSA) is a system in which trained listeners rate a variety of perceptual features of connected speech samples, representing the disruptions and abnormalities that commonly occur in aphasia. APROCSA has shown promise as an approach for quantifying expressive speech and language function in individuals with aphasia. The aim of this study was to acquire and share a set of audiovisual recordings of connected speech samples from a diverse group of individuals with aphasia, along with consensus ratings of APROCSA features, for future use as training materials to teach others how to use the APROCSA system. Connected speech samples were obtained from six individuals with chronic post-stroke aphasia. The first five minutes of participant speech were excerpted from each sample, and five researchers independently evaluated each sample using APROCSA, rating its 27 features on a five-point scale. The researchers then discussed each feature in turn to obtain consensus ratings. The dataset will provide a useful, freely accessible resource for researchers, clinicians, and students to learn how to evaluate aphasic speech with an auditory-perceptual approach

Leukoaraiosis Is Not Associated With Recovery From Aphasia in the First Year After Stroke.

After a stroke, individuals with aphasia often recover to a certain extent over time. This recovery process may be dependent on the health of surviving brain regions. Leukoaraiosis (white matter hyperintensities on MRI reflecting cerebral small vessel disease) is one indication of compromised brain health and is associated with cognitive and motor impairment. Previous studies have suggested that leukoaraiosis may be a clinically relevant predictor of aphasia outcomes and recovery, although findings have been inconsistent. We investigated the relationship between leukoaraiosis and aphasia in the first year after stroke. We recruited 267 patients with acute left hemispheric stroke and coincident fluid attenuated inversion recovery MRI. Patients were evaluated for aphasia within 5 days of stroke, and 174 patients presented with aphasia acutely. Of these, 84 patients were evaluated at ∼3 months post-stroke or later to assess longer-term speech and language outcomes. Multivariable regression models were fit to the data to identify any relationships between leukoaraiosis and initial aphasia severity, extent of recovery, or longer-term aphasia severity. We found that leukoaraiosis was present to varying degrees in 90% of patients. However, leukoaraiosis did not predict initial aphasia severity, aphasia recovery, or longer-term aphasia severity. The lack of any relationship between leukoaraiosis severity and aphasia recovery may reflect the anatomical distribution of cerebral small vessel disease, which is largely medial to the white matter pathways that are critical for speech and language function

IRT modeling of the VNT (Fergadiotis et al., 2023)

Purpose: Item response theory (IRT) is a modern psychometric framework with several advantageous properties as compared with classical test theory. IRT has been successfully used to model performance on anomia tests in individuals with aphasia; however, all efforts to date have focused on noun production accuracy. The purpose of this study is to evaluate whether the Verb Naming Test (VNT), a prominent test of action naming, can be successfully modeled under IRT and evaluate its reliability. Method: We used responses on the VNT from 107 individuals with chronic aphasia from AphasiaBank. Unidimensionality and local independence, two assumptions prerequisite to IRT modeling, were evaluated using factor analysis and Yen’s Q3 statistic (Yen, 1984), respectively. The assumption of equal discrimination among test items was evaluated statistically via nested model comparisons and practically by using correlations of resulting IRT-derived scores. Finally, internal consistency, marginal and empirical reliability, and conditional reliability were evaluated. Results: The VNT was found to be sufficiently unidimensional with the majority of item pairs demonstrating adequate local independence. An IRT model in which item discriminations are constrained to be equal demonstrated fit equivalent to a model in which unique discrimination parameters were estimated for each item. All forms of reliability were strong across the majority of IRT ability estimates. Conclusions: Modeling the VNT using IRT is feasible, yielding ability estimates that are both informative and reliable. Future efforts are needed to quantify the validity of the VNT under IRT and determine the extent to which it measures the same construct as other anomia tests. Supplemental Material S1. A complete participant ID list from AphasiaBank. Supplemental Material S2. A parallel analysis plot. Supplemental Material S3. Ability estimates and their standard errors. Fergadiotis, G., Casilio, M., Dickey, M. W., Steel, S., Nicholson, H., Fleegle, M., Swiderski, A., & Hula, W. D. (2023). Item response theory modeling of the Verb Naming Test. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2023_JSLHR-22-00458</p

Multivariate lesion symptom mapping for predicting trajectories of recovery from aphasia.

Individuals with post-stroke aphasia tend to recover their language to some extent; however, it remains challenging to reliably predict the nature and extent of recovery that will occur in the long term. The aim of this study was to quantitatively predict language outcomes in the first year of recovery from aphasia across multiple domains of language and at multiple timepoints post-stroke. We recruited 217 patients with aphasia following acute left hemisphere ischaemic or haemorrhagic stroke and evaluated their speech and language function using the Quick Aphasia Battery acutely and then acquired longitudinal follow-up data at up to three timepoints post-stroke: 1 month (n = 102), 3 months (n = 98) and 1 year (n = 74). We used support vector regression to predict language outcomes at each timepoint using acute clinical imaging data, demographic variables and initial aphasia severity as input. We found that ∼60% of the variance in long-term (1 year) aphasia severity could be predicted using these models, with detailed information about lesion location importantly contributing to these predictions. Predictions at the 1- and 3-month timepoints were somewhat less accurate based on lesion location alone, but reached comparable accuracy to predictions at the 1-year timepoint when initial aphasia severity was included in the models. Specific subdomains of language besides overall severity were predicted with varying but often similar degrees of accuracy. Our findings demonstrate the feasibility of using support vector regression models with leave-one-out cross-validation to make personalized predictions about long-term recovery from aphasia and provide a valuable neuroanatomical baseline upon which to build future models incorporating information beyond neuroanatomical and demographic predictors