Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0

Stuttering is a varied speech disorder that harms an individual's communication ability. Persons who stutter (PWS) often use speech therapy to cope with their condition. Improving speech recognition systems for people with such non-typical speech or tracking the effectiveness of speech therapy would require systems that can detect dysfluencies while at the same time being able to detect speech techniques acquired in therapy. This paper shows that fine-tuning wav2vec 2.0 [1] for the classification of stuttering on a sizeable English corpus containing stuttered speech, in conjunction with multi-task learning, boosts the effectiveness of the general-purpose wav2vec 2.0 features for detecting stuttering in speech; both within and across languages. We evaluate our method on FluencyBank , [2] and the German therapy-centric Kassel State of Fluency (KSoF) [3] dataset by training Support Vector Machine classifiers using features extracted from the finetuned models for six different stuttering-related event types: blocks, prolongations, sound repetitions, word repetitions, interjections, and - specific to therapy - speech modifications. Using embeddings from the fine-tuned models leads to relative classification performance gains up to 27% w.r.t. F1-score.Comment: Accepted at Interspeech 202

Pre-schoolers who stutter score lower in verbal skills than their non-stuttering peers

Purpose: The study aimed at the examination of a link between stuttering and verbal skills (speech comprehension, articulation, grammar, vocabulary, and phonological short-term memory) in three- to five-year-old children.Method: Two samples with a total of 7,217 unselected German children were tested with the validated speech and language test Marburger Sprachscreening – revised version (MSSrev).  Linguistic domains were compared for pre-school children who stuttered (CWS; n=110) and those who did not (CWNS; n=7,107) by means of Mann-Whitney U tests, general linear models, Spearman correlations, and cross-tables.  Results: In both samples, CWS scored lower in grammar, articulation, and overall performance on the MSSrev. Statistically significant associations between stuttering and (a) sex of the child, and (b) language disorders in the family were identified.Conclusions: Taking into account the effect sizes, there appears to be a weak, but statistically significant link between stuttering and verbal skills.

Effectiveness of Stuttering Modification Treatment in School-Age Children Who Stutter:A Randomized Clinical Trial

PURPOSE: This study investigated the effectiveness of the stuttering modification intervention Kinder Dürfen Stottern (KIDS) in school-age children who stutter. METHOD: Seventy-three children who stutter were included in this multicenter, two-group parallel, randomized, wait-list controlled trial with a follow-up of 12 months. Children aged 7-11 years were recruited from 34 centers for speech therapy and randomized to either the immediate-treatment group or the 3 months delayed-treatment group. KIDS was provided by 26 clinicians who followed a treatment manual. Although the primary outcome measure was the impact of stuttering (Overall Assessment of the Speaker's Experience of Stuttering-School-Age [OASES-S]), the secondary outcomes included objective and subjective data on stuttering severity. RESULTS: At 3 months postrandomization, the mean score changes of the OASES-S differed significantly between the experimental (n = 33) and control group (n = 29; p = .026). Furthermore, treatment outcomes up to 12 months were analyzed (n = 59), indicating large effects of time on the OASES-S score (p &lt; .001, partial η2= .324). This was paralleled by significant improvements in parental ratings and objective ratings (stuttering severity, frequency, and physical concomitants). CONCLUSIONS: The significant short-term treatment effects in the OASES-S are in line with the (initial) focus of KIDS on cognitive and affective aspects of stuttering. Over 12 months, these changes were maintained and accompanied by behavioral improvements. The results suggest that individual treatment with KIDS is an adequate treatment option for this age group. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.24207864.</p

The effects of imitation and synchronization on the pronunciation of selected phonemes in L2 English and German: a pilot study

ISBN 978-0-646-80069-1International audienceThis study focuses on investigating possible effects of synchronous speech, an experimental version of joint speech [3], on L2 pronunciation at the segmental level. While repetition and imitation are traditionally used in pronunciation teaching and learning of L2 phonetic and phonological acquisition, synchronous speech has been seldom studied in an L2 learning environment. What are the L2 linguistic aspects that synchronous speech would influence? Is there any effect of L2 phonetic convergence found when learners are speaking and listening at the same time? We studied the effects of synchronization and imitation on the acquisition of eight phonemes in L2 English and German that are known to be problematic for French learners. A series of acoustic analysis revealed that while some acoustic parameters such as formant frequencies and vocalic duration improved in both speech practices, changes brought on by synchronous speech of consonantal acoustic parameters were more subtle to determine

A Comparison of Hybrid and End-to-End Models for Syllable Recognition

This paper presents a comparison of a traditional hybrid speech recognition system (kaldi using WFST and TDNN with lattice-free MMI) and a lexicon-free end-to-end (TensorFlow implementation of multi-layer LSTM with CTC training) models for German syllable recognition on the Verbmobil corpus. The results show that explicitly modeling prior knowledge is still valuable in building recognition systems. With a strong language model (LM) based on syllables, the structured approach significantly outperforms the end-to-end model. The best word error rate (WER) regarding syllables was achieved using kaldi with a 4-gram LM, modeling all syllables observed in the training set. It achieved 10.0% WER w.r.t. the syllables, compared to the end-to-end approach where the best WER was 27.53%. The work presented here has implications for building future recognition systems that operate independent of a large vocabulary, as typically used in a tasks such as recognition of syllabic or agglutinative languages, out-of-vocabulary techniques, keyword search indexing and medical speech processing.Comment: 22th International Conference of Text, Speech and Dialogue TSD201

Speech motor profiles in primary progressive aphasia

Purpose: Previous research on motor speech disorders (MSDs) in primary progressive aphasia (PPA) has largely focused on patients with the nonfluent/agrammatic variant of PPA (nfvPPA), with few systematic descriptions of MSDs in variants other than nfvPPA. There has also been an emphasis on studying apraxia of speech, whereas less is known about dysarthria or other forms of MSDs. This study aimed to examine the qualitative and quantitative characteristics of MSDs in a prospective sample of individuals with PPA independent of subtype. Method: We included 38 participants with a root diagnosis of PPA according to current consensus criteria, including one case with primary progressive apraxia of speech. Speech tasks comprised various speech modalities and levels of complexity. Expert raters used a novel protocol for auditory speech analyses covering all major dimensions of speech. Results: Of the participants, 47.4% presented with some form of MSD. Individual speech motor profiles varied widely with respect to the different speech dimensions. Besides apraxia of speech, we observed different dysarthria syndromes, special forms of MSDs (e.g., neurogenic stuttering), and mixed forms. Degrees of severity ranged from mild to severe. We also observed MSDs in patients whose speech and language profiles were incompatible with nfvPPA. Conclusions: The results confirm that MSDs are common in PPA and can manifest in different syndromes. The findings emphasize that future studies of MSDs in PPA should be extended to all clinical variants and should take into account the qualitative characteristics of motor speech dysfunction across speech dimensions

Arabic Fluency Assessment: Procedures for Assessing Stuttering in Arabic Preschool Children

The primary aim of this thesis was to screen school-aged (4+) children for two separate types of fluency issues and to distinguish both groups from fluent children. The two fluency issues are Word-Finding Difficulty (WFD) and other speech disfluencies (primarily stuttering). The cohort examined consisted of children who spoke Arabic and English. We first designed a phonological assessment procedure that can equitably test Arabic and English children, called the Arabic English non-word repetition task (AEN_NWR). Riley’s Stuttering Severity Instrument (SSI) is the standard way of assessing fluency for speakers of English. There is no standardized version of SSI for Arabic speakers. Hence, we designed a scheme to measure disfluency symptoms in Arabic speech (Arabic fluency assessment). The scheme recognizes that Arabic and English differ at all language levels (lexically, phonologically and syntactically). After the children with WFD had been separated from those with stuttering, our second aim was to develop and deliver appropriate interventions for the different cohorts. Specifically, we aimed to develop treatments for the children with WFD using short procedures that are suitable for conducting in schools. Children who stutter are referred to SLTs to receive the appropriate type of intervention. To treat WFD, another set of non-word materials was designed to include phonemic patterns not used in the speaker’s native language that are required if that speaker uses another targeted language (e.g. phonemic patterns that occur in English, but not Arabic). The goal was to use these materials in an intervention to train phonemic sequences that are not used in the child’s additional language such as the phonemic patterns that occur in English, but not Arabic. The hypothesis is that a native Arabic speaker learning English would be expected to struggle on those phonotactic patterns not used in Arabic that are required for English. In addition to the screening and intervention protocols designed, self-report procedures are desirable to assess speech fluency when time for testing is limited. To that end, the last chapter discussed the importance of designing a fluency questionnaire that can assess fluency in the entire population of speakers. Together with the AEN_NWR, the brief self-report instrument forms a package of assessment procedures that facilitate screening of speech disfluencies in Arabic children (aged 4+) when they first enter school. The seven chapters, described in more detail below, together constitute a package that achieves the aims of identifying speech problems in children using Arabic and/or English and offering intervention to treat WFD

The ACM Multimedia 2022 Computational Paralinguistics Challenge: vocalisations, stuttering, activity, & mosquitoes

The ACM Multimedia 2022 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the Vocalisations and Stuttering Sub-Challenges, a classification on human non-verbal vocalisations and speech has to be made; the Activity Sub-Challenge aims at beyond-audio human activity recognition from smartwatch sensor data; and in the Mosquitoes Sub-Challenge, mosquitoes need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' ComParE and BoAW features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectrum toolkit; in addition, we add end-to-end sequential modelling, and a log-mel-128-BNN

A comparison of stuttering behavior and fluency improvement in english-mandarin bilinguals who stutter

Despite the number of bilinguals and speakers of English and Mandarin worldwide, up till now there have been no investigations of stuttering in any of the Chinese languages, or in bilinguals who speak both English and Mandarin. Hence, it is not known whether stuttering behavior in Mandarin mimics that in English, or whether speech restructuring techniques such as Prolonged Speech produce the same fluency outcomes in Mandarin speakers as they do for English speakers. Research into stuttering in bilinguals is available but far from adequate. Although the limited extant studies show that bilinguals who stutter (BWS) may stutter either the same or differently across languages, and that treatment effects in one language can automatically carry over to the other language, it is unclear whether these findings are influenced by factors such as language dominance or language structure. These issues need to be clarified because speech language pathologists (SLPs) who work with bilinguals often do not speak the dominant language of their clients. Thus, the language of assessment and treatment becomes an important clinical consideration. The aim of this thesis was to investigate (a) whether the severity and type of stuttering was different in English and Mandarin in English-Mandarin bilingual adults, (b) whether this difference was influenced by language dominance, (c) whether stuttering reductions in English generalized to Mandarin following treatment in English only, and (d) whether treatment generalization was influenced by language dominance. To achieve these aims, a way of establishing the dominant language in bilinguals was a necessary first step. The first part of this thesis reviews the disorder of stuttering and the treatment for adults who stutter, the differences between English and Chinese languages, and stuttering in bilinguals. Part Two of this thesis describes the development of a tool for determining language dominance in a multilingual Asian population such as that found in Singapore. This study reviews the complex issues involved in assessing language dominance. It presents the rationale for and description of a self-report classification tool for identifying the dominant language in English-Mandarin bilingual Singaporeans. The decision regarding language dominance was based on a predetermined set of criteria using self-report questionnaire data on language proficiency, frequency of language use, and domain of language use. The tool was administered to 168 English-Mandarin bilingual participants, and the self-report data were validated against the results of a discriminant analysis. The discriminant analysis revealed a reliable three-way classification into English-dominant, Mandarin-dominant, and balanced bilinguals. Scores on a single word receptive vocabulary test supported these dominance classifications. Part Three of this thesis contains two studies investigating stuttering in BWS. The second study of this thesis examined the influence of language dominance on the manifestation of stuttering in English-Mandarin BWS. Results are presented for 30 English-Mandarin BWS who were divided according to their bilingual classification group: 15 English-dominant, four Mandarin-dominant, and 11 balanced bilinguals. All participants underwent comprehensive speech evaluations in both languages. The English-dominant and Mandarin-dominant BWS were found to exhibit greater stuttering in their less dominant language, whereas the balanced bilinguals evidenced similar levels of stuttering in both languages. An analysis of the types of stutter using the Lidcombe Behavioral Data Language showed no significant differences between English and Mandarin for all bilingual groups. In the third study of this thesis, the influence of language dominance on the generalization of stuttering reductions from English to Mandarin was investigated. Results are provided for seven English-dominant, three Mandarin-dominant, and four balanced bilinguals who underwent a Smooth Speech intensive program in English only. A comparison of stuttering between their pretreatment scores and three posttreatment interval scores indicated that the degree of fluency transfer from the treated to the untreated language was disproportionate. English-dominant and Mandarin-dominant participants showed greater fluency improvement in their dominant language even if this language was not directly treated. In the final chapter, Part Four, a hypothesis is provided to explain the findings of this thesis. A discussion of the limitations of the thesis and suggestions for future research are also presented. The chapter concludes with a summary of the main contributions that this thesis makes to the field of stuttering in bilinguals

A comparison of stuttering behavior and fluency improvement in english-mandarin bilinguals who stutter

Despite the number of bilinguals and speakers of English and Mandarin worldwide, up till now there have been no investigations of stuttering in any of the Chinese languages, or in bilinguals who speak both English and Mandarin. Hence, it is not known whether stuttering behavior in Mandarin mimics that in English, or whether speech restructuring techniques such as Prolonged Speech produce the same fluency outcomes in Mandarin speakers as they do for English speakers. Research into stuttering in bilinguals is available but far from adequate. Although the limited extant studies show that bilinguals who stutter (BWS) may stutter either the same or differently across languages, and that treatment effects in one language can automatically carry over to the other language, it is unclear whether these findings are influenced by factors such as language dominance or language structure. These issues need to be clarified because speech language pathologists (SLPs) who work with bilinguals often do not speak the dominant language of their clients. Thus, the language of assessment and treatment becomes an important clinical consideration. The aim of this thesis was to investigate (a) whether the severity and type of stuttering was different in English and Mandarin in English-Mandarin bilingual adults, (b) whether this difference was influenced by language dominance, (c) whether stuttering reductions in English generalized to Mandarin following treatment in English only, and (d) whether treatment generalization was influenced by language dominance. To achieve these aims, a way of establishing the dominant language in bilinguals was a necessary first step. The first part of this thesis reviews the disorder of stuttering and the treatment for adults who stutter, the differences between English and Chinese languages, and stuttering in bilinguals. Part Two of this thesis describes the development of a tool for determining language dominance in a multilingual Asian population such as that found in Singapore. This study reviews the complex issues involved in assessing language dominance. It presents the rationale for and description of a self-report classification tool for identifying the dominant language in English-Mandarin bilingual Singaporeans. The decision regarding language dominance was based on a predetermined set of criteria using self-report questionnaire data on language proficiency, frequency of language use, and domain of language use. The tool was administered to 168 English-Mandarin bilingual participants, and the self-report data were validated against the results of a discriminant analysis. The discriminant analysis revealed a reliable three-way classification into English-dominant, Mandarin-dominant, and balanced bilinguals. Scores on a single word receptive vocabulary test supported these dominance classifications. Part Three of this thesis contains two studies investigating stuttering in BWS. The second study of this thesis examined the influence of language dominance on the manifestation of stuttering in English-Mandarin BWS. Results are presented for 30 English-Mandarin BWS who were divided according to their bilingual classification group: 15 English-dominant, four Mandarin-dominant, and 11 balanced bilinguals. All participants underwent comprehensive speech evaluations in both languages. The English-dominant and Mandarin-dominant BWS were found to exhibit greater stuttering in their less dominant language, whereas the balanced bilinguals evidenced similar levels of stuttering in both languages. An analysis of the types of stutter using the Lidcombe Behavioral Data Language showed no significant differences between English and Mandarin for all bilingual groups. In the third study of this thesis, the influence of language dominance on the generalization of stuttering reductions from English to Mandarin was investigated. Results are provided for seven English-dominant, three Mandarin-dominant, and four balanced bilinguals who underwent a Smooth Speech intensive program in English only. A comparison of stuttering between their pretreatment scores and three posttreatment interval scores indicated that the degree of fluency transfer from the treated to the untreated language was disproportionate. English-dominant and Mandarin-dominant participants showed greater fluency improvement in their dominant language even if this language was not directly treated. In the final chapter, Part Four, a hypothesis is provided to explain the findings of this thesis. A discussion of the limitations of the thesis and suggestions for future research are also presented. The chapter concludes with a summary of the main contributions that this thesis makes to the field of stuttering in bilinguals