A lightly supervised approach to detect stuttering in children's speech

© 2018 International Speech Communication Association. All rights reserved. In speech pathology, new assistive technologies using ASR and machine learning approaches are being developed for detecting speech disorder events. Classically-trained ASR model tends to remove disfluencies from spoken utterances, due to its focus on producing clean and readable text output. However, diagnostic systems need to be able to track speech disfluencies, such as stuttering events, in order to determine the severity level of stuttering. To achieve this, ASR systems must be adapted to recognise full verbatim utterances, including pseudo-words and non-meaningful part-words. This work proposes a training regime to address this problem, and preserve a full verbatim output of stuttering speech. We use a lightly-supervised approach using task-oriented lattices to recognise the stuttering speech of children performing a standard reading task. This approach improved the WER by 27.8% relative to a baseline that uses word-lattices generated from the original prompt. The improved results preserved 63% of stuttering events (including sound, word, part-word and phrase repetition, and revision). This work also proposes a separate correction layer on top of the ASR that detects prolongation events (which are poorly recog-nised by the ASR). This increases the percentage of preserved stuttering events to 70%

The Effect of Attention to Self-Regulation of Speech Sound Productions on Speech Fluency in Oral Reading

Purpose: This study ultimately sought to test whether a condition of heightened attention to speech sound production during connected speech serves to trigger increased disfluencies. Disfluencies, or disruptions in the flow of speech, are highly variable in form and location, both within and across individuals and situations. Research to identify conditions that can predictably trigger disfluencies has the potential to provide insight into their elusive nature. A review of related literature covered the cognitive-linguistic theories related to speech fluency and stuttering. This review of previous literature also served as the foundation for why it was proposed that disfluencies would be triggered by heightened self-monitoring attention to how speech sounds are made during connected speech. Methods: Participants included 10 male and 10 female normally fluent adult college students. Their tasks included a baseline oral reading of a 330-word passage, learning of two new speech sounds, followed by an experimental reading of the same passage again. During the experimental reading, target sounds, which were indicated by highlighted locations within the passage, had to be replaced with the newly learned speech sounds. Participants indicated much greater attention was given to how speech sounds were produced during the experimental oral reading than in the baseline oral reading, to support and validate the nature of the task. Results: Disfluencies and oral reading rates were examined using descriptive statistics and analyzed by means of the negative binomial distribution model. Secondary analyses of oral reading rates were conducted with the Wilcoxon’s Signed Rank test. The results revealed that the experimental reading task was associated with a significant increase in Stuttering-Like Disfluency (SLD) and Other Disfluency (OD), and a significant decrease in oral reading rate. Furthermore, SLDs increased significantly more than ODs from the first to the second reading. Discussion: Results supported the hypothesis that disfluency, especially SLD, can be triggered by a condition of increased attention to self-monitoring how speech sounds are produced during connected speech. These findings support theories explaining disfluencies as a symptom of a speaker’s cognitive-linguistic speech planning processes being over-burdened. Implications are raised for specific populations that may be at risk-for more disfluencies: young children learning language, second-language learners, and children in speech therapy. Future research directions are recommended to better understand how to prevent disfluencies in at-risk populations and clarify the enigmatic relationship among attentional processes, phonological production planning, and stuttering

Acoustic analysis in stuttering: a machine-learning study

BackgroundStuttering is a childhood-onset neurodevelopmental disorder affecting speech fluency. The diagnosis and clinical management of stuttering is currently based on perceptual examination and clinical scales. Standardized techniques for acoustic analysis have prompted promising results for the objective assessment of dysfluency in people with stuttering (PWS).ObjectiveWe assessed objectively and automatically voice in stuttering, through artificial intelligence (i.e., the support vector machine – SVM classifier). We also investigated the age-related changes affecting voice in stutterers, and verified the relevance of specific speech tasks for the objective and automatic assessment of stuttering.MethodsFifty-three PWS (20 children, 33 younger adults) and 71 age−/gender-matched controls (31 children, 40 younger adults) were recruited. Clinical data were assessed through clinical scales. The voluntary and sustained emission of a vowel and two sentences were recorded through smartphones. Audio samples were analyzed using a dedicated machine-learning algorithm, the SVM to compare PWS and controls, both children and younger adults. The receiver operating characteristic (ROC) curves were calculated for a description of the accuracy, for all comparisons. The likelihood ratio (LR), was calculated for each PWS during all speech tasks, for clinical-instrumental correlations, by using an artificial neural network (ANN).ResultsAcoustic analysis based on machine-learning algorithm objectively and automatically discriminated between the overall cohort of PWS and controls with high accuracy (88%). Also, physiologic ageing crucially influenced stuttering as demonstrated by the high accuracy (92%) of machine-learning analysis when classifying children and younger adults PWS. The diagnostic accuracies achieved by machine-learning analysis were comparable for each speech task. The significant clinical-instrumental correlations between LRs and clinical scales supported the biological plausibility of our findings.ConclusionAcoustic analysis based on artificial intelligence (SVM) represents a reliable tool for the objective and automatic recognition of stuttering and its relationship with physiologic ageing. The accuracy of the automatic classification is high and independent of the speech task. Machine-learning analysis would help clinicians in the objective diagnosis and clinical management of stuttering. The digital collection of audio samples here achieved through smartphones would promote the future application of the technique in a telemedicine context (home environment)

History of the Department of Speech Pathology and Audiology at the University of North Dakota

The Department, of Speech Pathology and Audiology at the University of North Dakota has been in existence since 1 February 1967 The purpose of this study was to accumulate a complete history of the department from the time a program in speech correction was begun as a part of the Department of Speech in the late 1940\u27s through the spring semester 1978. This thesis begins with a statement of purpose and a brief description of the present status of the department in Chapter I. The first section of Chapter II includes information about the early history of speech pathology in North Dakota. The next two sections of this chapter are a chronological progression of the beginning and development of speech correction from 1948 through 1966. The final section of Chapter II describes the separation of speech pathology and audiology from the Department of Speech. The next six chapters of this thesis are broken down into specific topic areas. Chapter III includes information pertaining to faculty— persons who were employed for each academic year from 1967-68 through 1977-78— and their publications and papers, national offices, and salaries. Chapter IV includes information about undergraduate and graduate students, and Chapter V deals with undergraduate and graduate courses. Information about the Speech and Hearing Clinic is contained in Chapter VI, and Chapter VII describes the physical facilities of the department. Significant historical developments such as the program planning, departmental examination, the evaluation of students involved in practicum, departmental bylaws, a curriculum conference and questionnaire to evaluate the department, a thesis to describe former graduate students and their evaluation of the program, student teaching,’ departmental accreditation, and a ten-year planning report are included in Chapter VIII. Chapter IX consists of a summary and conclusions drawn from the information gathered

Automatic Screening of Childhood Speech Sound Disorders and Detection of Associated Pronunciation Errors

Speech disorders in children can affect their fluency and intelligibility. Delay in their diagnosis and treatment increases the risk of social impairment and learning disabilities. With the significant shortage of Speech and Language Pathologists (SLPs), there is an increasing interest in Computer-Aided Speech Therapy tools with automatic detection and diagnosis capability. However, the scarcity and unreliable annotation of disordered child speech corpora along with the high acoustic variations in the child speech data has impeded the development of reliable automatic detection and diagnosis of childhood speech sound disorders. Therefore, this thesis investigates two types of detection systems that can be achieved with minimum dependency on annotated mispronounced speech data. First, a novel approach that adopts paralinguistic features which represent the prosodic, spectral, and voice quality characteristics of the speech was proposed to perform segment- and subject-level classification of Typically Developing (TD) and Speech Sound Disordered (SSD) child speech using a binary Support Vector Machine (SVM) classifier. As paralinguistic features are both language- and content-independent, they can be extracted from an unannotated speech signal. Second, a novel Mispronunciation Detection and Diagnosis (MDD) approach was introduced to detect the pronunciation errors made due to SSDs and provide low-level diagnostic information that can be used in constructing formative feedback and a detailed diagnostic report. Unlike existing MDD methods where detection and diagnosis are performed at the phoneme level, the proposed method achieved MDD at the speech attribute level, namely the manners and places of articulations. The speech attribute features describe the involved articulators and their interactions when making a speech sound allowing a low-level description of the pronunciation error to be provided. Two novel methods to model speech attributes are further proposed in this thesis, a frame-based (phoneme-alignment) method leveraging the Multi-Task Learning (MTL) criterion and training a separate model for each attribute, and an alignment-free jointly-learnt method based on the Connectionist Temporal Classification (CTC) sequence to sequence criterion. The proposed techniques have been evaluated using standard and publicly accessible adult and child speech corpora, while the MDD method has been validated using L2 speech corpora

Factors in the identification and treatment of stuttering.

A large number of children with a diagnosis of stuttering will recover, often without formal treatment. This recovery pattern highlights the importance of a clear, early diagnosis and has implications for therapeutic practice. This thesis investigated three factors that could assist speech and language therapists in their diagnosis and treatment of children who stutter (CWS). Those factors were social, motor and speech skills. A pilot study investigating a fourth factor, communication attitude, is reported as an appendix. All factors were investigated from the perspective of the EXPLAN model of fluency failure. EXPLAN suggests that a combination of speech timing and phonological difficulty is an important source of fluency failures. The investigation into the social skills of CWS indicated that there is a trend for CWS to hold a lower social position to that of age matched controls. CWS were more likely to be bullied at school than their peers. The relationship between stuttering severity and social status was not significant. The motor skills study, using a battery of tests of cerebellar function (Dow & Moruzzi, 1958), indicated that CWS showed a deficit in performance on balance/posture tests at a young age and on complex movement tasks at teenage when compared to age matched controls. These differences are discussed with relation to auditory and cerebellar function. The fluency of a group of CWS was examined using phonological word analysis (Au-Yeung & Howell, 1998). Five children were producing predominantly part- word repetitions at initial assessment. Four of these children had persisted in their stutter when followed up three years later. Results suggest that information regarding motor skills and linguistic analysis of speech may be useful in the diagnosis and treatment of CWS. The results of the experimental work are discussed with relation to their theoretical and clinical significance

A comparison of stuttering behavior and fluency improvement in english-mandarin bilinguals who stutter

Despite the number of bilinguals and speakers of English and Mandarin worldwide, up till now there have been no investigations of stuttering in any of the Chinese languages, or in bilinguals who speak both English and Mandarin. Hence, it is not known whether stuttering behavior in Mandarin mimics that in English, or whether speech restructuring techniques such as Prolonged Speech produce the same fluency outcomes in Mandarin speakers as they do for English speakers. Research into stuttering in bilinguals is available but far from adequate. Although the limited extant studies show that bilinguals who stutter (BWS) may stutter either the same or differently across languages, and that treatment effects in one language can automatically carry over to the other language, it is unclear whether these findings are influenced by factors such as language dominance or language structure. These issues need to be clarified because speech language pathologists (SLPs) who work with bilinguals often do not speak the dominant language of their clients. Thus, the language of assessment and treatment becomes an important clinical consideration. The aim of this thesis was to investigate (a) whether the severity and type of stuttering was different in English and Mandarin in English-Mandarin bilingual adults, (b) whether this difference was influenced by language dominance, (c) whether stuttering reductions in English generalized to Mandarin following treatment in English only, and (d) whether treatment generalization was influenced by language dominance. To achieve these aims, a way of establishing the dominant language in bilinguals was a necessary first step. The first part of this thesis reviews the disorder of stuttering and the treatment for adults who stutter, the differences between English and Chinese languages, and stuttering in bilinguals. Part Two of this thesis describes the development of a tool for determining language dominance in a multilingual Asian population such as that found in Singapore. This study reviews the complex issues involved in assessing language dominance. It presents the rationale for and description of a self-report classification tool for identifying the dominant language in English-Mandarin bilingual Singaporeans. The decision regarding language dominance was based on a predetermined set of criteria using self-report questionnaire data on language proficiency, frequency of language use, and domain of language use. The tool was administered to 168 English-Mandarin bilingual participants, and the self-report data were validated against the results of a discriminant analysis. The discriminant analysis revealed a reliable three-way classification into English-dominant, Mandarin-dominant, and balanced bilinguals. Scores on a single word receptive vocabulary test supported these dominance classifications. Part Three of this thesis contains two studies investigating stuttering in BWS. The second study of this thesis examined the influence of language dominance on the manifestation of stuttering in English-Mandarin BWS. Results are presented for 30 English-Mandarin BWS who were divided according to their bilingual classification group: 15 English-dominant, four Mandarin-dominant, and 11 balanced bilinguals. All participants underwent comprehensive speech evaluations in both languages. The English-dominant and Mandarin-dominant BWS were found to exhibit greater stuttering in their less dominant language, whereas the balanced bilinguals evidenced similar levels of stuttering in both languages. An analysis of the types of stutter using the Lidcombe Behavioral Data Language showed no significant differences between English and Mandarin for all bilingual groups. In the third study of this thesis, the influence of language dominance on the generalization of stuttering reductions from English to Mandarin was investigated. Results are provided for seven English-dominant, three Mandarin-dominant, and four balanced bilinguals who underwent a Smooth Speech intensive program in English only. A comparison of stuttering between their pretreatment scores and three posttreatment interval scores indicated that the degree of fluency transfer from the treated to the untreated language was disproportionate. English-dominant and Mandarin-dominant participants showed greater fluency improvement in their dominant language even if this language was not directly treated. In the final chapter, Part Four, a hypothesis is provided to explain the findings of this thesis. A discussion of the limitations of the thesis and suggestions for future research are also presented. The chapter concludes with a summary of the main contributions that this thesis makes to the field of stuttering in bilinguals

A comparison of stuttering behavior and fluency improvement in english-mandarin bilinguals who stutter

Despite the number of bilinguals and speakers of English and Mandarin worldwide, up till now there have been no investigations of stuttering in any of the Chinese languages, or in bilinguals who speak both English and Mandarin. Hence, it is not known whether stuttering behavior in Mandarin mimics that in English, or whether speech restructuring techniques such as Prolonged Speech produce the same fluency outcomes in Mandarin speakers as they do for English speakers. Research into stuttering in bilinguals is available but far from adequate. Although the limited extant studies show that bilinguals who stutter (BWS) may stutter either the same or differently across languages, and that treatment effects in one language can automatically carry over to the other language, it is unclear whether these findings are influenced by factors such as language dominance or language structure. These issues need to be clarified because speech language pathologists (SLPs) who work with bilinguals often do not speak the dominant language of their clients. Thus, the language of assessment and treatment becomes an important clinical consideration. The aim of this thesis was to investigate (a) whether the severity and type of stuttering was different in English and Mandarin in English-Mandarin bilingual adults, (b) whether this difference was influenced by language dominance, (c) whether stuttering reductions in English generalized to Mandarin following treatment in English only, and (d) whether treatment generalization was influenced by language dominance. To achieve these aims, a way of establishing the dominant language in bilinguals was a necessary first step. The first part of this thesis reviews the disorder of stuttering and the treatment for adults who stutter, the differences between English and Chinese languages, and stuttering in bilinguals. Part Two of this thesis describes the development of a tool for determining language dominance in a multilingual Asian population such as that found in Singapore. This study reviews the complex issues involved in assessing language dominance. It presents the rationale for and description of a self-report classification tool for identifying the dominant language in English-Mandarin bilingual Singaporeans. The decision regarding language dominance was based on a predetermined set of criteria using self-report questionnaire data on language proficiency, frequency of language use, and domain of language use. The tool was administered to 168 English-Mandarin bilingual participants, and the self-report data were validated against the results of a discriminant analysis. The discriminant analysis revealed a reliable three-way classification into English-dominant, Mandarin-dominant, and balanced bilinguals. Scores on a single word receptive vocabulary test supported these dominance classifications. Part Three of this thesis contains two studies investigating stuttering in BWS. The second study of this thesis examined the influence of language dominance on the manifestation of stuttering in English-Mandarin BWS. Results are presented for 30 English-Mandarin BWS who were divided according to their bilingual classification group: 15 English-dominant, four Mandarin-dominant, and 11 balanced bilinguals. All participants underwent comprehensive speech evaluations in both languages. The English-dominant and Mandarin-dominant BWS were found to exhibit greater stuttering in their less dominant language, whereas the balanced bilinguals evidenced similar levels of stuttering in both languages. An analysis of the types of stutter using the Lidcombe Behavioral Data Language showed no significant differences between English and Mandarin for all bilingual groups. In the third study of this thesis, the influence of language dominance on the generalization of stuttering reductions from English to Mandarin was investigated. Results are provided for seven English-dominant, three Mandarin-dominant, and four balanced bilinguals who underwent a Smooth Speech intensive program in English only. A comparison of stuttering between their pretreatment scores and three posttreatment interval scores indicated that the degree of fluency transfer from the treated to the untreated language was disproportionate. English-dominant and Mandarin-dominant participants showed greater fluency improvement in their dominant language even if this language was not directly treated. In the final chapter, Part Four, a hypothesis is provided to explain the findings of this thesis. A discussion of the limitations of the thesis and suggestions for future research are also presented. The chapter concludes with a summary of the main contributions that this thesis makes to the field of stuttering in bilinguals