Auditory-motor adaptation is reduced in adults who stutter but not in children who stutter

Previous studies have shown that adults who stutter produce smaller corrective motor responses to compensate for unexpected auditory perturbations in comparison to adults who do not stutter, suggesting that stuttering may be associated with deficits in integration of auditory feedback for online speech monitoring. In this study, we examined whether stuttering is also associated with deficiencies in integrating and using discrepancies between expect ed and received auditory feedback to adaptively update motor programs for accurate speech production. Using a sensorimotor adaptation paradigm, we measured adaptive speech responses to auditory formant frequency perturbations in adults and children who stutter and their matched nonstuttering controls. We found that the magnitude of the speech adaptive response for children who stutter did not differ from that of fluent children. However, the adaptation magnitude of adults who stutter in response to formant perturbation was significantly smaller than the adaptation magnitude of adults who do not stutter. Together these results indicate that stuttering is associated with deficits in integrating discrepancies between predicted and received auditory feedback to calibrate the speech production system in adults but not children. This auditory-motor integration deficit thus appears to be a compensatory effect that develops over years of stuttering

Simulating online compensation for pitch-shifted auditory feedback with the target approximation model

This study attempts to achieve modeling simulation of the well-known phenomenon of online compensation for pitch-shifted auditory feedback. We used the Target Approximation (TA) model as the underlying kinematic mechanism of pitch contour generation, and simulated feedback compensation through responsive perturbation of the height parameter of the TA model. Results show that both withinsyllable and cross-syllable pitch compensation in disyllabic utterances can be replicated. Furthermore, our data analysis also revealed an over-rectification phenomenon. By adjusting the height parameter back and beyond its original value after the compensation, the over-rectification was also replicated, further improving the overall simulation results

Learning and adaptation in speech production without a vocal tract

How is the complex audiomotor skill of speaking learned? To what extent does it depend on the specific characteristics of the vocal tract? Here, we developed a touchscreen-based speech synthesizer to examine learning of speech production independent of the vocal tract. Participants were trained to reproduce heard vowel targets by reaching to locations on the screen without visual feedback and receiving endpoint vowel sound auditory feedback that depended continuously on touch location. Participants demonstrated learning as evidenced by rapid increases in accuracy and consistency in the production of trained targets. This learning generalized to productions of novel vowel targets. Subsequent to learning, sensorimotor adaptation was observed in response to changes in the location-sound mapping. These findings suggest that participants learned adaptable sensorimotor maps allowing them to produce desired vowel sounds. These results have broad implications for understanding the acquisition of speech motor control.Published versio

An Examination of the Factors that Dictate the Relative Weighting of Feedback and Feedforward Input for Speech Motor Control

Speech is arguably the most important form of human communication. Fluent speech production relies on auditory feedback for the planning, execution, and monitoring of speech movements. Auditory feedback is particularly important during the acquisition of speech, however, it has been suggested that over time speakers rely less on auditory feedback as they develop robust sensorimotor representations that allow speech motor commands to be executed in a feedforward manner. The studies reported in this thesis recorded speaker’s vocal and neural responses to altered auditory feedback in order to explore the factors that dictate the relative importance of auditory feedback for speech motor control. More specifically, studies 1 through 3 examined how the role of auditory feedback changes throughout development, while studies 4 and 5 examined the relationship between vocal variability and auditory feedback control, and lastly study 6 looked at how the predictability of auditory feedback errors influences the role of auditory feedback for speech motor control. Results of the first study demonstrated that toddlers use auditory feedback to regulate their speech motor commands, supporting the long held notion that auditory feedback is important during the acquisition of speech. While mapping out the developmental trajectory of vocal and event related potential responses to altered auditory feedback, the second study demonstrated that vocal variability, rather than age, best predicts responses to altered auditory feedback. Importantly, this suggests that the maturation of the speech motor control system is not strictly dependent on age. The third study in this thesis demonstrated that children and adults show similar rates of sensorimotor adaptation, suggesting that once speech is acquired, speakers are proficient at using sensory information to modify the planning of future speech motor commands. However, since adults produced larger compensatory responses, these results also suggested that adults are more proficient at comparing incoming auditory feedback with the feedback predicted by their sensorimotor representations, as a result of possessing more precisely mapped sensorimotor representations. The results of studies four and five demonstrated that vocal variability can be used to predict the size of compensatory responses and sensorimotor adaptation to changes in one’s auditory feedback, respectively. Furthermore, these studies demonstrated that increased variability was related to increased auditory feedback control of speech. Finally, the sixth study in this thesis demonstrated that experimentally induced predictability and variability can be used to induce increases in feedforward and auditory feedback control, respectively. In conclusion, the results reported in this thesis demonstrate that age and vocal variability, both naturally occurring and experimentally induced, are important determinants of the role of auditory feedback in speech motor control

Fundamental frequency modelling: an articulatory perspective with target approximation and deep learning

Current statistical parametric speech synthesis (SPSS) approaches typically aim at state/frame-level acoustic modelling, which leads to a problem of frame-by-frame independence. Besides that, whichever learning technique is used, hidden Markov model (HMM), deep neural network (DNN) or recurrent neural network (RNN), the fundamental idea is to set up a direct mapping from linguistic to acoustic features. Although progress is frequently reported, this idea is questionable in terms of biological plausibility. This thesis aims at addressing the above issues by integrating dynamic mechanisms of human speech production as a core component of F0 generation and thus developing a more human-like F0 modelling paradigm. By introducing an articulatory F0 generation model – target approximation (TA) – between text and speech that controls syllable-synchronised F0 generation, contextual F0 variations are processed in two separate yet integrated stages: linguistic to motor, and motor to acoustic. With the goal of demonstrating that human speech movement can be considered as a dynamic process of target approximation and that the TA model is a valid F0 generation model to be used at the motor-to-acoustic stage, a TA-based pitch control experiment is conducted first to simulate the subtle human behaviour of online compensation for pitch-shifted auditory feedback. Then, the TA parameters are collectively controlled by linguistic features via a deep or recurrent neural network (DNN/RNN) at the linguistic-to-motor stage. We trained the systems on a Mandarin Chinese dataset consisting of both statements and questions. The TA-based systems generally outperformed the baseline systems in both objective and subjective evaluations. Furthermore, the amount of required linguistic features were reduced first to syllable level only (with DNN) and then with all positional information removed (with RNN). Fewer linguistic features as input with limited number of TA parameters as output led to less training data and lower model complexity, which in turn led to more efficient training and faster synthesis

Oromotor Kinematics of Speech In Children and the Effect of an External Rhythmic Auditory Stimulus

The purpose of this study was to determine the effect of an external auditory rhythmic stimulus on the kinematics of the oromotor musculature during speech production in children and adults. To this effect, the research questions were: 1) Do children entrain labiomandibular movements to an external auditory stimulus? 2) Does the ability to entrain labiomandibular movements to an external auditory stimulus change with age? 3) Does an external auditory stimulus change the coordination and stability of the upper lip, lower lip, and jaw when producing speech sounds? The oromotor kinematics of two groups of children, age eight to ten (n = 6) and eleven to fourteen (n = 6), were compared to the oromotor kinematics of adults (n = 12) while producing bilabial syllables with and without an external auditory stimulus. The kinematic correlates of speech production were recorded using video-based 4-dimensional motion capture technology and included measures of upper lip, lower lip and jaw displacement and their respective derivatives. The Spatiotemporal Index (a single number indication of motor stability and pattern formation) and Synchronization Error (a numerical indication of phase deviations) were calculated for each participant within each condition. There were no statistically significant differences between age groups for the Spatiotemporal Index or for Synchronization Error. Results indicated that there were statistically significant differences in the Spatiotemporal Index for condition; with Post-hoc tests indicating that the difference was between the first condition (no rhythm) and the second condition (self-paced rhythm). Results indicated that both child groups were able to synchronize to an external auditory stimulus. Furthermore, the older child group was able to establish oromotor synchrony with near-adult abilities

An Interactive Music Synthesizer for Gait Training in Neurorehabilitation

(Abstract to follow

16th Sound and Music Computing Conference SMC 2019 (28–31 May 2019, Malaga, Spain)

The 16th Sound and Music Computing Conference (SMC 2019) took place in Malaga, Spain, 28-31 May 2019 and it was organized by the Application of Information and Communication Technologies Research group (ATIC) of the University of Malaga (UMA). The SMC 2019 associated Summer School took place 25-28 May 2019. The First International Day of Women in Inclusive Engineering, Sound and Music Computing Research (WiSMC 2019) took place on 28 May 2019. The SMC 2019 TOPICS OF INTEREST included a wide selection of topics related to acoustics, psychoacoustics, music, technology for music, audio analysis, musicology, sonification, music games, machine learning, serious games, immersive audio, sound synthesis, etc