Unifying Amplitude and Phase Analysis: A Compositional Data Approach to Functional Multivariate Mixed-Effects Modeling of Mandarin Chinese

Mandarin Chinese is characterized by being a tonal language; the pitch (or $F_0$) of its utterances carries considerable linguistic information. However, speech samples from different individuals are subject to changes in amplitude and phase which must be accounted for in any analysis which attempts to provide a linguistically meaningful description of the language. A joint model for amplitude, phase and duration is presented which combines elements from Functional Data Analysis, Compositional Data Analysis and Linear Mixed Effects Models. By decomposing functions via a functional principal component analysis, and connecting registration functions to compositional data analysis, a joint multivariate mixed effect model can be formulated which gives insights into the relationship between the different modes of variation as well as their dependence on linguistic and non-linguistic covariates. The model is applied to the COSPRO-1 data set, a comprehensive database of spoken Taiwanese Mandarin, containing approximately 50 thousand phonetically diverse sample $F_0$ contours (syllables), and reveals that phonetic information is jointly carried by both amplitude and phase variation.Comment: 49 pages, 13 figures, small changes to discussio

Characterizing intonation deficit in motor speech disorders : an autosegmental-metrical analysis of spontaneous speech in hypokinetic dysarthria, ataxic dysarthria and foreign accent syndrome

The autosegmental-metrical (AM) framework represents an established methodology for intonational analysis in unimpaired speaker populations but has found little application in describing intonation in motor speech disorders (MSDs). This study compared the intonation patterns of unimpaired participants (CON) and those with Parkinson's disease (PD), ataxic dysarthria (AT), and foreign accent syndrome (FAS) to evaluate the approach's potential for distinguishing types of MSDs from each other and from unimpaired speech. Spontaneous speech from 8 PD, 8 AT, 4 FAS, and 10 CON speakers were analyzed in relation to inventory and prevalence of pitch patterns, accentuation, and phrasing. Acoustic-phonetic baseline measures (maximum-phonation-duration, speech rate, and F0-variability) were also performed. Results: The analyses yielded differences between MSD and CON groups and between the clinical groups in regard to prevalence, accentuation, and phrasing. AT and FAS speakers used more rising and high pitch accents than PD and CON speakers. The AT group used the highest number of pitch accents per phrase, and all 3 MSD groups produced significantly shorter phrases than the CON group. The study succeeded in differentiating MSDs on the basis of intonational performances by using the AM approach, thus, demonstrating its potential for charting intonational profiles in clinical populations

Improved status following behavioural intervention in a case of severe dysarthria with stroke aetiology

There is little published intervention outcome literature concerning dysarthria acquired from stroke. Single case studies have the potential to provide more detailed specification and interpretation than is generally possible with larger participant numbers and are thus informative for clinicians who may deal with similar cases. Such research also contributes to the future planning of larger scale investigations. Behavioural intervention is described which was carried out with a man with severe dysarthria following stroke, beginning at seven and ending at nine months after stroke. Pre-intervention stability between five and seven months contrasted with significant improvements post-intervention on listener-rated measures of word and reading intelligibility and communication effectiveness in conversation. A range of speech analyses were undertaken (comprising of rate, pause and intonation characteristics in connected speech and phonetic transcription of single word production), with the aim of identifying components of speech which might explain the listeners’ perceptions of improvement. Pre- and post intervention changes could be detected mainly in parameters related to utterance segmentation and intonation. The basis of improvement in dysarthria following intervention is complex, both in terms of the active therapeutic dimensions and also the specific speech alterations which account for changes to intelligibility and effectiveness. Single case results are not necessarily generalisable to other cases and outcomes may be affected by participant factors and therapeutic variables, which are not readily controllable

Automatic prosodic analysis for computer aided pronunciation teaching

Correct pronunciation of spoken language requires the appropriate modulation of acoustic characteristics of speech to convey linguistic information at a suprasegmental level. Such prosodic modulation is a key aspect of spoken language and is an important component of foreign language learning, for purposes of both comprehension and intelligibility. Computer aided pronunciation teaching involves automatic analysis of the speech of a non-native talker in order to provide a diagnosis of the learner's performance in comparison with the speech of a native talker. This thesis describes research undertaken to automatically analyse the prosodic aspects of speech for computer aided pronunciation teaching. It is necessary to describe the suprasegmental composition of a learner's speech in order to characterise significant deviations from a native-like prosody, and to offer some kind of corrective diagnosis. Phonological theories of prosody aim to describe the suprasegmental composition of speech..

A Particle Module for the PLUTO code: II - Hybrid Framework for Modeling Non-thermal emission from Relativistic Magnetized flows

We describe a new hybrid framework to model non-thermal spectral signatures from highly energetic particles embedded in a large-scale classical or relativistic MHD flow. Our method makes use of \textit{Lagrangian} particles moving through an Eulerian grid where the (relativistic) MHD equations are solved concurrently. Lagrangian particles follow fluid streamlines and represent ensembles of (real) relativistic particles with a finite energy distribution. The spectral distribution of each particle is updated in time by solving the relativistic cosmic ray transport equation based on local fluid conditions. This enables us to account for a number of physical processes, such as adiabatic expansion, synchrotron and inverse Compton emission. An accurate semi-analytically numerical scheme that combines the method of characteristics with a Lagrangian discretization in the energy coordinate is described. In presence of (relativistic) magnetized shocks, a novel approach to consistently model particle energization due to diffusive shock acceleration has been presented. Our approach relies on a refined shock-detection algorithm and updates the particle energy distribution based on the shock compression ratio, magnetic field orientation and amount of (parameterized) turbulence. The evolved distribution from each \textit{Lagrangian} particle is further used to produce observational signatures like emission maps and polarization signals accounting for proper relativistic corrections. We further demonstrate the validity of this hybrid framework using standard numerical benchmarks and evaluate the applicability of such a tool to study high energy emission from extra-galactic jets.Comment: 23 pages, 14 figures, Accepted for publication in The Astrophysical Journa

Continuous Interaction with a Virtual Human

Attentive Speaking and Active Listening require that a Virtual Human be capable of simultaneous perception/interpretation and production of communicative behavior. A Virtual Human should be able to signal its attitude and attention while it is listening to its interaction partner, and be able to attend to its interaction partner while it is speaking – and modify its communicative behavior on-the-fly based on what it perceives from its partner. This report presents the results of a four week summer project that was part of eNTERFACE’10. The project resulted in progress on several aspects of continuous interaction such as scheduling and interrupting multimodal behavior, automatic classification of listener responses, generation of response eliciting behavior, and models for appropriate reactions to listener responses. A pilot user study was conducted with ten participants. In addition, the project yielded a number of deliverables that are released for public access