Toward invariant functional representations of variable surface fundamental frequency contours: Synthesizing speech melody via model-based stochastic learning

Variability has been one of the major challenges for both theoretical understanding and computer synthesis of speech prosody. In this paper we show that economical representation of variability is the key to effective modeling of prosody. Specifically, we report the development of PENTAtrainer—A trainable yet deterministic prosody synthesizer based on an articulatory–functional view of speech. We show with testing results on Thai, Mandarin and English that it is possible to achieve high-accuracy predictive synthesis of fundamental frequency contours with very small sets of parameters obtained through stochastic learning from real speech data. The first key component of this system is syllable-synchronized sequential target approximation—implemented as the qTA model, which is designed to simulate, for each tonal unit, a wide range of contextual variability with a single invariant target. The second key component is the automatic learning of function-specific targets through stochastic global optimization, guided by a layered pseudo-hierarchical functional annotation scheme, which requires the manual labeling of only the temporal domains of the functional units. The results in terms of synthesis accuracy demonstrate that effective modeling of the contextual variability is the key also to effective modeling of function-related variability. Additionally, we show that, being both theory-based and trainable (hence data-driven), computational systems like PENTAtrainer can serve as an effective modeling tool in basic research, with which the level of falsifiability in theory testing can be raised, and also a closer link between basic and applied research in speech science can be developed

Datenbasierte und linguistisch interpretierbare Intonationsmodellierung

In this thesis a data-driven and linguistically interpretable intonation model for the automatic analysis and synthesis of fundamental frequency (F0) contours was developed. The model can be characterised as parametric, contour-based, and superpositional. Its intonation representation consists of a superposition of global and local contour classes and can be derived in a purely data-driven manner, which guarantees consistency and easy adaptability to new data. The model's linguistic interpretability was examined by automatic linguistic corpus analyses resulting in hypotheses about possible relations between contours and linguistic concepts. These hypotheses were subsequently tested by perception experiments. By these means a systematic linguistic anchoring of the model was achieved in form of a decision tree to predict the linguistically appropriate contour class. The adequacy of its predictions was assured by a further perception test. Due to its simultaneous signal proximity and linguistic anchoring, the model covers the entire chain from text to signal and therefore can be used for intonation analysis and generation on a linguistic as well as on a phonetic-acoustic level. It is qualified for employment in speech technology applications as well as in phonetic fundamental research to automatically analyse raw speech data