Real World Approaches for Multilingual and Non-native Speech Recognition

This thesis proposes a scalable architecture for multilingual speech recognition on embedded devices. In theory multiple languages can be recognized just as one language. However, current state of the art speech recognition systems are based on statistical models with many parameters. Extending such models to multiple languages requires more resources. Therefore a lot of research in the area of multilingual speech recognition has proposed techniques to reduce this need for more resources through parameter tying across languages. After an evaluation of the previous work, this thesis was able to show that tying at the density level offers the greatest flexibility for the design of a multilingual acoustic model. Furthermore, there were also hints in the literature that densities from the native language of the speakers can be useful for the modeling of non-native accents of speakers. Based on these findings, this thesis developed an algorithm for the creation of Multilingual Weighted Codebooks (MWCs) that adds Gaussians from the spoken languages to the native language codebook ( = set of Gaussians) of the speaker. A key advantage of this algorithm is that it optimally models the native language o