Exploiting Acoustic Feature Correlations By Joint Neural Vector Quantizer Design In A Discrete HMM System

In previous work about hybrid speech recognizers with discrete HMMs we have shown that VQs, that are trained according to an MMI criterion, are well suited for ML estimated Bayes classifiers. This is only valid for single VQ systems. In this paper we extend the theory to speech recognizers with multiple VQs. This leads to a joint training criterion for arbitrary multiple neural VQs that considers the inter VQ correlation during parameter estimation. The idea of a gradient based joint training method is derived. Experimental results indicate that inter VQ correlations can cause some degradation of recognition performance. The joint multiple VQ training decorrelates the quantizer labels and improves system performance. In addition the new training criterion allows for a less careful way of splitting up the feature vector into multiple streams that do not have to be statistically independent. In particular the usage of highly correlated features in conjunction with the novel training crit..