An evaluation of automatic speech recognition in the Spanish version of windows 7: effects of language variety, speaking style and gender

This study consists in an evaluation of the Spanish version of the automatic speech recognizer embedded in what is currently one of the most widespread operating systems: Microsoft’s Windows 7. Emphasis is placed upon the effects of gender, language variety and speaking style on system performance. Two groups of subjects were included in the tests: one of them was composed of 20 speakers of a Peninsular variety (Spanish as spoken in Catalonia) and the second one, of 20 speakers of a Latin American variety (Spanish as spoken in Buenos Aires), 10 female and 10 male speakers within each group. The test set consisted of three different tasks aimed at evaluating command recognition as well as automatic dictation. These tasks were carried out in one-to-one meetings with each of the selected subjects. Results revealed higher error rates for the group of Latin American speakers in comparison to Peninsular speakers. Word error rate (WER) in the dictation tasks was 28.2% for the former group and 23.1% for the latter. Regarding the task on commands, 88% of these were correctly recognized for the Peninsular group, whereas the group from Buenos Aires obtained a recognition percentage of 82.5%. With respect to speaking style, the system performed worse for speech exhibiting a higher degree of spontaneity and informality (WER = 30.7%) than for semi-scripted speech on relatively formal topics (WER = 22.8%). In contrast, results corresponding to the speech of men and women only showed slight differences which in general did not prove significant. For male speakers, 86.5% of the commands were correctly recognized, compared to 84% for female speakers, and WER for the automatic dictation tasks was 24.9% for the former group and 26.6% for the latter