VISUAL FEATURES FOR MULTIMODAL SPEECH RECOGNITION

Abstract

This paper demonstrates the use of visual parameters extracted from video for automatic recognition of phoneme strings. Encouraged by previous works utilizing ”visually clean” data we investigate their efficiency in non-ideal conditions which are introduced by meeting audio-visual data employed in our experiments.