Empirical Evaluation of Interactive Multimodal Error Correction

Abstract

Recently, the first commercial dictation systems for continuous speech have become available. Although they generally received positive reviews, error correction is still limited to choosing from list of alternatives, speaking again or typing. We developed a set of multimodal interactive correction methods which allow the user to switch modality between continuous speech, spelling, handwriting and pen gestures. We integrated these correction methods with our large vocabulary speech recognition system to build a prototypical multimodal listening typewriter. We designed an experiment to empirically evaluate the efficiency of different error correction methods. The experiment compares multimodal correction with methods available in current speech recognition applications. We confirm the hypothesis that switching modality can significantly expedite corrections. However in applications where a keyboard is acceptable, typing correction remains the fastest method to correct errors for users..