Introducing Handwriting into a Multimodal LATEX Formula Editor

Handwriting has been shown to be a useful input modality for math. However, math recognizers are imperfect, especially when recognizing complex expressions. Instead of improving the recognizer itself, we explore ways to best visualize the recognizer\u27s output to help the user fix recognition mistakes more efficiently. To do this, we propose changes to the visual editing operations in MathDeck, a math-aware search engine and formula editor, as well as the addition of an n-best list of results for each symbol in the recognizer\u27s output. We present two experiments to help us find good ways to help users fix errors in the recognizer, and to test whether these changes help novices input formulas more efficiently than they would if they did not have handwriting as an input modality. In the first experiment, users had the option to fix errors with an in-place drop-down menu of alternate symbols, a side symbol correction panel, or by typing the symbols themselves or dragging them from a symbol palette. In our experiment, most users preferred to fix the errors manually by typing the correct symbols or using the symbol palette. In the second experiment, participants entered formulas using handwriting and/or LaTeX. We found evidence that suggests that novices can input formulas faster when they have access to handwriting, but experts still do better when they can just type LaTeX

Intelligently Aiding Human-Guided Correction of Speech Recognition

Correcting recognition errors is often necessary in a speech interface. These errors not only reduce users’ overall entry rate, but can also lead to frustration. While making fewer recognition errors is undoubtedly helpful, facilities for supporting user-guided correction are also critical. We explore how to better support user corrections using Parakeet – a continuous speech recognition system for mobile touch-screen devices. Parakeet’s interface is designed for easy error correction on a handheld device. Users correct errors by selecting alternative words from a word confusion network and by typing on a predictive software keyboard. Our interface design was guided by computational experiments and used a variety of information sources to aid the correction process. In user studies, participants were able to write text effectively despite sometimes high initial recognition error rates. Using Parakeet as an example, we discuss principles we think are important for building effective speech correction interfaces

Intelligently aiding human-guided correction of speech recognition

Correcting recognition errors is often necessary in a speech interface. The process of correcting errors can not only reduce users' performance, but can also lead to frustration. While making fewer recognition errors is undoubtedly helpful, facilities for supporting user-guided correction are also critical. We explore how to better support user corrections using Parakeet - a continuous speech recognition system for text entry. Parakeet's interface is designed for easy error correction on a mobile touch-screen device. Users correct errors by selecting alternative words from a word confusion network and by typing on a predictive software keyboard. Our interface design was guided by computational experiments and used a variety of information sources to aid the correction process. In user studies, participants were able to write text efficiently despite sometimes high initial recognition error rates. Using Parakeet as an example, we discuss principles we found were important for building an effective speech correction interface. Copyright © 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved