Multimodal interfaces with voice and gesture input

The modalities of speech and gesture have different strengths and weaknesses, but combined they create synergy where each modality corrects the weaknesses of the other. We believe that a multimodal system such a one interwining speech and gesture must start from a different foundation than ones which are based solely on pen input. In order to provide a basis for the design of a speech and gesture system, we have examined the research in other disciplines such as anthropology and linguistics. The result of this investigation was a taxonomy that gave us material for the incorporation of gestures whose meanings are largely transparent to the users. This study describes the taxonomy and gives examples of applications to pen input systems

Semi-structured capture and display of telephone conversations

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1992.Includes bibliographical references (leaves 81-85).by Debby Hindus.M.S

Observations on Using Speech Input for Window Navigation

We discuss the suitability of speech recognition for navigating within a window system and we describe Xspeak, an iinpleinentation of voice control for the X Window System. We made this interface available to a number of student programmers, ancl compared the use of speech and a pointer for window navigation through empirical and observational means. Our experience indicates that speech was attractive for some users, and we coininent on their activities and recognition accuracy. These observations reveal pitfalls and advantages of using speech input in wiilclows systems