Keyboardless Visual Programming Using Voice, Handwriting, and Gesture

Visual programming languages have facilitated the application development process, improving our ability to express programs, as well as our ability to view, edit and interact with them. Yet even in programming environments, productivity is restricted by the primary input sources: the mouse and the keyboard. As an alternative, we investigate a program development interface which responds to the most natural human communication technologies: voice, handwriting and gesture. Speech- and pen-based systems have yet to find broad acceptance in everyday life because they are insufficiently advantageous to overcome problems with reliability. However, we believe that a visual programming environment with a multimodal user interface properly constrained so as not to exceed the limits of the current technology has the potential to increase programming productivity for not only those people who are manually or visually impaired, but for the general population as well. In this paper we report on such a system

Requirements for Graphical Front Ends for Visual Languages

Although great progress has been made in the development of visual languages, little attention has been paid to how diagrams in such languages should be entered into computers. This issue will have a great impact, however, on users' acceptance of visual languages. We present an analysis of the features of visual languages influencing diagram entry, and also the results of an experiment comparing users' performance on different types of diagram editors. This data is then used to suggest design guidelines for usable visual language front ends. 1: Introduction Visual notations for programs have been in existence virtually from the invention of programming, and visual languages that can be interpreted by computers have been in existence for the last 15-20 years. Since that time, the visual notations have become more complex as language designers have become more aware of the power of visual notations and as available technology has become more sophisticated. In contrast to these advances,..