Submitted to Communications of the ACM Version 9 (7/3/99) Embodied User Interfaces for Really Direct Manipulation

A major event in the history of human-computer interaction (HCI) was the advent at Xerox PARC in the 1970s of the Graphical User Interface (GUI). The GUI was based on a bitmapped display, making the interface to the computer primarily a visual medium, along with a pointing device (typically a mouse), enabling the user to point to parts of the display. The GUI transformed HCI from communication in an arcane textual language to a visual and manipulative activity. With the GUI, the virtual world inside the computer is portrayed graphically on the display. This graphical world is a metaphoric representation of artifacts in the office workplace, such as documents and folders. Icons represent these artifacts on the display, where they can be manipulated with the mouse, such as dragging a document icon into a folder icon to “file ” it. An artifact can also be “opened up ” so that its content can be seen and manipulated, such as by scrolling or by jumping between pages of a long document. The GUI paradigm, recognized as a breakthrough, was labeled “direct manipulation ” by Shneiderman [6] in 1983. It is interesting to note that while a whole sensory-motor world is created within th