Synergy Effects in Natural Language-Based Multimodal Interaction

Abstract

A scheme for integration of a natural language interface into a multimodal environment is presented with emphasis on the synergetic results that can be achieved, which are argued to be: 1)Complementary expressiveness; 2) Mutual illumination of language syntax and semantics; 3) Robust pragmatics and graceful recovery from failed natural language analyses through the reification of discourse objects to enable user control of discourse management