Automatic design of multimodal presentations

We describe our attempt to integrate multiple AI components such as planning, knowledge representation, natural language generation, and graphics generation into a functioning prototype called WIP that plans and coordinates multimodal presentations in which all material is generated by the system. WIP allows the generation of alternate presentations of the same content taking into account various contextual factors such as the user\u27s degree of expertise and preferences for a particular output medium or mode. The current prototype of WIP generates multimodal explanations and instructions for assembling, using, maintaining or repairing physical devices. This paper introduces the task, the functionality and the architecture of the WIP system. We show that in WIP the design of a multimodal document is viewed as a non-monotonic process that includes various revisions of preliminary results, massive replanning and plan repairs, and many negotiations between design and realization components in order to achieve an optimal division of work between text and graphics. We describe how the plan-based approach to presentation design can be exploited so that graphics generation influences the production of text and vice versa. Finally, we discuss the generation of cross-modal expressions that establish referential relationships between text and graphics elements

Plan-based integration of natural language and graphics generation

Multimodal interfaces combining natural language and graphics take advantage of both the individual strength of each communication mode and the fact that several modes can be employed in parallel. The central claim of this paper is that the generation of a multimodal presentation system WIP which allows the generation of alternate presentations of the same content taking into account various contextual factors. We discuss how the plan-based approach to presentation design can be exploited so that graphics generation influences the production of text and vice versa. We show that well-known concepts from the area of natural language processing like speech acts, anaphora, and rhetorical relations take on an extended meaning in the context of multimodal communication. Finally, we discuss two detailed examples illustrating and reinforcing our theoretical claims

DFKI Workshop on Natural Language Generation

On the Saarbrücken campus sites as well as at DFKI, many research activities are pursued in the field of Natural Language Generation (NLG). We felt that too little is known about the total of these activities and decided to organize a workshop in order to share ideas and promote the results. This DFKI workshop brought together local researchers working on NLG. Several papers are co-authored by international researchers. Although not all NLG activities are covered in the present document, the papers reviewed for this workshop clearly demonstrate that Saarbrücken counts among the important NLG sites in the world

DFKI Workshop on Natural Language Generation

On the Saarbrücken campus sites as well as at DFKI, many research activities are pursued in the field of Natural Language Generation (NLG). We felt that too little is known about the total of these activities and decided to organize a workshop in order to share ideas and promote the results. This DFKI workshop brought together local researchers working on NLG. Several papers are co-authored by international researchers. Although not all NLG activities are covered in the present document, the papers reviewed for this workshop clearly demonstrate that Saarbrücken counts among the important NLG sites in the world