Generating multimedia briefings: coordinating language and illustration

AbstractCommunication can be more effective when several media (such as text, speech, or graphics) are integrated and coordinated to present information. This changes the nature of media-specific generation (e.g., language or graphics generation), which must take into account the multimedia context in which it occurs. This paper presents work on coordinating and integrating speech, text, static and animated three-dimensional graphics, and stored images, as part of several systems we have developed at Columbia University. A particular focus of our work has been on the generation of presentations that brief a user on information of interes

A Task-Centered Visualization Design Environment and a Method for Measuring the Complexity of Visualization Designs

Recent years have seen a growing interest in the emerging area of computer security visualization which is about developing visualization methods to help solve computer security problems. In this thesis, we will first present a method for measuring the complexity of information visualization designs. The complexity is measured in terms of visual integration, number of separable dimensions for each visual unit, the complexity of interpreting the visual attributes, number of visual units, and the efficiency of visual search. This method is designed to better assist fellow developers to quickly evaluate multiple design choices, potentially enables computer to automatically measure the complexity of visualization data. We will also analyze the design space of network security visualization. Our main contribution is a new taxonomy that consists of three dimensions – data, visualizations, and tasks. Each dimension is further divided into hierarchical layers, and for each layer we have identified key parameters for making major design choices. This new taxonomy provides a comprehensive framework that can guide network security visualization developers to systematically explore the design space and make informed design decisions. It can also help developers or users systematically evaluate existing network security visualization techniques and systems. Finally it helps developers identify gaps in the design space and create new techniques. Taxonomy showed that most of the existing computer security visualization programs are data centered. However, some studies have shown that task centered visualization is perhaps more effective. To test this hypothesis, we propose a task centered visualization design framework, in which tasks are explicitly identified and organized and visualizations are constructed for specific tasks and their related data parameters. The center piece of this framework is a task tree which dynamically links the raw data with automatically generated visualization. The task tree serves as a high level interaction technique that allows users to conduct problem solving naturally at the task level, while still giving end users flexible control over the visualization construction. This work is currently being extended by building a prototype visualization system based on a Task-centered Visualization Design Architecture

Automated Visual Discourse Synthesis: Coherence, Versatility, and Interactivity

In this proposal, we present comprehensive and systematic approaches of building systems that can automatically generate coherent visual discourse for interactive environments. A visual discourse refers to a series of connected visual displays. A coherent visual discourse requires smooth transitions between displays, consistent designs within and among displays, and effective integration of various components. While our main research goal is to develop approaches to automatically create coherent, versatile, and interactive visual discourse, we also emphasize integrating these approaches into a general framework to provide a reference model in which a specific system is considered an instantiation of the framework. In other words, the framework should consist of a knowledge base, an inference engine, a visual realizer, and an interaction handler. As a consequence, not only can a general framework serve as a template from which a specific generation system can be instantiated,but the framework also can be used as a base for comparing or evaluating different systems. We concentrate on the basic issues involved in developing comprehensive and systematic approaches to ensure a visual discourse's coherence, versatility, and interactivity. In particular, To ensure coherence, we have established a set of comprehensive design criteria to measure both expressiveness and effectiveness of a visual discourse. To provide versatility, we address the design of visual discourse for heterogeneous information. Within such discussions, heterogeneous information refers to both quantitative and qualitative, or static and dynamic information. In addition, we are also concerned with characterizing and employing different visual media and a wide variety of visual techniques. To support interaction, we integrate many conventional user interface metaphors and styles into visual discourse design and explore reactive planning approaches to provide proper response to user interactions. To establish the framework, we identify various knowledge sources and determine effective knowledge representation paradigms in constructing the knowledge base. We emphasize the efficiency, usability, and flexibility issues in modeling the inference engine. We are concerned with portability and parallelization issues in building the visual realizer, and we also take into account interaction capabilities for interactive environments. We describe a system called IMPROVISE (Illustrative Metaphor Production in Reactive Object-oriented VISual Environments) that serves as a proof-of-concept prototype. IMPROVISE is built based on our framework, aiming to automatically generate coherent visual discourse for various application domains in interactive environments. IMPROVISE has been used in two testbed application domains to demonstrate its generality and flexibility. Examples from both domains will be given to illustrate IMPROVISE's generation process and to identify the future research areas

Top-down hierarchical planning of coherent visual discourse

A visual discourse is a series of connected visual displays. A coherent visual discourse requires smooth transitions between displays, consistent design within and across displays, and successful integration of new information into existing displays. We present an approach for automatically designing a coherent visual discourse. A top-down, hierarchical-decomposition partial-order planner is used to efficiently plan the visual discourse. Visual representations are modelled as visual objects, graphical techniques are employed as planning operators, and design policies are encoded as constraints. This approach not only improves the computational efficiency compared to search-based approaches, but also facilitates knowledge encoding, and ensures global coherency. Keywords: Top-down hierarchical planning, automated graphics generation, knowledge-based user interfaces