Automatic design of multimodal presentations

We describe our attempt to integrate multiple AI components such as planning, knowledge representation, natural language generation, and graphics generation into a functioning prototype called WIP that plans and coordinates multimodal presentations in which all material is generated by the system. WIP allows the generation of alternate presentations of the same content taking into account various contextual factors such as the user\u27s degree of expertise and preferences for a particular output medium or mode. The current prototype of WIP generates multimodal explanations and instructions for assembling, using, maintaining or repairing physical devices. This paper introduces the task, the functionality and the architecture of the WIP system. We show that in WIP the design of a multimodal document is viewed as a non-monotonic process that includes various revisions of preliminary results, massive replanning and plan repairs, and many negotiations between design and realization components in order to achieve an optimal division of work between text and graphics. We describe how the plan-based approach to presentation design can be exploited so that graphics generation influences the production of text and vice versa. Finally, we discuss the generation of cross-modal expressions that establish referential relationships between text and graphics elements

The Augmented Reality Systems in a Maintenance of Equipment tutorial context

O desenvolvimento da Realidade Aumentada como um conceito e as técnicas disponíveis relacionadas têm vindo a estabilizar ao longo dos anos, permitindo o aparecimento de uma numerosa quantidade de aplicações de Realidade Aumentada em áreas como a medicina, design, manutenção e reparação, anotação e visualização, planeamento de trajectória de robôs, aviação militar e entretimento que tem recebido mais atenção.Embora muitas dessas áreas tenham demonstrado as vantagens da Realidade Aumentada, ainda existem algumas aberturas por preencher. A montagem e manutenção de equipmento, por exemplo, tem sido um tema de estudo investigado minuciosamente.Várias aplicações têm sido desenvolvidas que demonstram os benefícios da Realidade Aumentada em tarefas de montagem, mas o ponto principal de inovação desta dissertação foca-se em permitir a criação de Tutoriais de montagem utilizando Realidade Aumentada através do mesmo sistema de Realidade Aumentada que permite a execução do tutorial. A criação de tutoriais seria efectuada através da utilização de um digitalizador 3D, de modo a identificar cuidadosamente a localização da próxima peça a ser montada.Augmented Reality's development as a concept and available related techniques have been stabilizing throughout the years, allowing a numerous of AR applications to appear in several areas such as medical, design, maintenance and repair, annotation and visualization, robot path planning, military aircraft and entertainment which has received the most attention.While many of those areas have shown the advantages AR can bring, there are still some loopholes that need to be filled. Assembly and maintenance of equipment, for instance, has been a thoroughly investigated subject of study in AR.Many applications have been developed which demonstrate the benefits of AR in assembly tasks, but this dissertation's main innovative point is to allow the creation of AR assembly tutorials using the same AR system that allows its tutorial execution. The creation of tutorials would be carried out through the use of a manual 3D Digitizer, in order to carefully pinpoint each location of the next assembly component

Variable Resolution & Dimensional Mapping For 3d Model Optimization

Three-dimensional computer models, especially geospatial architectural data sets, can be visualized in the same way humans experience the world, providing a realistic, interactive experience. Scene familiarization, architectural analysis, scientific visualization, and many other applications would benefit from finely detailed, high resolution, 3D models. Automated methods to construct these 3D models traditionally has produced data sets that are often low fidelity or inaccurate; otherwise, they are initially highly detailed, but are very labor and time intensive to construct. Such data sets are often not practical for common real-time usage and are not easily updated. This thesis proposes Variable Resolution & Dimensional Mapping (VRDM), a methodology that has been developed to address some of the limitations of existing approaches to model construction from images. Key components of VRDM are texture palettes, which enable variable and ultra-high resolution images to be easily composited; texture features, which allow image features to integrated as image or geometry, and have the ability to modify the geometric model structure to add detail. These components support a primary VRDM objective of facilitating model refinement with additional data. This can be done until the desired fidelity is achieved as practical limits of infinite detail are approached. Texture Levels, the third component, enable real-time interaction with a very detailed model, along with the flexibility of having alternate pixel data for a given area of the model and this is achieved through extra dimensions. Together these techniques have been used to construct models that can contain GBs of imagery data

NASA/NBS (National Aeronautics and Space Administration/National Bureau of Standards) standard reference model for telerobot control system architecture (NASREM)

The document describes the NASA Standard Reference Model (NASREM) Architecture for the Space Station Telerobot Control System. It defines the functional requirements and high level specifications of the control system for the NASA space Station document for the functional specification, and a guideline for the development of the control system architecture, of the 10C Flight Telerobot Servicer. The NASREM telerobot control system architecture defines a set of standard modules and interfaces which facilitates software design, development, validation, and test, and make possible the integration of telerobotics software from a wide variety of sources. Standard interfaces also provide the software hooks necessary to incrementally upgrade future Flight Telerobot Systems as new capabilities develop in computer science, robotics, and autonomous system control

Plan-based integration of natural language and graphics generation

Multimodal interfaces combining natural language and graphics take advantage of both the individual strength of each communication mode and the fact that several modes can be employed in parallel. The central claim of this paper is that the generation of a multimodal presentation system WIP which allows the generation of alternate presentations of the same content taking into account various contextual factors. We discuss how the plan-based approach to presentation design can be exploited so that graphics generation influences the production of text and vice versa. We show that well-known concepts from the area of natural language processing like speech acts, anaphora, and rhetorical relations take on an extended meaning in the context of multimodal communication. Finally, we discuss two detailed examples illustrating and reinforcing our theoretical claims

Vertex classification for non-uniform geometry reduction.

Complex models created from isosurface extraction or CAD and highly accurate 3D models produced from high-resolution scanners are useful, for example, for medical simulation, Virtual Reality and entertainment. Often models in general require some sort of manual editing before they can be incorporated in a walkthrough, simulation, computer game or movie. The visualization challenges of a 3D editing tool may be regarded as similar to that of those of other applications that include an element of visualization such as Virtual Reality. However the rendering interaction requirements of each of these applications varies according to their purpose. For rendering photo-realistic images in movies computer farms can render uninterrupted for weeks, a 3D editing tool requires fast access to a model's fine data. In Virtual Reality rendering acceleration techniques such as level of detail can temporarily render parts of a scene with alternative lower complexity versions in order to meet a frame rate tolerable for the user. These alternative versions can be dynamic increments of complexity or static models that were uniformly simplified across the model by minimizing some cost function. Scanners typically have a fixed sampling rate for the entire model being scanned, and therefore may generate large amounts of data in areas not of much interest or that contribute little to the application at hand. It is therefore desirable to simplify such models non-uniformly. Features such as very high curvature areas or borders can be detected automatically and simplified differently to other areas without any interaction or visualization. However a problem arises when one wishes to manually select features of interest in the original model to preserve and create stand alone, non-uniformly reduced versions of large models, for example for medical simulation. To inspect and view such models the memory requirements of LoD representations can be prohibitive and prevent storage of a model in main memory. Furthermore, although asynchronous rendering of a base simplified model ensures a frame rate tolerable to the user whilst detail is paged, no guarantees can be made that what the user is selecting is at the original resolution of the model or of an appropriate LoD owing to disk lag or the complexity of a particular view selected by the user. This thesis presents an interactive method in the con text of a 3D editing application for feature selection from any model that fits in main memory. We present a new compression/decompression of triangle normals and colour technique which does not require dedicated hardware that allows for 87.4% memory reduction and allows larger models to fit in main memory with at most 1.3/2.5 degrees of error on triangle normals and to be viewed interactively. To address scale and available hardware resources, we reference a hierarchy of volumes of different sizes. The distances of the volumes at each level of the hierarchy to the intersection point of the line of sight with the model are calculated and these distances sorted. At startup an appropriate level of the tree is automatically chosen by separating the time required for rendering from that required for sorting and constraining the latter according to the resources available. A clustered navigation skin and depth buffer strategy allows for the interactive visualisation of models of any size, ensuring that triangles from the closest volumes are rendered over the navigation skin even when the clustered skin may be closer to the viewer than the original model. We show results with scanned models, CAD, textured models and an isosurface. This thesis addresses numerical issues arising from the optimisation of cost functions in LoD algorithms and presents a semi-automatic solution for selection of the threshold on the condition number of the matrix to be inverted for optimal placement of the new vertex created by an edge collapse. We show that the units in which a model is expressed may inadvertently affect the condition of these matrices, hence affecting the evaluation of different LoD methods with different solvers. We use the same solver with an automatically calibrated threshold to evaluate different uniform geometry reduction techniques. We then present a framework for non-uniform reduction of regular scanned models that can be used in conjunction with a variety of LoD algorithms. The benefits of non-uniform reduction are presented in the context of an animation system. (Abstract shortened by UMI.)

Automatic 3D model creation with velocity-based surface deformations

The virtual worlds of Computer Graphics are populated by geometric objects, called models. Researchers have addressed the problem of synthesizing models automatically. Traditional modeling approaches often require a user to guide the synthesis process and to look after the geometry being synthesized, but user attention is expensive, and reducing user interaction is therefore desirable. I present a scheme for the automatic creation of geometry by deforming surfaces. My scheme includes a novel surface representation; it is an explicit representation consisting of points and edges, but it is not a traditional polygonal mesh. The novel surface representation is paired with a resampling policy to control the surface density and its evolution during deformation. The surface deforms with velocities assigned to its points through a set of deformation operators. Deformation operators avoid the manual computation and assignment of velocities, the operators allow a user to interactively assign velocities with minimal effort. Additionally, Petri nets are used to automatically deform a surface by mimicking a user assigning deformation operators. Furthermore, I present an algorithm to translate from the novel surface representations to a polygonal mesh. I demonstrate the utility of my model generation scheme with a gallery of models created automatically. The scheme's surface representation and resampling policy enables a surface to deform without requiring a user to control the deformation; self-intersections and hole creation are automatically prevented. The generated models show that my scheme is well suited to create organic-like models, whose surfaces have smooth transitions between surface features, but can also produce other kinds of models. My scheme allows a user to automatically generate varied instances of richly detailed models with minimal user interaction