Modelling Reactive Multimedia: Design and Authoring

Multimedia document authoring is a multifaceted activity, and authoring tools tend to concentrate on a restricted set of the activities involved in the creation of a multimedia artifact. In particular, a distinction may be drawn between the design and the implementation of a multimedia artifact. This paper presents a comparison of three different authoring paradigms, based on the common case study of a simple interactive animation. We present details of its implementation using the three different authoring tools, MCF, Fran and SMIL 2.0, and we discuss the conclusions that may be drawn from our comparison of the three approaches

The design and implementation of Object Grammars

An Object Grammar is a variation on traditional BNF grammars, where the notation is extended to support declarative bidirectional mappings between text and object graphs. The two directions for interpreting Object Grammars are parsing and formatting. Parsing transforms text into an object graph by recognizing syntactic features and creating the corresponding object structure. In the reverse direction, formatting recognizes object graph features and generates an appropriate textual presentation. The key to Object Grammars is the expressive power of the mapping, which decouples the syntactic structure from the graph structure. To handle graphs, Object Grammars support declarative annotations for resolving textual names that refer to arbitrary objects in the graph structure. Predicates on the semantic structure provide additional control over the mapping. Furthermore, Object Grammars are compositional so that languages may be defined in a modular fashion. We have implemented our approach to Object Grammars as one of the foundations of the Ens (o) over bar system and illustrate the utility of our approach by showing how it enables definition and composition of domain-specific languages (DSLs). (C) 2014 Elsevier B.V. All rights reserved.</p

Constraint-based graphical layout of multimodal presentations

When developing advanced multimodal interfaces, combining the characteristics of different modalities such as natural language, graphics, animation, virtual realities, etc., the question of automatically designing the graphical layout of such presentations in an appropriate format becomes increasingly important. So, to communicate information to the user in an expressive and effective way, a knowledge-based layout component has to be integrated into the architecture of an intelligent presentation system. In order to achieve a coherent output, it must be able to reflect certain semantic and pragmatic relations specified by a presentation planner to arrange the visual appearance of a mixture of textual and graphic fragments delivered by mode-specific generators. In this paper we will illustrate by the example of LayLab, the layout manager of the multimodal presentation system WIP, how the complex positioning problem for multimodal information can be treated as a constraint satisfaction problem. The design of an aesthetically pleasing layout is characterized as a combination of a general search problem in a finite discrete search space and an optimization problem. Therefore, we have integrated two dedicated constraint solvers, an incremental hierarchy solver and a finite domain solver, in a layered constraint solver model CLAY, which is triggered from a common metalevel by rules and defaults. The underlying constraint language is able to encode graphical design knowledge expressed by semantic/pragmatic, geometrical/topological, and temporal relations. Furthermore, this mechanism allows one to prioritize the constraints as well as to handle constraint solving over finite domains. As graphical constraints frequently have only local effects, they are incrementally generated by the system on the fly. Ultimately, we will illustrate the functionality of LayLab by some snapshots of an example run

Formalizing graphical notations

The thesis describes research into graphical notations for software engineering, with a principal interest in ways of formalizing them. The research seeks to provide a theoretical basis that will help in designing both notations and the software tools that process them. The work starts from a survey of literature on notation, followed by a review of techniques for formal description and for computational handling of notations. The survey concentrates on collecting views of the benefits and the problems attending notation use in software development; the review covers picture description languages, grammars and tools such as generic editors and visual programming environments. The main problem of notation is found to be a lack of any coherent, rigorous description methods. The current approaches to this problem are analysed as lacking in consensus on syntax specification and also lacking a clear focus on a defined concept of notated expression. To address these deficiencies, the thesis embarks upon an exploration of serniotic, linguistic and logical theory; this culminates in a proposed formalization of serniosis in notations, using categorial model theory as a mathematical foundation. An argument about the structure of sign systems leads to an analysis of notation into a layered system of tractable theories, spanning the gap between expressive pictorial medium and subject domain. This notion of 'tectonic' theory aims to treat both diagrams and formulae together. The research gives details of how syntactic structure can be sketched in a mathematical sense, with examples applying to software development diagrams, offering a new solution to the problem of notation specification. Based on these methods, the thesis discusses directions for resolving the harder problems of supporting notation design, processing and computer-aided generic editing. A number of future research areas are thereby opened up. For practical trial of the ideas, the work proceeds to the development and partial implementation of a system to aid the design of notations and editors. Finally the thesis is evaluated as a contribution to theory in an area which has not attracted a standard approach

Efficient Analysis of Complex Diagrams using Constraint-Based Parsing

This paper describes substantial advances in the analysis (parsing) of diagrams using constraint grammars. The addition of set types to the grammar and spatial indexing of the data make it possible to efficiently parse real diagrams of substantial complexity. The system is probably the first to demonstrate efficient diagram parsing using grammars that easily be retargeted to other domains. The work assumes that the diagrams are available as a flat collection of graphics primitives: lines, polygons, circles, Bezier curves and text. This is appropriate for future electronic documents or for vectorized diagrams converted from scanned images. The classes of diagrams that we have analyzed include x,y data graphs and genetic diagrams drawn from the biological literature, as well as finite state automata diagrams (states and arcs). As an example, parsing a four-part data graph composed of 133 primitives required 35 sec using Macintosh Common Lisp on a Macintosh Quadra 700.Comment: 9 pages, Postscript, no fonts, compressed, uuencoded. Composed in MSWord 5.1a for the Mac. To appear in ICDAR '95. Other versions at ftp://ftp.ccs.neu.edu/pub/people/futrell

Unit Testing in ASPIDE

Answer Set Programming (ASP) is a declarative logic programming formalism, which is employed nowadays in both academic and industrial real-world applications. Although some tools for supporting the development of ASP programs have been proposed in the last few years, the crucial task of testing ASP programs received less attention, and is an Achilles' heel of the available programming environments. In this paper we present a language for specifying and running unit tests on ASP programs. The testing language has been implemented in ASPIDE, a comprehensive IDE for ASP, which supports the entire life-cycle of ASP development with a collection of user-friendly graphical tools for program composition, testing, debugging, profiling, solver execution configuration, and output-handling.Comment: 12 pages, 4 figures, Proceedings of the 25th Workshop on Logic Programming (WLP 2011

A Visual Notation for Declarative Behaviour Specification

Logical programming has many merits that should appeal to modellers. It enables declarative specifications that are free from implementation details and even (mostly) abstracts away from control flow specification. However, the textual syntax of, for example PROLOG, most likely represents a barrier to the adoption of such languages in the modelling community. The visual notation presented in this paper aims to facilitate the understanding of behaviour specifications based on logic programming. I anticipate that the dataflow-like nature of the resulting diagrams will appeal to modellers. I believe the visual notation to be an improvement over the traditional textual syntax for the purpose of specifying PROLOG programs as such, but the ultimate hope is to have found a vehicle to make declarative logic programming a commonplace activity in multi-paradigm modelling

Modelling systems that integrate programming language and environment mechanisms

Once we relax the assumption that it must be possible to specify programs solely in terms of text and enter them in isolation from other tools, the range of possible program development mechanisms is significantly increased. Thus, in the light of advances in the field of integrated software development environments and in view of the wider availability of suitable workstations, we should reconsider the way we perceive (and, hence design) programming languages. This paper describes on-going work aimed at exploring the role of the programming language in the context of modern software development environments. The work is currently focused on two fronts: the development of a formalism for describing both a programming language and associated environment mechanisms, and the design of environment mechanisms that support software maintenance and reuse, complementing those traditionally provided by programming languages. This paper focuses on work in the first of these two areas

Terminological reasoning and partial inductive definitions

There are two motivations for this paper. (i) In terminological systems in the tradition of KL-ONE the taxonomic and conceptual knowledge of a particular problem domain can be represented by so called concepts. The intensional definitions of these concepts can be analyzed and checked for plausibility using certain reasoning services (e.g. subsumption) that make the user conscious of some of the consequences of his definitions. A hybrid knowledge base can then rely on these checked definitions. In this paper a terminological formalism is embedded into the formalism of partial inductive definitions (PID) such that a flexible environment for experimenting with this kind of hybrid systems and the terminological formalism itself is obtained. (ii) Terminological formalisms provide (terminating) decision procedures for their reasoning services dealing with a restricted kind of quantification. Mapping these algorithms to PID improves the understanding of control and explicit quantification in PID