Generating graphical and projectional editors

En ingénierie dirigée par les modèles, les langages spécifiques au domaine (DSL) offrent des notations adaptées à un domaine précis pour représenter ses différents concepts. De nombreux outils permettent la définition de DSLs en explicitant les relations entre un concept et ses représentations. En fonction de la sémantique du domaine, l’ingénieur du langage peut choisir entre des notations textuelles ou graphiques. Les langages de modélisation graphique nécessitent une gestion de la position, la taille et la disposition des éléments visuels afin de maximiser leur expressivité visuelle. La plupart des éditeurs de modélisation manquent de support automatique pour gérer ces propriétés de la syntaxe concrète. Les éditeurs projectionnels permettent aux utilisateurs de se concentrer sur la conception de leur modèle en limitant les modifications de la syntaxe concrète. Cependant, bien qu’ils offrent de multiples notations, ces éditeurs ne permettent pas la création de langage graphique. Dans ce mémoire, nous proposons une nouvelle approche pour concevoir des éditeurs graphiques et projectionnels. Nous avons créé une extension d’un éditeur projectionnel orienté vers le web, Gentleman, qui nous a permis d’extraire différentes exigences. Au cours du mémoire, nous décrivons leurs impacts sur les projections et proposons des lignes directrices ainsi que des exemples d’implémentation. Comme l’édition projectionnelle demande une gestion spécifique de l’interaction, nous présentons différentes approches pour interagir avec les représentations graphiques utilisant les nouvelles informations disponibles dans les projections. Étant donné que la plupart des exigences se concentrent sur la disposition des projections, nous avons défini plusieurs algorithmes simples de disposition qui couvrent une large gamme de structures pouvant être retrouvées dans un éditeur graphique. Enfin, afin d’évaluer cette approche, nous avons exploré la génération de trois éditeurs graphiques et projectionnels pour différents domaines: les machines d’états, les diagrammes de séquences et les partitions de musique.In model-driven engineering, domain specific-languages (DSL) provide tailored notations towards a specific problem domain to represent its different concepts. Multiple tools allow the definition of DSL by specifying the relations between a concept and its representations. Depending on the semantics of the domain, the language engineer can choose between textual or graphical notations. Graphical modeling languages require proper management of position, size, and layout to maximize their visual expressiveness. Most modeling editors lack automated support to manage these graphical concrete syntax properties. It is a time-consuming effort that affects the understandability of the model. Projectional editors prevent end-users from modifying the concrete syntax so they can focus on the modeling task. However, while they offer multiple notations, these editors lack support for graphical languages. During this thesis, we propose a new approach to design graphical and projectional editors. We created an extension of a web-oriented projectional editor, Gentleman, that allowed us to extract different requirements. During the thesis, we describe their impact on the projections and propose guidelines and examples of implementation. Because projectional editing requires specific management of the interaction, we present multiple approaches to interact with the graphical representations, using the new information available in the graphics. Since most of the requirements were focusing on the disposition of the projection, we define multiple simple layout algorithms that cover a large range of structures that can be found in a graphical editor. Finally, we explore the generation of three graphical and projectional editors for different domains: statecharts, sequence diagrams, and music sheet

Programmatic and Direct Manipulation, Together at Last

Direct manipulation interfaces and programmatic systems have distinct and complementary strengths. The former provide intuitive, immediate visual feedback and enable rapid prototyping, whereas the latter enable complex, reusable abstractions. Unfortunately, existing systems typically force users into just one of these two interaction modes. We present a system called Sketch-n-Sketch that integrates programmatic and direct manipulation for the particular domain of Scalable Vector Graphics (SVG). In Sketch-n-Sketch, the user writes a program to generate an output SVG canvas. Then the user may directly manipulate the canvas while the system immediately infers a program update in order to match the changes to the output, a workflow we call live synchronization. To achieve this, we propose (i) a technique called trace-based program synthesis that takes program execution history into account in order to constrain the search space and (ii) heuristics for dealing with ambiguities. Based on our experience with examples spanning 2,000 lines of code and from the results of a preliminary user study, we believe that Sketch-n-Sketch provides a novel workflow that can augment traditional programming systems. Our approach may serve as the basis for live synchronization in other application domains, as well as a starting point for yet more ambitious ways of combining programmatic and direct manipulation.Comment: PLDI 2016 Paper + Supplementary Appendice

Verovio: A Library for Engraving MEI Music Notation into SVG

Rendering symbolic music notation is a common component of many MIR applications, and many tools are available for this task. There is, however, a need for a tool that can natively render the Music Encoding Initiative (MEI) notation encodings that are increasingly used in music research projects. In this paper, we present Verovio, a library and toolkit for rendering MEI. A significant advantage of Verovio is that it implements MEI’s structure internally, making it the best suited solution for rendering features that make MEI unique. Verovio is designed as a fast, portable, lightweight tool written in pure standard C++ with no dependencies on third-party frameworks or libraries. It can be used as a command-line rendering tool, as a library, or it can be compiled to JavaScript using the Emscripten LLVM-to-JavaScript compiler. This last option is particularly interesting because it provides a complete in-browser music MEI typesetter. The SVG output from Verovio is organized in such a way that the MEI structure is preserved as much as possible. Since every graphic in SVG is an XML element that is easily addressable, Verovio is particularly well-suited for interactive applications, especially in web browsers. Verovio is available under the GPL open-source license

VGLib2D: class library for the 2D graphic visualisation of large volumes of data

This article presents the implementation of a class library in C# language, which uses the recent .Net technologies, aimed at the creation of documents or images in Scalable Vector Graphics (SVG) format, for the representation, visualisation and printing of large volumes of two-dimensional (2D) data. The goal is to take advantage of SVG for applications implementation effects that, for instance, are Web based and where the graphical representation of large volumes of data, both dynamically and in real time, is necessary, along with its immediate availability for the purposes of visualisation, printing and downloading

Interactive and Animated Scalable Vector Graphics and R Data Displays

We describe an approach to creating interactive and animated graphical displays using R's graphics engine and Scalable Vector Graphics, an XML vocabulary for describing two-dimensional graphical displays. We use the svg() graphics device in R and then post-process the resulting XML documents. The post-processing identities the elements in the SVG that correspond to the different components of the graphical display, e.g., points, axes, labels, lines. One can then annotate these elements to add interactivity and animation effects. One can also use JavaScript to provide dynamic interactive effects to the plot, enabling rich user interactions and compelling visualizations. The resulting SVG documents can be embedded withinHTML documents and can involve JavaScript code that integrates the SVG and HTML objects. The functionality is provided via the SVGAnnotation package and makes static plots generated via R graphics functions available as stand-alone, interactive and animated plots for the Web and other venues

Concrete syntax definition for modeling languages

Model Driven Engineering (MDE) promotes the use of models as primary artefacts of a software development process, as an attempt to handle complexity through abstraction, e.g. to cope with the evolution of execution platforms. MDE follows a stepwise approach, by prescribing to develop abstract models further improved to integrate little by little details relative to the final deployment platforms. Thus, the application of an MDE process results in various models residing at various levels of abstraction. Each one of these models is expressed in a modeling language, in which one may find appropriate concepts for the abstraction level considered. Many advocate to use the right (modeling) language for the right purpose. This means that it is sometimes better approach to use small languages specific to the considered domain and abstraction level, than to use general purpose languages (e.g. UML) when they do not perfectly fit the (modeling) needs. As a matter of fact, an MDE development process, which involves many different domains and abstraction levels, should also involve a large variety of modeling languages. Project managers who want to apply an MDE process need to deal with this language proliferation to such an extent that, in the long run, one may infer that language engineers can become major actors of software development teams. We believe that comprehensive modeling language management facilities may considerably alleviate that MDE drawback. Such facilities may include modeling language definition, extension, adaptation, or composition. To define a (modeling) language, one needs to define its abstract syntax, its semantics, and one or more concrete syntaxes. This thesis focuses on concrete syntax definition for modeling languages, when the abstract syntax is given in the form of a metamodel. We will provide solutions both for textual and graphical concrete syntaxes. Some of our experiences in building textual languages (as MTL, a model transformation language), and graphical languages (as Netsilon, a web-application modeler) has shown that a lot of work was spent in implementing interface using traditional techniques, be it a text processor generated from a compiler compiler specification, or a modeler making use of modern 2D graphical libraries. Indeed, abstract and concrete syntax were implemented in a disconnected way, and it was then necessary to assemble them, which became rapidly clumsy while abstract syntax evolved. We built our solution to concrete syntax definition as companions of the abstract syntax. The definition of concrete syntax we propose here made it possible to build automatic tools able to analyze or synthesize models from/to text, and to create graphical modelers. We will present a metamodel for textual concrete syntax definition to construct constructive reversible grammars. We will also propose a technique for graphical concrete syntax definition following a two-step process: specification and realization. Specification is a restrictive approach in which a metamodel defines a graphical concrete syntax. Both relations with abstract syntax and spatial relationships are expressed by means of constraints. The realization step proposes a way to provide the concrete syntax tree a meaning, by attributing it a graphical appearance, and by expressing possible user interactions. The structure of the document is the following. After introducing in deeper details the problem and the general structure of the solution we propose, we will take a tour of MDE, text and graph grammars. Then, we will present Netsilon as an example of an MDE tool to MDE development, which required both the definition of a graphical and a textual modeling language. The two following sections will present the solutions we propose for textual and graphical concrete syntax definition, respectively. Final remarks and possible improvements, especially regarding reusability in general of MDE meta-artifacts (like metamodels or model transformations), and of concrete syntax in particular, will conclude the document