Specifying Software Languages: Grammars, Projectional Editors, and Unconventional Approaches

We discuss several approaches for defining software languages, together with Integrated Development Environments for them. Theoretical foundation is grammar-based models: they can be used where proven correctness of specifications is required. From a practical point of view, we discuss how language specification can be made more accessible by focusing on language workbenches and projectional editing, and discuss how it can be formalized. We also give a brief overview of unconventional ideas to language definition, and outline three open problems connected to the approaches we discuss

Recognizing Interspersed sketches quickly

Sketch recognition is the automated recognition of hand-drawn diagrams. When allowing users to sketch as they would naturally, users may draw shapes in an interspersed manner, starting a second shape before finishing the first. In order to provide freedom to draw interspersed shapes, an exponential combination of subshapes must be considered. Because of this, most sketch recognition systems either choose not to handle interspersing, or handle only a limited pre-defined amount of interspersing. Our goal is to eliminate such interspersing drawing constraints from the sketcher. This paper presents a high-level recognition algorithm that, while still exponential, allows for complete interspersing freedom, running in near real-time through early effective sub-tree pruning. At the core of the algorithm is an indexing technique that takes advantage of geometric sketch recognition techniques to index each shape for efficient access and fast pruning during recognition. We have stresstested our algorithm to show that the system recognizes shapes in less than a second even with over a hundred candidate subshapes on screen.National Science Foundation (U.S.) (IIS Creative IT Grant #0757557

Operator interfaces for the lifecycle support of component based automation systems

Current manufacturing automation systems (specifically the powertrain sector) have been facing challenges with constant pressures of globalisation, environmental concerns and ICT (Information and Communication Technology) innovations. These challenges instigate new demands for shorter product lifecycles and require customised products to be manufactured as efficiently as possible. Manufacturing systems must therefore be agile to remain competitive by supporting frequent reconfigurations involving distributed engineering activities. [Continues.

Extracción automática de modelos UML contenidos en imágenes

Aunque parezca extraño, pese a no poder encontrar sitios web especializados en ofertar diseños de software representados mediante diagramas UML, existe una ingente cantidad de documentación a disposición de cualquiera, y que contiene dichos modelos: como imágenes en documentos textuales. Este universo de información no se encuentra fácilmente accesible para los desarrolladores porque no es posible, con la tecnología actual, buscar de forma precisa información semántica dentro de imágenes. Lo único que pueden hacer los desarrolladores es intentar buscar documentos relevantes, leerlos, y decidir si los diseños le sirven a sus intereses. Para evitar este problema, y conseguir poner a disposición de toda la comunidad de desarrolladores centenas de miles de diseños, este trabajo pretende desarrollar la metodología necesaria para poder extraer la información textual y gráfica de las imágenes que representen diagramas UML, y convertirla en información pura UML (es decir, en modelos UML reales). El poner a disposición de los analistas, desarrolladores de software, o interesados tal cantidad de diagramas y modelos de software permitirá la aplicación de técnicas modernas de reutilización de software basadas en la búsqueda de diagramas UML. La búsqueda de diagramas UML de todo tipo (estáticos, dinámicos, arquitecturales, de Casos de Uso etc.) mediante similitud a uno dado permitirá potenciar los desarrollos de software de calidad, controlados en el coste, y en el tiempo de desarrollo: las tres virtudes de la reutilización de software. La complejidad de esta propuesta radica en muchos aspectos, todos ellos entrelazados: por un lado hay que considerar que la información de partida se encuentra representada con diferentes tipos de calidad, mediante bits de colores o tonos de grises. Por otro lado su semántica viene dada por la combinación de texto en lenguaje natural y estructuras gráficas. Estas estructuras gráficas tienen asociada una información semántica, accesible a la interpretación humana, que depende del tipo de diagrama. Los diagramas que representan diseños de software son documentos en formato visual con alta estructuración y contenido semántico, que se deben distinguir unos de otros. Debido a su formato en forma de imagen requieren un preprocesado mediante técnicas de visión artificial, OCR y técnicas de clusterización o clasificación basadas en aprendizaje automático. Precisamente este será el principal cometido de esta tesis: la extracción de la semántica de los diagramas en forma de imágenes encontrados en la web. La información obtenida de estos diagramas, principalmente UML, debe incluir información textual e información estructural. A la información textual se obtendrá mediante técnicas de OCR mientras que la información estructurada será detectada mediante reconocimiento de formas combinado con Inteligencia Artificial. El resultado de esta propuesta sería una metodología que podría ser aplicada para cargar repositorios de diagramas UML a partir de imágenes existentes en internet, con vistas a su posterior aplicación y puesta a disposición de los usuarios: un GOOGLE de diagramas UML.There are many interesting sites in the web offering reuse of source code, but no one giving the choice to identify, find and reuse design models using UML. However, even if this data seems to be sad, a simple web search can give you astonishing results: Get into GOOGLE images and search for “UML Class diagram”. Thousands of images will suddenly be available for you. The bad news: they are images. You cannot find anything on them, you cannot find them by content. You cannot compare them. You can, simple, download them. Could you be interested in working with those images, finding similar ones, etc.? In order to solve this problem, and reach hundreds of thousands of UML designs, this work intends to develop the necessary methodology to extract the textual and graphical information contented in UML based images, and convert them in, exactly, UML information (real UML Models represented in a UML object model) The possibility to offer such amount of diagrams to software analysts, software developers, or simply interested stakeholders will allow them to apply real, systematic and modern software reuse based on UML diagrams information retrieval. The possibility to find all kinds of diagrams (static, dynamic, architectural, Use Case, etc.) by similar content will strengthen software development based on the best quality, controlled cost and time to market principles: the three real benefits of Software reuse. This proposal has several difficulties in different fronts: to start with, one must consider that all the information is usually stored in low resolution images, where texts are difficult to read and understand and boxes and arrows are not properly drawn. And, on the other side, the semantics comes from the combination of text represented in Natural Language and graphical structures. These structures have associated semantic information, understandable by humans, which depend and change with the diagram types. Due to these problems, Artificial Vision, OCR, classification and automatic learning algorithm must be used in this thesis. This thesis, therefore, will attempt to extract semantic information for images representing UML Diagrams found in the web. The information extracted will be both textual and graphical. OCR technology (existing already) would be used to textual information. In order to extract graphical information a semantic model combined with AI will be used. The result of the proposal will be a methodology that will allow repositories (in the web or private) to offer UML diagrams based on (and pointing to) images found in the web, for further reuse.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: Antonio de Amescua Seco.- Secretario: Susana Irene Díaz Rodríguez.- Vocal: Pascual Campoy Cerver

An online corpus of UML Design Models : construction and empirical studies

We address two problems in Software Engineering. The first problem is how to assess the severity of software defects? The second problem we address is that of studying software designs. Automated support for assessing the severity of software defects helps human developers to perform this task more efficiently and more accurately. We present (MAPDESO) for assessing the severity of software defects based on IEEE Standard Classification for Software Anomalies. The novelty of the approach lies in its use of uses ontologies and ontology-based reasoning which links defects to system level quality properties. One of the main reasons that makes studying of software designs challenging is the lack of their availability. We decided to collect software designs represented by UML models stored in image formats and use image processing techniques to convert them to models. We present the 'UML Repository' which contains UML diagrams (in image and XMI format) and design metrics. We conducted a series of empirical studies using the UML Repository. These empirical studies are a drop in the ocean empirical studies that can be conducted using the repository. Yet these studies show the versatility of useful studies that can be based on this novel repository of UML designs.Erasmus Mundus program (JOSYLEEN)Algorithms and the Foundations of Software technolog

Perceptually-based language to simplify sketch recognition user interface development

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 473-495).Diagrammatic sketching is a natural modality of human-computer interaction that can be used for a variety of tasks, for example, conceptual design. Sketch recognition systems are currently being developed for many domains. However, they require signal-processing expertise if they are to handle the intricacies of each domain, and they are time-consuming to build. Our goal is to enable user interface designers and domain experts who may not have expertise in sketch recognition to be able to build these sketch systems. We created and implemented a new framework (FLUID - f acilitating user interface development) in which developers can specify a domain description indicating how domain shapes are to be recognized, displayed, and edited. This description is then automatically transformed into a sketch recognition user interface for that domain. LADDER, a language using a perceptual vocabulary based on Gestalt principles, was developed to describe how to recognize, display, and edit domain shapes. A translator and a customizable recognition system (GUILD - a generator of user interfaces using ladder descriptions) are combined with a domain description to automatically create a domain specific recognition system.(cont.) With this new technology, by writing a domain description, developers are able to create a new sketch interface for a domain, greatly reducing the time and expertise for the task Continuing in pursuit of our goal to facilitate UI development, we noted that 1) human generated descriptions contained syntactic and conceptual errors, and that 2) it is more natural for a user to specify a shape by drawing it than by editing text. However, computer generated descriptions from a single drawn example are also flawed, as one cannot express all allowable variations in a single example. In response, we created a modification of the traditional model of active learning in which the system selectively generates its own near-miss examples and uses the human teacher as a source of labels. System generated near-misses offer a number of advantages. Human generated examples are tedious to create and may not expose problems in the current concept. It seems most effective for the near-miss examples to be generated by whichever learning participant (teacher or student) knows better where the deficiencies lie; this will allow the concepts to be more quickly and effectively refined.(cont.) When working in a closed domain such as this one, the computer learner knows exactly which conceptual uncertainties remain, and which hypotheses need to be tested and confirmed. The system uses these labeled examples to automatically build a LADDER shape description, using a modification of the version spaces algorithm that handles interrelated constraints, and which also has the ability to learn negative and disjunctive constraints.by Tracy Anne Hammond.Ph.D