Building program understanding tools using visitor combinators

Program understanding tools manipulate program representations, such as abstract syntax trees, control-flow graphs, or data-flow graphs. This paper deals with the use of visitor combinators to conduct such manipulations. Visitor combinators are an extension of the well-known visitor design pattern. They are small, reusable classes that carry out specific visiting steps. They can be composed in different constellations to build more complex visitors. We evaluate the expressiveness, reusability, ease of development, and applicability of visitor combinators to the construction of program understanding tools. To that end, we conduct a case study in the use of visitor combinators for control-flow analysis and visualization as used in a commercial Cobol program understanding tool

Matching objects without language extension

Pattern matching is a powerful programming concept which has proven its merits in declarative programming. The absence of pattern-matching in object-oriented programming languages is felt especially when tackling source code processing problems. But existing proposals for pattern matching in such languages rely on language extension, which makes their adoption overly intrusive. We propose an approach to support pattern matching in mainstream object-oriented languages without language extension. In this approach, a pattern is a first-class entity, which can be created, be passed as argument, and receive method invocations, just like any other object. We demonstrate how our approach can be used in conjunction with existing parser generators to perform pattern matching on various kinds of abstract syntax representation. We elaborate our approach to include concrete syntax patterns, and mixing of patterns and visitors for the construction of sophisticated syntax tree traversals.Thanks to Rob van der Leek of the Software Improvement Group for valuable feedback regarding this paper and the MatchO library. The author is recipient of a research grant from the Fundacao para a Ciencia e a Tecnologia, under grant number SFRH/BPD/11609/2002

Effective Strategic Programming for Java Developers

International audienceIn object programming languages, the Visitor design pattern allows separation of algorithms and data-structures. When applying this pattern to tree-like structures, programmers are always confronted with the difficulty of making their code evolve. One reason is that the code implementing the algorithm is interwound with the code implementing the traversal inside the Visitor. When implementing algorithms such as data analyses or transformations, encoding the traversal directly into the algorithm turns out to be cumbersome as this type of algorithm only focuses on a small part of the data-structure model (e.g., program optimization). Unfortunately, typed programming languages like Java do not offer simple solutions for expressing generic traversals. Rewrite-based languages like ELAN or Stratego have introduced the notion of strategies to express both generic traversal and rule application control in a declarative way. Starting from this approach, our goal was to make the notion of strategic programming available in a widely used language such as Java and thus to offer generic traversals in typed Java structures. In this paper, we present the strategy language SL that provides programming support for strategies in Java

Strategic term rewriting and its application to a VDM-SL to SQL conversion

We constructed a tool, called VooDooM, which converts datatypes in Vdm-sl into Sql relational data models. The conversion involves transformation of algebraic types to maps and products, and pointer introduction. The conversion is specified as a theory of refinement by calculation. The implementation technology is strategic term rewriting in Haskell, as supported by the Strafunski bundle. Due to these choices of theory and technology, the road from theory to practise is straightforward.Fundação para a Ciência e a Tecnologia (FCT) - POSI/ICHS/44304/2002Agência de Inovação (ADI) - ∑!223

Concrete Syntax with Black Box Parsers

Context: Meta programming consists for a large part of matching, analyzing, and transforming syntax trees. Many meta programming systems process abstract syntax trees, but this requires intimate knowledge of the structure of the data type describing the abstract syntax. As a result, meta programming is error-prone, and meta programs are not resilient to evolution of the structure of such ASTs, requiring invasive, fault-prone change to these programs. Inquiry: Concrete syntax patterns alleviate this problem by allowing the meta programmer to match and create syntax trees using the actual syntax of the object language. Systems supporting concrete syntax patterns, however, require a concrete grammar of the object language in their own formalism. Creating such grammars is a costly and error-prone process, especially for realistic languages such as Java and C++. Approach: In this paper we present Concretely, a technique to extend meta programming systems with pluggable concrete syntax patterns, based on external, black box parsers. We illustrate Concretely in the context of Rascal, an open-source meta programming system and language workbench, and show how to reuse existing parsers for Java, JavaScript, and C++. Furthermore, we propose Tympanic, a DSL to declaratively map external AST structures to Rascal's internal data structures. Tympanic allows implementors of Concretely to solve the impedance mismatch between object-oriented class hierarchies in Java and Rascal's algebraic data types. Both the algebraic data type and AST marshalling code is automatically generated. Knowledge: The conceptual architecture of Concretely and Tympanic supports the reuse of pre-existing, external parsers, and their AST representation in meta programming systems that feature concrete syntax patterns for matching and constructing syntax trees. As such this opens up concrete syntax pattern matching for a host of realistic languages for which writing a grammar from scratch is time consuming and error-prone, but for which industry-strength parsers exist in the wild. Grounding: We evaluate Concretely in terms of source lines of code (SLOC), relative to the size of the AST data type and marshalling code. We show that for real programming languages such as C++ and Java, adding support for concrete syntax patterns takes an effort only in the order of dozens of SLOC. Similarly, we evaluate Tympanic in terms of SLOC, showing an order of magnitude of reduction in SLOC compared to manual implementation of the AST data types and marshalling code. Importance: Meta programming has applications in reverse engineering, reengineering, source code analysis, static analysis, software renovation, domain-specific language engineering, and many others. Processing of syntax trees is central to all of these tasks. Concrete syntax patterns improve the practice of constructing meta programs. The combination of Concretely and Tympanic has the potential to make concrete syntax patterns available with very little effort, thereby improving and promoting the application of meta programming in the general software engineering context

Traversal, Case Analysis, and Lowering for C++ Program Analysis

To work effectively, programmers need tools to support their typical development activities, such as the creation, analysis, and transformation of source code. Analysis and transformation tools can be difficult to write for modern programming languages and, without a reusable framework, each tool must separately implement nontrivial algorithms like name lookup and type checking. This thesis describes an extension to one such framework, named Pivot, that focuses on programs written in C++. This extension, named Filter, assists the tool builder in traversal, case analysis, and lowering of the data structure representing C++ programs. Comparisons described in the thesis show a 2-4x code reduction when solving basic problems (e.g., searching for uses of a given declaration) using the extension and a performance overhead that drops below 2x for larger problems (e.g., checking C++ layout compatibility)

VooDooM : support for understanding and re-engineering of VDM-SL specifications

Tese mestrado informáticaThe main purpose of this work is to define steady ground for supporting the understanding and re-engineering of VDM-SL specifications. Understanding and re-engineering are justified by Lehman’s laws of software evolution which state, for instance, that systems must be continually adapted and as a program evolves its complexity increases unless specific work is done to reduce it. This thesis reports the implementation of understanding and re-enginering techniques in a tool called VooDooM, which was built in three well defined steps. First, development of the language front-end to recognize the VDMSL language, using a grammar-centered approach, supported by the SDF formalism, in which a wide variety of components are automatically generated from a single grammar; Second, development of understanding support, in which graphs are extracted and derived and subsequently used as input to strongly-connected components, formal concept analysis and metrication. Last, development of re-engineering support, through the development of a relational calculator that transforms a formal specification into an equivalent model which can be translated to SQL. In all steps of the work we thoroughly document the path from theory to practice and we conclude by reporting successful results obtained in two test cases.O objectivo principal deste trabalho é a definiçãoo de uma infra-estrutura para suportar compreensão e re-engenharia de especificações escritas em VDM-SL. compreensão e re-engenharia justificam-se pelas leis de evolução do software. Estas leis, formuladas por Lehman, definem, por exemplo, que um qualquer sistema deve ser continuamente adaptado e `a medida que os programas evoluem a sua complexidade tende sempre a aumentar. Esta tese descreve o estudo de técnicas de compreensão e re-engenharia que foram implementadas numa ferramenta chamada VooDooM. Esta implementação foi efectuada em três etapas bem definidas. Primeiro, foi desenvolvido um parser (front-end) para reconhecer a linguagem VDM-SL. Para tal, foi utilizada uma abordagem centrada na gramática, suportada no formalismo SDF, que está equipado com ferramentas de geração automática de diversos componentes. Segundo, para o suporte de compreensão, foram desenvolvidas funcionalidades para extrair e derivar grafos que são utilizados em técnicas de análise como componentes fortemente relacionados, análise de conceitos (formal concept analysis) e métricas. Por último, para o suporte de re-engenharia, foi prototipada uma calculadora relacional que transforma um modelo, definido numa especificação formal, no seu equivalente relacional que pode ser traduzido para SQL. Em todas as etapas realizadas h a preocupação de documentar o percurso entre teoria para a prática. A análise de resultados obtida no estudo de caso revela o sucesso da abordagem e as suas potencialidades para desenvolvimentos futuros