Beyond Notations: Hygienic Macro Expansion for Theorem Proving Languages

In interactive theorem provers (ITPs), extensible syntax is not only crucial to lower the cognitive burden of manipulating complex mathematical objects, but plays a critical role in developing reusable abstractions in libraries. Most ITPs support such extensions in the form of restrictive "syntax sugar" substitutions and other ad hoc mechanisms, which are too rudimentary to support many desirable abstractions. As a result, libraries are littered with unnecessary redundancy. Tactic languages in these systems are plagued by a seemingly unrelated issue: accidental name capture, which often produces unexpected and counterintuitive behavior. We take ideas from the Scheme family of programming languages and solve these two problems simultaneously by proposing a novel hygienic macro system custom-built for ITPs. We further describe how our approach can be extended to cover type-directed macro expansion resulting in a single, uniform system offering multiple abstraction levels that range from supporting simplest syntax sugars to elaboration of formerly baked-in syntax. We have implemented our new macro system and integrated it into the upcoming version (v4) of the Lean theorem prover. Despite its expressivity, the macro system is simple enough that it can easily be integrated into other systems.Comment: accepted to IJCAR 202

The Lean 4 Theorem Prover and Programming Language

Lean 4 is a reimplementation of the Lean interactive theorem prover (ITP) in Lean itself. It addresses many shortcomings of the previous versions and contains many new features. Lean 4 is fully extensible: users can modify and extend the parser, elaborator, tactics, decision procedures, pretty printer, and code generator. The new system has a hygienic macro system custom-built for ITPs. It contains a new typeclass resolution procedure based on tabled resolution, addressing significant performance problems reported by the growing user base. Lean 4 is also an efficient functional programming language based on a novel programming paradigm called functional but in-place. Efficient code generation is crucial for Lean users because many write custom proof automation procedures in Lean itself

Domain specific language implementation via compile-time meta-programming.

Domain specific languages (DSLs) are mini-languages that are increasingly seen as being a valuable tool for software developers and non-developers alike. DSLs must currently be created in an ad-hoc fashion, often leading to high development costs and implementations of variable quality. In this article, I show how expressive DSLs can be hygienically embedded in the Converge programming language using its compile-time meta-programming facility, the concept of DSL blocks, and specialised error reporting techniques. By making use of pre-existing facilities, and following a simple methodology, DSL implementation costs can be significantly reduced whilst leading to higher quality DSL implementations

Implementation of Faceted Values in Node.JS.

Information flow analysis is the study of mechanisms by which developers may protect sensitive data within an ecosystem containing untrusted third-party code. Secure multi-execution is one such mechanism that reliably prevents undesirable information flows, but a programmer’s use of secure multi-execution is itself challenging and prone to error. Faceted values have been shown to provide an alternative to secure multi-execution which is, in theory, functionally equivalent. The purpose of this work is to show that the theory holds in practice by implementing usable faceted values in JavaScript via source code transformation. The primary contribution of this project is to provide a library that makes these transformations possible in any standard JavaScript runtime without requiring native support. We build a pipeline that takes JavaScript code with syntactic support for faceted values and, through source code transformation, produces platform-independent JavaScript code containing functional faceted values. Our findings include a method by which we may optimize the use of faceted values through static analysis of the program’s information flow

Maya: multiple-dispatch syntax extension in Java

technical reportWe have designed and implemented Maya, a version of Java that allows programmers to extend and reinterpret its syntax. Maya generalizes macro systems by treating grammar productions as generic functions, and semantic actions on productions as multimethods on the corresponding generic functions. Programmers can write new generic functions (i.e., grammar productions) and new multimethods (i.e., semantic actions), through which they can extend the grammar of the language and change the semantics of its syntactic constructs, respectively. Maya?s multimethods are compile-time metaprograms that transform abstract syntax: they execute at program compile-time, because they are semantic actions executed by the parser. Maya?s multimethods can be dispatched on the syntactic structure of the input, as well as the static, source-level types of expressions in the input. In this paper we describe what Maya can do and how it works. We describe how its novel parsing techniques work and how Maya can statically detect certain kinds of errors such as hygiene violations. Finally, to demonstrate Maya's expressiveness, we describe how Maya can be used to implement the MultiJava language, which was described by Clifton et al. at OOPSLA 2000

Modelling homogeneous generative meta-programming

Homogeneous generative meta-programming (HGMP) enables the generation of program fragments at compile-time or run-time. We present a foundational calculus which can model both compile-time and run-time evaluated HGMP, allowing us to model, for the first time, languages such as Template Haskell. The calculus is designed such that it can be gradually enhanced with the features needed to model many of the advanced features of real languages. We demonstrate this by showing how a simple, staged type system as found in Template Haskell can be added to the calculus

Evolving a DSL implementation

Domain Specific Languages (DSLs) are small languages designed for use in a specific domain. DSLs typically evolve quite radically throughout their lifetime, but current DSL implementation approaches are often clumsy in the face of such evolution. In this paper I present a case study of an DSL evolving in its syntax, semantics, and robustness, implemented in the Converge language. This shows how real-world DSL implementations can evolve along with changing requirements

Doctor of Philosophy

dissertationI present the design of a parser that adds Scheme-style language extensibility to languages with implicitly delimited and infix syntax. A key element of my design is an enforestation parsing step, which converts a flat stream of tokens into an S-expression-like tree, in addition to the initial "read" phase of parsing and interleaved with the "macro-expand" phase. My parser uses standard lexical scoping rules to communicate syntactic extensions to the parser. In this way extensions naturally compose locally as well as through module boundaries. I argue that this style of communication is better suited towards a useful extension system than tools not directly integrated with the compiler. This dissertation explores the limits of this design in a new language called Honu. I use the extensiblity provided by Honu to develop useful language extensions such as LINQ and a parser generator. I also demonstrate the generality of the parsing techniques by applying them to Java and Python

Compile-time meta-programming in converge.

Compile-time meta-programming allows programs to be constructed by the user at compile-time. Few modern languages are capable of compile-time meta-programming, and of those that do, many of the most powerful are statically typed functional languages. In this paper I present the dynamically typed, object orientated language Converge which allows compile-time meta-programming in the spirit of Template Haskell. Converge demonstrates that integrating powerful, safe compiletime meta-programming features into a dynamic language requires few restrictions to the exible development style facilitated by the paradigm