Reflections on metaprogramming

Journal ArticleThe spread of object-oriented technology has led to object-oriented programming languages with object-oriented implementations. By encapsulating part of t h e semantics of a language within a set of default classes and empowering the programmer to derive new versions of these base classes, a designer can provide a language whose semantics can be tailored by individual programmers. The degree to which such languages are simultaneously flexible and efficient is an open question. We address this question by reporting our experience with using this technique to incorporate support for persistence into the Common Lisp Object System via its metaobject protocol. For many aspects of our implementation we found that the metaobject protocol was perfectly suitable. In other cases we had to variously extend the protocol, pay an unacceptable performance penalty, or modify the language implementation directly. Based on our experience we propose some improvements to the protocol. We also present some performance measurements that reveal the need for improved language implementation techniques

Metamusing on object persistence

Journal ArticleThe need to "open up languages" has led to object-oriented programming languages with object-oriented implementations. By encapsulating the fundamental aspects of a language semantics within a set of default classes and giving t h e programmer t h e flexibility of deriving new versions of these base classes, a language whose semantics can be tailored to the needs of individual programmers can be provided. The degree of success in designing a language in this way to achieve flexibility and efficiency simultaneously is an open question. The Common Lisp Object System is designed with these techniques and we address this question by reporting our experience with the CLOS metaobject protocol in incorporating support for persistence into CLOS. For many aspects of our implementation we found that the metaobject protocol was perfectly suitable. In other cases we had to variously extend the protocol, pay an unacceptable performance penalty, or modify the language implementation directly. Based on our experience we propose some improvements to the protocol. We also present some performance measurements that reveal the need for improved language implementation techniques

Using metaprogramming to add persistence to CLOS

Journal ArticleThe need to "open up" languages and the spread of object-oriented technology have led to object-oriented programming languages with object-oriented implementations. By encapsulating the fundamental aspects of language semantics within a set of default classes and giving the programmer the flexibility of deriving new versions of these base classes, a language whose semantics can be tailored to the needs of individual programmers can be provided. The degree to which such languages are simultaneously flexible and efficient is an open question. We address this question by reporting our experience with using this technique to incorporate support for persistence into the Common Lisp Object System via its metaobject protocol. For many aspects of our implementation we found that the metaobject protocol was perfectly suitable. In other cases we had to variously extend the protocol, pay an unacceptable performance penalty, or modify the language implementation directly. Based on our experience we propose some improvements to the protocol. We also present some performance measurements that reveal the need for improved language implementation techniques

MetaBETA: Model and Implementation

Object-oriented programming languages are excellent for expressing abstractions in many application domains. The object-oriented programming methodology allows real-world concepts to modelled in an easy and direct fashion and it supports refinement of concepts. However, many object-oriented languages and their implementations fall short in two areas: dynamic extensibility and reflection.Dynamic extensibility is the ability to incorporate new classes into an application at runtime. Reflection makes it possible for a language to extend its own domain, e.g., to build type-orthogonal functionality. MetaBETA is an extension of the BETA language that supports dynamic extensibility and reflection. MetaBETA has a metalevel interface that provides access to the state of a running application and to the default implementation of language primities.This report presents the model behind MetaBETA. In particular, we discuss the execution model of a MetaBETA program and how type- orthogonal abstractions can be built. This includes precentation of dynamic slots, a mechanism that makes is possible ectend objects at runtime. The other main area covered in this report is the implementation of MetaBETA. The central component of the architecture is a runtime system, which is viewed as a virtual machine whose baselevel interface implements the functionality needed by the programming language

Bridging the Gap between Machine and Language using First-Class Building Blocks

High-performance virtual machines (VMs) are increasingly reused for programming languages for which they were not initially designed. Unfortunately, VMs are usually tailored to specific languages, offer only a very limited interface to running applications, and are closed to extensions. As a consequence, extensions required to support new languages often entail the construction of custom VMs, thus impacting reuse, compatibility and performance. Short of building a custom VM, the language designer has to choose between the expressiveness and the performance of the language. In this dissertation we argue that the best way to open the VM is to eliminate it. We present Pinocchio, a natively compiled Smalltalk, in which we identify and reify three basic building blocks for object-oriented languages. First we define a protocol for message passing similar to calling conventions, independent of the actual message lookup mechanism. The lookup is provided by a self-supporting runtime library written in Smalltalk and compiled to native code. Since it unifies the meta- and base-level we obtain a metaobject protocol (MOP). Then we decouple the language-level manipulation of state from the machine-level implementation by extending the structural reflective model of the language with object layouts, layout scopes and slots. Finally we reify behavior using AST nodes and first-class interpreters separate from the low-level language implementation. We describe the implementations of all three first-class building blocks. For each of the blocks we provide a series of examples illustrating how they enable typical extensions to the runtime, and we provide benchmarks validating the practicality of the approaches

Modular language processors as framework completions

Journal ArticleThe conceptual and specificational power of denotational semantics for programming language design has been amply demonstrated. We report here on a language implementation method that is similarly semantically motivated, but is based upon object-oriented design principles, and results in flexible and evolvable language processors. We apply this technique to the area of object-oriented (O-O) languages, in the form of a general metalevel architecture for objects and inheritance that facilitates the development of compilers and interpreters for 0-0 languages. This development strategy maintains architectural modularity by mapping conceptual language design decisions to isolatable parts of resulting language processors. Our architecture, which is presented as an OO framework, is characterized by (i) support for a broad set of modularity features including encapsulation and strong typing, and (ii) an "unbundled" view of inheritance, semantic features of which are decomposed by means of a set of module combination operations (combinators). We describe an implementation of our framework in C++, and assess its utility by constructing a compiler for a simple 0 - 0 extension to the programming language C. We further argue the flexibility of the resulting processor by outlining the incorporation of several significant extensions to the basic module language. We claim that the use of such a framework for compiler construction has many advantages, including a systematic language development method, processor software reuse, language extensibility, and potential for interoperability among languages.

Fully Reflective Execution Environments: Virtual Machines for More Flexible Software

VMs are complex pieces of software that implement programming language semantics in an efficient, portable, and secure way. Unfortunately, mainstream VMs provide applications with few mechanisms to alter execution semantics or memory management at run time. We argue that this limits the evolvability and maintainability of running systems for both, the application domain, e.g., to support unforeseen requirements, and the VM domain, e.g., to modify the organization of objects in memory. This work explores the idea of incorporating reflective capabilities into the VM domain and analyzes its impact in the context of software adaptation tasks. We characterize the notion of a fully reflective VM, a kind of VM that provides means for its own observability and modifiability at run time. This enables programming languages to adapt the underlying VM to changing requirements. We propose a reference architecture for such VMs and present TruffleMATE as a prototype for this architecture. We evaluate the mechanisms TruffleMATE provides to deal with unanticipated dynamic adaptation scenarios for security, optimization, and profiling aspects. In contrast to existing alternatives, we observe that TruffleMATE is able to handle all scenarios, using less than 50 lines of code for each, and without interfering with the application's logic

Bridging the Gap between Machine and Language using First-Class Building Blocks

High-performance virtual machines (VMs) are increasingly reused for programming languages for which they were not initially designed. Unfortunately, VMs are usually tailored to specific languages, offer only a very limited interface to running applications, and are closed to extensions. As a consequence, extensions required to support new languages often entail the construction of custom VMs, thus impacting reuse, compatibility and performance. Short of building a custom VM, the language designer has to choose between the expressiveness and the performance of the language. In this dissertation we argue that the best way to open the VM is to eliminate it. We present Pinocchio, a natively compiled Smalltalk, in which we identify and reify three basic building blocks for object-oriented languages. First we define a protocol for message passing similar to calling conventions, independent of the actual message lookup mechanism. The lookup is provided by a self-supporting runtime library written in Smalltalk and compiled to native code. Since it unifies the meta- and base-level we obtain a metaobject protocol (MOP). Then we decouple the language-level manipulation of state from the machine-level implementation by extending the structural reflective model of the language with object layouts, layout scopes and slots. Finally we reify behavior using AST nodes and first-class interpreters separate from the low-level language implementation. We describe the implementations of all three first-class building blocks. For each of the blocks we provide a series of examples illustrating how they enable typical extensions to the runtime, and we provide benchmarks validating the practicality of the approaches

Zero-Overhead Metaprogramming: Reflection and Metaobject Protocols Fast and without Compromises

Runtime metaprogramming enables many useful applications and is often a convenient solution to solve problems in a generic way, which makes it widely used in frameworks, middleware, and domain-specific languages. However, powerful metaobject protocols are rarely supported and even common concepts such as reflective method invocation or dynamic proxies are not optimized. Solutions proposed in literature either restrict the metaprogramming capabilities or require application or library developers to apply performance improving techniques. For overhead-free runtime metaprogramming, we demonstrate that dispatch chains, a generalized form of polymorphic inline caches common to self-optimizing interpreters, are a simple optimization at the language-implementation level. Our evaluation with self-optimizing interpreters shows that unrestricted metaobject protocols can be realized for the first time without runtime overhead, and that this optimization is applicable for just-in-time compilation of interpreters based on meta-tracing as well as partial evaluation. In this context, we also demonstrate that optimizing common reflective operations can lead to significant performance improvements for existing applications