Static Computation and Reflection

Thesis (PhD) - Indiana University, Computer Sciences, 2008Most programming languages do not allow programs to inspect their static type information or perform computations on it. C++, however, lets programmers write template metaprograms, which enable programs to encode static information, perform compile-time computations, and make static decisions about run-time behavior. Many C++ libraries and applications use template metaprogramming to build specialized abstraction mechanisms, implement domain-specific safety checks, and improve run-time performance. Template metaprogramming is an emergent capability of the C++ type system, and the C++ language specification is informal and imprecise. As a result, template metaprogramming often involves heroic programming feats and often leads to code that is difficult to read and maintain. Furthermore, many template-based code generation and optimization techniques rely on particular compiler implementations, rather than language semantics, for performance gains. Motivated by the capabilities and techniques of C++ template metaprogramming, this thesis documents some common programming patterns, including static computation, type analysis, generative programming, and the encoding of domain-specific static checks. It also documents notable shortcomings to current practice, including limited support for reflection, semantic ambiguity, and other issues that arise from the pioneering nature of template metaprogramming. Finally, this thesis presents the design of a foundational programming language, motivated by the analysis of template metaprogramming, that allows programs to statically inspect type information, perform computations, and generate code. The language is specified as a core calculus and its capabilities are presented in an idealized setting

Relay: A New IR for Machine Learning Frameworks

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet

Liberating Composition from Language Dictatorship

Historically, programming languages have been—although benevolent—dictators: fixing a lot of semantics into built-in language constructs. Over the years, (some) programming languages have freed the programmers from restrictions to use only built-in libraries, built-in data types, or built-in type checking rules. Even though, arguably, such freedom could lead to anarchy, or people shooting themselves in the foot, the contrary tends to be the case: a language that does not allow for extensibility, is depriving software engineers from the ability to construct proper abstractions and to structure software in the most optimal way. Instead, the software becomes less structured and maintainable than would be possible if the software engineer could express the behavior of the program with the most appropriate abstractions. The new idea proposed by this paper is to move composition from built-in language constructs to programmable, first-class abstractions in the language. As an emerging result, we present the Co-op concept of a language, which shows that it is possible with a relatively simple model to express a wide range of compositions as first-class concepts

Multi-stage languages in hardware design

As circuits increase in size and complexity, hardware description techniques have been trying to adopt features already well- established in software languages. In this paper, we investigate how different hardware description languages implement levels of abstraction over the hardware designs, and we examine how improvements have lead to features like parameterised circuits and generic descriptions, that enable users to efficiently model and reason about large regular-shaped structures and connection patterns. Nonetheless, the ability to include non-functional properties of circuits in the same description is still an open issue. Lately, proposed solutions are looking into meta-functional languages and multi-staging techniques. We examine how hardware description languages can benefit from the capabilities of meta-functional languages, which are able to reason about, and transform the circuit generators as data objects, thus providing a means to access both the functional and non-functional aspects of the generated circuits.peer-reviewe