Implementing Higher-Kinded Types in Dotty

dotty is a new, experimental Scala compiler based on DOT, the calculus of Dependent Object Types. Higher-kinded types are a natural extension of first-order lambda calculus, and have been a core construct of Haskell and Scala. As long as such types are just partial applications of generic classes, they can be given a meaning in DOT relatively straightforwardly. But general lambdas on the type level require extensions of the DOT calculus to be expressible. This paper is an experience report where we describe and discuss four implementation strategies that we have tried out in the last three years. Each strategy was fully implemented in the dotty compiler. We discuss the usability and expressive power of each scheme, and give some indications about the amount of implementation difficulties encountered

Generic programming using Higher Kinded Data

This thesis describes datatype-generic programming, what it is, and how it is done in Scala. The thesis covers ways of thinking about datatype-generic programming and today's tools and libraries for datatype-generic programming in Scala and summarize how to use them. A new library called perspective for datatype-generic programming with higher kinded data is presented. How these libraries, including perspective, work together with sum types is be covered. Benchmarks are presented on both runtime and compile time performance of perspective and other libraries. perspective manages to keep up in performance with other libraries while offering faster code at the expense of longer compile time if a developer opts into this. Finally, an example language-integrated query library built on perspective is described.Masteroppgave i informatikkINF399MAMN-INFMAMN-PRO

A Theory of Higher-Order Subtyping with Type Intervals (Extended Version)

The calculus of Dependent Object Types (DOT) has enabled a more principled and robust implementation of Scala, but its support for type-level computation has proven insufficient. As a remedy, we propose $F^\omega_{..}$, a rigorous theoretical foundation for Scala's higher-kinded types. $F^\omega_{..}$ extends $F^\omega_{<:}$ with interval kinds, which afford a unified treatment of important type- and kind-level abstraction mechanisms found in Scala, such as bounded quantification, bounded operator abstractions, translucent type definitions and first-class subtyping constraints. The result is a flexible and general theory of higher-order subtyping. We prove type and kind safety of $F^\omega_{..}$, as well as weak normalization of types and undecidability of subtyping. All our proofs are mechanized in Agda using a fully syntactic approach based on hereditary substitution.Comment: 73 pages; to be presented at the 26th ACM SIGPLAN International Conference on Functional Programming (ICFP 2021), 22-27 August 202

Higher-Order Subtyping with Type Intervals

Modern, statically typed programming languages provide various abstraction facilities at both the term- and type-level. Common abstraction mechanisms for types include parametric polymorphism -- a hallmark of functional languages -- and subtyping -- which is pervasive in object-oriented languages. Additionally, both kinds of languages may allow parametrized (or generic) datatype definitions in modules or classes. When several of these features are present in the same language, new and more expressive combinations arise, such as (1) bounded quantification, (2) bounded operator abstractions and (3) translucent type definitions. An example of such a language is Scala, which features all three of the aforementioned type-level constructs. This increases the expressivity of the language, but also the complexity of its type system. From a theoretical point of view, the various abstraction mechanisms have been studied through different extensions of Girard's higher-order polymorphic lambda-calculus F-omega. Higher-order subtyping and bounded polymorphism (1 and 2) have been formalized in F-omega-sub and its many variants; type definitions of various degrees of opacity (3) have been formalized through extensions of F-omega with singleton types. In this dissertation, I propose type intervals as a unifying concept for expressing (1--3) and other related constructs. In particular, I develop an extension of F-omega with interval kinds as a formal theory of higher-order subtyping with type intervals, and show how the familiar concepts of higher-order bounded quantification, bounded operator abstraction and singleton kinds can all be encoded in a semantics-preserving way using interval kinds. Going beyond the status quo, the theory is expressive enough to also cover less familiar constructs, such as lower-bounded operator abstractions and first-class, higher-order inequality constraints. I establish basic metatheoretic properties of the theory: I prove that subject reduction holds for well-kinded types w.r.t. full beta-reduction, that types and kinds are weakly normalizing, and that the theory is type safe w.r.t. its call-by-value operational reduction semantics. Key to this metatheoretic development is the use of hereditary substitution and the definition of an equivalent, canonical presentation of subtyping, which involves only normal types and kinds. The resulting metatheory is entirely syntactic, i.e. does not involve any model constructions, and has been fully mechanized in Agda. The extension of F-omega with interval kinds constitutes a stepping stone to the development of a higher-order version of the calculus of Dependent Object Types (DOT) -- the theoretical foundation of Scala's type system. In the last part of this dissertation, I briefly sketch a possible extension of the theory toward this goal and discuss some of the challenges involved in adapting the existing metatheory to that extension

Scala with Explicit Nulls

The Scala programming language unifies the object-oriented and functional styles of programming. One common source of errors in Scala programs is null references. In this dissertation, I present a modification to the Scala type system that makes nullability explicit in the types. This allows us to turn runtime errors into compile-time errors. I have implemented this design for explicit nulls as a fork of the Dotty (Scala 3) compiler. I evaluate the design by migrating a number of Scala libraries to use explicit nulls. In the second part of the dissertation, I give a theoretical foundation for explicit nulls. I do this in two, independent ways. First, I give a denotational semantics for type nullification, a key part of the explicit nulls design. Separately, I present a core calculus for null interoperability that models how languages with explicit nulls (like Scala) interact with languages where null remains implicit (like Java). Using the concept of blame from gradual typing, I show that if a well-typed program fails with certain kinds of nullability errors, an implicitly-nullable subterm can always be blamed for the failure

Choral: Object-Oriented Choreographic Programming

We present Choral, the first choreographic programming language based on mainstream abstractions. The key idea in Choral is a new notion of data type, which allows for expressing that data is distributed over different roles. We use this idea to reconstruct the paradigm of choreographic programming through object-oriented abstractions. Choreographies are classes, and instances of choreographies are objects with states and behaviours implemented collaboratively by roles. Choral comes with a compiler that, given a choreography, generates an implementation for each of its roles. These implementations are libraries in pure Java, whose types are under the control of the Choral programmer. Developers can then modularly compose these libraries in their own programs, in order to participate correctly in choreographies. Choral is the first incarnation of choreographic programming offering such modularity, which finally connects more than a decade of research on the paradigm to practical software development. The integration of choreographic and object-oriented programming yields other powerful advantages, where the features of one paradigm benefit the other in ways that go beyond the sum of the parts. The high-level abstractions and static checks from the world of choreographies can be used to write concurrent and distributed object-oriented software more concisely and correctly. We obtain a much more expressive choreographic language from object-oriented abstractions than in previous work. For example, object passing makes Choral the first higher-order choreographic programming language, whereby choreographies can be parameterised over other choreographies without any need for central coordination. Together with subtyping and generics, this allows Choral to elegantly support user-defined communication mechanisms and middleware

Design and implementation of a scalable domain specific language foundation for ScaFi with Scala 3

In the field of macroprogramming, one of the prominent engineering techniques is aggregate computing, which allows the definition of the overall behavior of a network of devices or agents through a single program, called the aggregate program. ScaFi is an aggregate programming framework that comprises an internal Domain Specific Language (DSL) written in Scala 2 with supporting components for simulation, visualization, and execution of aggregate systems, based on the Field Calculus formal language and computational model. Recently, a new formal language and computational model called Exchange Calculus has been proposed, extending the expressiveness of Field Calculus while simplifying its set of primitive constructs. In the meantime, Scala 2 has been succeeded by Scala 3, which introduces a relevant set of new features and improvements over its predecessor. These two novelties have promoted the development of a new ScaFi, called ScaFi-XC, entirely redesigned to be formally based on Exchange Calculus and to make the best use of its new host language, Scala 3, enhancing the original ScaFi with a more expressive and flexible DSL. This thesis presents the design and implementation of ScaFi-XC, focusing on the core DSL and associated components such as the engine and the simulator. In conclusion, ScaFi-XC has been successfully implemented and tested, delivering the expected results and fulfilling the requirements of the stakeholders, which are mainly the researchers and developers of the original ScaFi. Nevertheless, there is still work to be done to have a complete and stable version of the framework, such as porting all the support modules and improving the simulator

A Path to DOT: Formalizing Scala with Dependent Object Types

The goal of my thesis is to enable formal reasoning about the Scala programming language. To that end I present a core calculus that formalizes Scala's i) essential features in a ii) type-safe way and is iii) easy to extend with more features. I build on the Dependent Object Types (DOT) calculus that formalizes path-dependent types. My contributions are i) a generalization of DOT with types that depend on paths of arbitrary length, ii) a simple, extensible type-safety proof for DOT, and iii) an extension of DOT with mutable references. The simple proof makes designing smaller extensions such as mutation straightforward, and larger extensions, such as full support for paths, approachable. Adding fully path-dependent types to DOT allows us to model the key feature of Scala's type and module system. The calculi and proofs presented in my thesis are fully mechanized in Coq

Design and implementation of an optimizing type-centric compiler for a high-level language

Production compilers for programming languages face multiple requirements. They should be correct, as we rely on them to produce code. They should be fast, in order to provide a good developer experience. They should also be easy to maintain and evolve. This thesis shows how an expressive high level type system can be used to simplify the development of a compiler and demonstrates this on a compiler for Scala. First, it shows how expressive types of high level languages can be used to build internal data structures that provide a statically checked API, ensuring that important properties hold at compile time. Second, we also show how high level language features can be used to abstract the components of a compiler. We demonstrate this by introducing a type-safe layer on top of the bytecode emission phase. This makes it possible to abstract away the implementation details of the compiler frontend and run the same bytecode emission phase in two different Scala compilers. Third, it presents MiniPhases, a novel way to organize transformation passes in a compiler. MiniPhases impose constraints on the organization of passes that are beneficial for maintainability, performance, and testability. We include a detailed performance evaluation of MiniPhases which indicates that their speedup is due to improved cache friendliness and to a lower rate of promotions of objects into the old generations of garbage collectors. Finally, we demonstrate how the expressive type system of the language being compiled can be used for static analysis. We present a novel call graph construction algorithm which uses the typing context for context sensitivity. The resulting algorithm is both substantially faster and more precise than existing alternatives. We demonstrate the applicability of this analysis by extending common subexpression elimination to idempotent expression elimination

Towards Strong Normalization for Dependent Object Types (DOT)

The Dependent Object Types (DOT) family of calculi has been proposed as a new theoretic foundation for Scala and similar languages, unifying functional programming, object oriented programming and ML-style module systems. Following the recent type soundness proof for DOT, the present paper aims to establish stronger meta-theoretic properties. The main result is a fully mechanized proof of strong normalization for D_<:, a variant of DOT that excludes recursive functions and recursive types. We further discuss techniques and challenges for adding recursive types while maintaining strong normalization, and demonstrate that certain variants of recursive self types can be integrated successfully