Stream Processing using Grammars and Regular Expressions

In this dissertation we study regular expression based parsing and the use of grammatical specifications for the synthesis of fast, streaming string-processing programs. In the first part we develop two linear-time algorithms for regular expression based parsing with Perl-style greedy disambiguation. The first algorithm operates in two passes in a semi-streaming fashion, using a constant amount of working memory and an auxiliary tape storage which is written in the first pass and consumed by the second. The second algorithm is a single-pass and optimally streaming algorithm which outputs as much of the parse tree as is semantically possible based on the input prefix read so far, and resorts to buffering as many symbols as is required to resolve the next choice. Optimality is obtained by performing a PSPACE-complete pre-analysis on the regular expression. In the second part we present Kleenex, a language for expressing high-performance streaming string processing programs as regular grammars with embedded semantic actions, and its compilation to streaming string transducers with worst-case linear-time performance. Its underlying theory is based on transducer decomposition into oracle and action machines, and a finite-state specialization of the streaming parsing algorithm presented in the first part. In the second part we also develop a new linear-time streaming parsing algorithm for parsing expression grammars (PEG) which generalizes the regular grammars of Kleenex. The algorithm is based on a bottom-up tabulation algorithm reformulated using least fixed points and evaluated using an instance of the chaotic iteration scheme by Cousot and Cousot

Behavioural Types: from Theory to Tools

This book presents research produced by members of COST Action IC1201: Behavioural Types for Reliable Large-Scale Software Systems (BETTY), a European research network that was funded from October 2012 to October 2016. The technical theme of BETTY was the use of behavioural type systems in programming languages, to specify and verify properties of programs beyond the traditional use of type systems to describe data processing. A significant area within behavioural types is session types, which concerns the use of type-theoretic techniques to describe communication protocols so that static typechecking or dynamic monitoring can verify that protocols are implemented correctly. This is closely related to the topic of choreography, in which system design starts from a description of the overall communication flows. Another area is behavioural contracts, which describe the obligations of interacting agents in a way that enables blame to be attributed to the agent responsible for failed interaction. Type-theoretic techniques can also be used to analyse potential deadlocks due to cyclic dependencies between inter-process interactions. BETTY was organised into four Working Groups: (1) Foundations; (2) Security; (3) Programming Languages; (4) Tools and Applications. Working Groups 1–3 produced “state-of-the-art reports”, which originally intended to take snapshots of the field at the time the network started, but grew into substantial survey articles including much research carried out during the network [1–3]. The situation for Working Group 4 was different. When the network started, the community had produced relatively few implementations of programming languages or tools. One of the aims of the network was to encourage more implementation work, and this was a great success. The community as a whole has developed a greater interest in putting theoretical ideas into practice. The sixteen chapters in this book describe systems that were either completely developed, or substantially extended, during BETTY. The total of 41 co-authors represents a significant proportion of the active participants in the network (around 120 people who attended at least one meeting). The book is a report on the new state of the art created by BETTY in xv xvi Preface the area of Working Group 4, and the title “Behavioural Types: from Theory to Tools” summarises the trajectory of the community during the last four years. The book begins with two tutorials by Atzei et al. on contract-oriented design of distributed systems. Chapter 1 introduces the CO2 contract specifi- cation language and the Diogenes toolchain. Chapter 2 describes how timing constraints can be incorporated into the framework and checked with the CO2 middleware. Part of the CO2 middleware is a monitoring system, and the theme of monitoring continues in the next two chapters. In Chapter 3, Attard et al. present detectEr, a runtime monitoring tool for Erlang programs that allows correctness properties to be expressed in Hennessy-Milner logic. In Chapter 4, which is the first chapter about session types, Neykova and Yoshida describe a runtime verification framework for Python programs. Communication protocols are specified in the Scribble language, which is based on multiparty session types. The next three chapters deal with choreographic programming. In Chap- ter 5, Debois and Hildebrandt present a toolset for working with dynamic condition response (DCR) graphs, which are a graphical formalism for choreography. Chapter 6, by Lange et al., continues the graphical theme with ChorGram, a tool for synthesising global graphical choreographies from collections of communicating finite-state automata. Giallorenzo et al., in Chapter 7, consider runtime adaptation. They describe AIOCJ, a choreographic programming language in which runtime adaptation is supported with a guarantee that it doesn’t introduce deadlocks or races. Deadlock analysis is important in other settings too, and there are two more chapters about it. In Chapter 8, Padovani describes the Hypha tool, which uses a type-based approach to check deadlock-freedom and lock-freedom of systems modelled in a form of pi-calculus. In Chapter 9, Garcia and Laneve present a tool for analysing deadlocks in Java programs; this tool, called JaDA, is based on a behavioural type system. The next three chapters report on projects that have added session types to functional programming languages in order to support typechecking of communication-based code. In Chapter 10, Orchard and Yoshida describe an implementation of session types in Haskell, and survey several approaches to typechecking the linearity conditions required for safe session implemen- tation. In Chapter 11, Melgratti and Padovani describe an implementation of session types in OCaml. Their system uses runtime linearity checking. In Chapter 12, Lindley and Morris describe an extension of the web programming language Links with session types; their work contrasts with the previous two chapters in being less constrained by an existing language design. Continuing the theme of session types in programming languages, the next two chapters describe two approaches based on Java. Hu’s work, presented in Chapter 13, starts with the Scribble description of a multiparty session type and generates an API in the form of a collection of Java classes, each class containing the communication methods that are available in a particular state of the protocol. Dardha et al., in Chapter 14, also start with a Scribble specification. Their StMungo tool generates an API as a single class with an associated typestate specification to constrain sequences of method calls. Code that uses the API can be checked for correctness with the Mungo typechecker. Finally, there are two chapters about programming with the MPI libraries. Chapter 15, by Ng and Yoshida, uses an extension of Scribble, called Pabble, to describe protocols that parametric in the number of runtime roles. From a Pabble specification they generate C code that uses MPI for communication and is guaranteed correct by construction. Chapter 16, by Ng et al., describes the ParTypes framework for analysing existing C+MPI programs with respect to protocols defined in an extension of Scribble. We hope that the book will serve a useful purpose as a report on the activities of COST Action IC1201 and as a survey of programming languages and tools based on behavioural types

Behavioural Types: from Theory to Tools

This book presents research produced by members of COST Action IC1201: Behavioural Types for Reliable Large-Scale Software Systems (BETTY), a European research network that was funded from October 2012 to October 2016. The technical theme of BETTY was the use of behavioural type systems in programming languages, to specify and verify properties of programs beyond the traditional use of type systems to describe data processing. A significant area within behavioural types is session types, which concerns the use of type-theoretic techniques to describe communication protocols so that static typechecking or dynamic monitoring can verify that protocols are implemented correctly. This is closely related to the topic of choreography, in which system design starts from a description of the overall communication flows. Another area is behavioural contracts, which describe the obligations of interacting agents in a way that enables blame to be attributed to the agent responsible for failed interaction. Type-theoretic techniques can also be used to analyse potential deadlocks due to cyclic dependencies between inter-process interactions. BETTY was organised into four Working Groups: (1) Foundations; (2) Security; (3) Programming Languages; (4) Tools and Applications. Working Groups 1–3 produced “state-of-the-art reports”, which originally intended to take snapshots of the field at the time the network started, but grew into substantial survey articles including much research carried out during the network [1–3]. The situation for Working Group 4 was different. When the network started, the community had produced relatively few implementations of programming languages or tools. One of the aims of the network was to encourage more implementation work, and this was a great success. The community as a whole has developed a greater interest in putting theoretical ideas into practice. The sixteen chapters in this book describe systems that were either completely developed, or substantially extended, during BETTY. The total of 41 co-authors represents a significant proportion of the active participants in the network (around 120 people who attended at least one meeting). The book is a report on the new state of the art created by BETTY in xv xvi Preface the area of Working Group 4, and the title “Behavioural Types: from Theory to Tools” summarises the trajectory of the community during the last four years. The book begins with two tutorials by Atzei et al. on contract-oriented design of distributed systems. Chapter 1 introduces the CO2 contract specifi- cation language and the Diogenes toolchain. Chapter 2 describes how timing constraints can be incorporated into the framework and checked with the CO2 middleware. Part of the CO2 middleware is a monitoring system, and the theme of monitoring continues in the next two chapters. In Chapter 3, Attard et al. present detectEr, a runtime monitoring tool for Erlang programs that allows correctness properties to be expressed in Hennessy-Milner logic. In Chapter 4, which is the first chapter about session types, Neykova and Yoshida describe a runtime verification framework for Python programs. Communication protocols are specified in the Scribble language, which is based on multiparty session types. The next three chapters deal with choreographic programming. In Chap- ter 5, Debois and Hildebrandt present a toolset for working with dynamic condition response (DCR) graphs, which are a graphical formalism for choreography. Chapter 6, by Lange et al., continues the graphical theme with ChorGram, a tool for synthesising global graphical choreographies from collections of communicating finite-state automata. Giallorenzo et al., in Chapter 7, consider runtime adaptation. They describe AIOCJ, a choreographic programming language in which runtime adaptation is supported with a guarantee that it doesn’t introduce deadlocks or races. Deadlock analysis is important in other settings too, and there are two more chapters about it. In Chapter 8, Padovani describes the Hypha tool, which uses a type-based approach to check deadlock-freedom and lock-freedom of systems modelled in a form of pi-calculus. In Chapter 9, Garcia and Laneve present a tool for analysing deadlocks in Java programs; this tool, called JaDA, is based on a behavioural type system. The next three chapters report on projects that have added session types to functional programming languages in order to support typechecking of communication-based code. In Chapter 10, Orchard and Yoshida describe an implementation of session types in Haskell, and survey several approaches to typechecking the linearity conditions required for safe session implemen- tation. In Chapter 11, Melgratti and Padovani describe an implementation of session types in OCaml. Their system uses runtime linearity checking. In Chapter 12, Lindley and Morris describe an extension of the web programming language Links with session types; their work contrasts with the previous two chapters in being less constrained by an existing language design. Continuing the theme of session types in programming languages, the next two chapters describe two approaches based on Java. Hu’s work, presented in Chapter 13, starts with the Scribble description of a multiparty session type and generates an API in the form of a collection of Java classes, each class containing the communication methods that are available in a particular state of the protocol. Dardha et al., in Chapter 14, also start with a Scribble specification. Their StMungo tool generates an API as a single class with an associated typestate specification to constrain sequences of method calls. Code that uses the API can be checked for correctness with the Mungo typechecker. Finally, there are two chapters about programming with the MPI libraries. Chapter 15, by Ng and Yoshida, uses an extension of Scribble, called Pabble, to describe protocols that parametric in the number of runtime roles. From a Pabble specification they generate C code that uses MPI for communication and is guaranteed correct by construction. Chapter 16, by Ng et al., describes the ParTypes framework for analysing existing C+MPI programs with respect to protocols defined in an extension of Scribble. We hope that the book will serve a useful purpose as a report on the activities of COST Action IC1201 and as a survey of programming languages and tools based on behavioural types

Extending the Finite Domain Solver of GNU Prolog

International audienceThis paper describes three significant extensions for the Finite Domain solver of GNU Prolog. First, the solver now supports negative integers. Second, the solver detects and prevents integer overflows from occurring. Third, the internal representation of sparse domains has been redesigned to overcome its current limitations. The preliminary performance evaluation shows a limited slowdown factor with respect to the initial solver. This factor is widely counterbalanced by the new possibilities and the robustness of the solver. Furthermore these results are preliminary and we propose some directions to limit this overhead

Implementing a Functional Logic Programming Language via the Fair Scheme

This document presents a new compiler for the Functional Logic programming language Curry based on a novel pull-tabbing evaluation strategy called the Fair Scheme. A simple version of the Fair Scheme is proven sound, complete, and optimal. An elaborated version is also developed, which supports narrowing computations and other features of Curry, such as constraint programming, equational constraints, and set functions. The Fair Scheme is used to develop a new Curry system called Sprite, a high-quality, performant implementation whose aims are to promote practical uses of Curry and to serve as a laboratory for further research. An important aspect of Sprite is its integration with the popular imperative language Python. This combination allows one to write hybrid programs in which the programmer may move between declarative and non-declarative styles with relative ease. Benchmarking data show Sprite to be more complete than other Curry systems and competitive in terms of execution time, particularly for non-deterministic programs

Programming with Specifications

This thesis explores the use of specifications for the construction of correct programs. We go beyond their standard use as run-time assertions, and present algorithms, techniques and implementations for the tasks of 1) program verification, 2) declarative programming and 3) software synthesis. These results are made possible by our advances in the domains of decision procedure design and implementation. In the first part of this thesis, we present a decidability result for a class of logics that support user-defined recursive function definitions. Constraints in this class can encode expressive properties of recursive data structures, such as sortedness of a list, or balancing of a search tree. As a result, complex verification conditions can be stated concisely and solved entirely automatically. We also present a new decision procedure for a logic to reason about sets and constraints over their cardinalities. The key insight lies in a technique to decompose con- straints according to mutual dependencies. Compared to previous techniques, our algorithm brings significant improvements in running times, and for the first time integrates reasoning about cardinalities within the popular DPLL(T ) setting. We integrated our algorithmic ad- vances into Leon, a static analyzer for functional programs. Leon can reason about constraints involving arbitrary recursive function definitions, and has the desirable theoretical property that it will always find counter-examples to assertions that do not hold. We illustrate the flexibility and efficiency of Leon through experimental evaluation, where we used it to prove detailed correctness properties of data structure implementations. We then illustrate how program specifications can be used as a high-level programming construct ; we present Kaplan, an extension of Scala with first-class logical constraints. Kaplan allows programmers to create, manipulate and combine constraints as they would any other data structure. Our implementation of Kaplan illustrates how declarative programming can be incorporated into an existing mainstream programming language. Moreover, we examine techniques to transform, at compile-time, program specifications into efficient executable code. This approach of software synthesis combines the correctness benefits of declarative programming with the efficiency of imperative or functional programming

Mechanising and evolving the formal semantics of WebAssembly: the Web's new low-level language

WebAssembly is the first new programming language to be supported natively by all major Web browsers since JavaScript. It is designed to be a natural low-level compilation target for languages such as C, C++, and Rust, enabling programs written in these languages to be compiled and executed efficiently on the Web. WebAssembly’s specification is managed by the W3C WebAssembly Working Group (made up of representatives from a number of major tech companies). Uniquely, the language is specified by way of a full pen-and-paper formal semantics. This thesis describes a number of ways in which I have both helped to shape the specification of WebAssembly, and built upon it. By mechanising the WebAssembly formal semantics in Isabelle/HOL while it was being drafted, I discovered a number of errors in the specification, drove the adoption of official corrections, and provided the first type soundness proof for the corrected language. This thesis also details a verified type checker and interpreter, and a security type system extension for cryptography primitives, all of which have been mechanised as extensions of my initial WebAssembly mechanisation. A major component of the thesis is my work on the specification of shared memory concurrency in Web languages: correcting and verifying properties of JavaScript’s existing relaxed memory model, and defining the WebAssembly-specific extensions to the corrected model which have been adopted as the basis of WebAssembly’s official threads specification. A number of deficiencies in the original JavaScript model are detailed. Some errors have been corrected, with the verified fixes officially adopted into subsequent editions of the language specification. However one discovered deficiency is fundamental to the model, an instance of the well-known "thin-air problem". My work demonstrates the value of formalisation and mechanisation in industrial programming language design, not only in discovering and correcting specification errors, but also in building confidence both in the correctness of the language’s design and in the design of proposed extensions.2019 Google PhD Fellowship in Programming Technology and Software Engineering Peterhouse Research Fellowshi

SusTrainable: Promoting Sustainability as a Fundamental Driver in Software Development Training and Education. 2nd Teacher Training, January 23-27, 2023, Pula, Croatia. Revised lecture notes

This volume exhibits the revised lecture notes of the 2nd teacher training organized as part of the project Promoting Sustainability as a Fundamental Driver in Software Development Training and Education, held at the Juraj Dobrila University of Pula, Croatia, in the week January 23-27, 2023. It is the Erasmus+ project No. 2020-1-PT01-KA203-078646 - Sustrainable. More details can be found at the project web site https://sustrainable.github.io/ One of the most important contributions of the project are two summer schools. The 2nd SusTrainable Summer School (SusTrainable - 23) will be organized at the University of Coimbra, Portugal, in the week July 10-14, 2023. The summer school will consist of lectures and practical work for master and PhD students in computing science and closely related fields. There will be contributions from Babe\c{s}-Bolyai University, E\"{o}tv\"{o}s Lor\'{a}nd University, Juraj Dobrila University of Pula, Radboud University Nijmegen, Roskilde University, Technical University of Ko\v{s}ice, University of Amsterdam, University of Coimbra, University of Minho, University of Plovdiv, University of Porto, University of Rijeka. To prepare and streamline the summer school, the consortium organized a teacher training in Pula, Croatia. This was an event of five full days, organized by Tihana Galinac Grbac and Neven Grbac. The Juraj Dobrila University of Pula is very concerned with the sustainability issues. The education, research and management are conducted with sustainability goals in mind. The contributions in the proceedings were reviewed and provide a good overview of the range of topics that will be covered at the summer school. The papers in the proceedings, as well as the very constructive and cooperative teacher training, guarantee the highest quality and beneficial summer school for all participants.Comment: 85 pages, 8 figures, 3 code listings and 1 table; editors: Tihana Galinac Grbac, Csaba Szab\'{o}, Jo\~{a}o Paulo Fernande

Dependent Object Types

A scalable programming language is one in which the same concepts can describe small as well as large parts. Towards this goal, Scala unifies concepts from object and module systems. In particular, objects can contain type members, which can be selected as types, called path-dependent types. Focusing on path-dependent types, we develop a type-theoretic foundation for Scala: the calculus of Dependent Object Types (DOT). We derive DOT from System F, we add a lower bound to each type variable, in addition to its usual upper bound, (2) in System D, we turn each type variable into a regular term variable containing a type, (3) for a full subtyping lattice, we add intersection and union types, (4) for objects, we consolidate all values into records, (5) for objects that close over a self, we introduce a recursive type, binding a self term variable, (6) for recursive types, we first extend the theory in typing and then also in subtyping. Through this bottom-up exploration, we discover a sound, uniform yet powerful design for DOT. We devise strategies and techniques for proving soundness that scale through this iterative step-by-step process: (1) "pushback" of subtyping transitivity or subsumption, to concisely capture inversion of subtyping or typing, (2) distinction between concrete vs. abstract context variables, to resolve tension between preservation of types vs. preservation of type abstractions, (3) and, specifically for big-step semantics, a type that closes over an environment, to relate context-dependent types across closures. While ultimately, we have developed sound models of DOT in both big-step and small-step operational semantics, historically, the shift to big-step semantics has been helpful in focusing the requirements. In particular, by developing a novel big-step soundness proof for System F<:, calculi like System D<: emerge as straightforward generalizations, almost like removing artificial restrictions. Interesting in their own right, our type soundness techniques for definitional interpreters extend to mutable references without use of co-induction. The DOT calculus finally grounds languages like Scala in firm theory. The DOT calculus helps in finding bugs in Scala, and in understanding feature interaction better as well as requirements. The DOT calculus serves as a good basis for future work which studies extensions or encodings on top of the core, bridging the gap from DOT to Dotty / Scala