Discourje: Runtime verification of communication protocols in clojure

This paper presents Discourje: a runtime verification framework for communication protocols in Clojure. Discourje guarantees safety of protocol implementations relative to specifications, based on an expressive new version of multiparty session types. The framework has a formal foundation and is itself implemented in Clojure to offer a seamless specification–implementation experience. Benchmarks show Discourje’s overhead can be less than 5% for real/existing concurrent programs

Understanding and Evaluating Dynamic Race Detection with Go

With the prevalence of concurrent software and the difficulties that come with properly synchronizing threads, it is easy to introduce data races into programs. A data race is a software bug which occurs when at least two threads simultaneously access shared memory, with at least one of them performing a write. Because of the indeterministic nature of thread scheduling, data races lead to undefined behavior which may only manifest in rare thread schedules. The difficulty lies in detecting hidden data races based on observed program executions. Go (Golang) is a modern, open source programming language designed for simplicity, efficiency, and reliability. With the introduction of Goroutines and other primitives, Go enables quick development of highly concurrent software. However, this model makes it necessary to apply race detection to programs written in Go. We present an in-depth study of the Go race detector, a dynamic analysis tool developed by Google. The Go race detector traces program events and provides detailed information regarding detected races. Furthermore, we evaluate the race detector on Kubernetes, the most popular open source project written in Go

Understanding Concurrency Vulnerabilities in Linux Kernel

While there is a large body of work on analyzing concurrency related software bugs and developing techniques for detecting and patching them, little attention has been given to concurrency related security vulnerabilities. The two are different in that not all bugs are vulnerabilities: for a bug to be exploitable, there needs be a way for attackers to trigger its execution and cause damage, e.g., by revealing sensitive data or running malicious code. To fill the gap, we conduct the first empirical study of concurrency vulnerabilities reported in the Linux operating system in the past ten years. We focus on analyzing the confirmed vulnerabilities archived in the Common Vulnerabilities and Exposures (CVE) database, which are then categorized into different groups based on bug types, exploit patterns, and patch strategies adopted by developers. We use code snippets to illustrate individual vulnerability types and patch strategies. We also use statistics to illustrate the entire landscape, including the percentage of each vulnerability type. We hope to shed some light on the problem, e.g., concurrency vulnerabilities continue to pose a serious threat to system security, and it is difficult even for kernel developers to analyze and patch them. Therefore, more efforts are needed to develop tools and techniques for analyzing and patching these vulnerabilities.Comment: It was finished in Oct 201

Interaction-Oriented Software Engineering:Programming abstractions for autonomy and decentralization

We review the main ideas and elements of Interaction-Oriented Software Engineering (IOSE), a program of research that we have pursued for the last two decades, a span of time in which it has grown from philosophy to practical programming abstractions. What distinguishes IOSE from any other program of research is its emphasis on supporting autonomy by modeling the meaning of communication and using that as the basis for engineering decentralized sociotechnical systems. Meaning sounds esoteric but is the basis for practical decision making and a holy grail for the field of distributed systems. We describe our contributions so far, directions for research, and the potential for broad impact on computing

Safe sessions of channel actions in Clojure: A tour of the Discourje Project

To simplify shared-memory concurrent programming, in addition to low-level synchronisation primitives, several modern programming languages have started to offer core support for higher-level communication primitives as well, in the guise of message passing through channels. Yet, a growing body of evidence suggests that channel-based programming abstractions for shared memory also have their issues. The Discourje project aims to help programmers cope with message-passing concurrency bugs in Clojure programs, based on run-time verification and dynamic monitoring. The idea is that programmers write not only implementations, but also specifications (of sessions of channel actions). Discourje then offers a library to ensure that implementations run safely relative to specifications (= “bad” channel actions never happen). This paper gives a tour of the current state of Discourje, by example; it is intended to serve both as a general overview for readers who are unfamiliar with previous work on Discourje, and as an introduction to new features for readers who are familiar

Session Kotlin: A hybrid session type embedding in Kotlin

Concurrency and distribution have become essential for building modern applications. However, developing and maintaining these apps is not an easy task. Communication errors are a common source of problems: unexpected messages cause runtime errors, and mutual dependencies lead to deadlocks. To address these issues, developers can define communication protocols that detail the structure and order of the transmitted messages, but maintaining protocol fidelity can be complex if carried out manually. Session types formalize this concept by materializing the communication protocol as a type that can be enforced by the language’s type system. In this thesis we present the first embedding of session types in Kotlin: we propose a Domain-Specific Language (DSL) for multiparty ses- sion types that lets developers write safe concurrent applications, with built-in validation and integrating code generation in the language’s framework.A concorrência e a distribuição têm-se tornado essenciais na construção de aplicações modernas. No entanto, desenvolver e manter estas aplicações não é tarefa fácil. Erros de comunicação são uma fonte comum de problemas: mensagens inesperadas causam erros durante a execução de código, e dependências mútuas levam a deadlocks. Para resolver estas questões, é tipico definir protocolos de comunicação que detalham a estrutura e a ordem das mensagens transmitidas, mas garantir o seu cumprimento pode ser complexo se feito manualmente. Os tipos de sessão formalizam este conceito ao materializar o protocolo de comunicação como um tipo que pode ser gerido pelo sistema de tipos da linguagem. Nesta tese apresentamos o primeiro embedding de tipos de sessão em Kotlin: propomos uma Linguagem de Domínio Específica para tipos de sessão com múltiplos participantes que permite aos programadores a escrita de aplicações concorrentes seguras, incorporando validação e integrando a geração de código no framework da linguagem

Automatic detection and resolution of deadlocks in Go programs

The Go programming language is acquiring momentum in the development of concurrent software. Even though Go supports the shared-memory model, the message-passing alternative is the favoured idiomatic approach. Naturally, this practice is not exempt of the usual difficulties: programs may deadlock and the language run-time has only very basic support for deadlock detection. Previous research on deadlock detection mainly focused on shared-memory concurrency models. For mainstream languages, tools and approaches specific to the message-passing paradigm are scarce and incipient. There is however a large body of work on models of concurrency that only recently started to be applied to languages like Go. Since the Go run-time lets many deadlocks pass unnoticed, and the existing solutions provided by third party tools detect many deadlocks but only try to fix a limited set of specific patterns, imposing severe conditions to do so, there is a clear need for more general deadlock resolution strategies, going beyond prevention and avoidance. To gain insight on real-world deadlock bugs, we first built and categorized a collection of bugs sourced from high-profile open-source Go projects. Next, we extended and implemented an algorithm that takes an abstraction of the communication behaviour of the program and, when all the communication operations on channels necessary for progress are present, but a deadlock is possible, presents the problem and offers a possible resolution for the error. The extensions allows our approach to analyse a much wider range of real world programs. We conclude with an evaluation, comparing with two other state-of-the-art solutions.A linguagem de programação Go tem ganhado tração no desenvolvimento de software concorrente. Apesar de o Go suportar o modelo de partilha de memória, o modelo alternativo de partilha de mensagens é a abordagem idiomática. Naturalmente, esta prática não está isenta das dificuldades usuais: os programas podem bloquear e o runtime da linguagem só possui um suporte muito básico para a deteção destes bloqueios. Investigação anterior na deteção de bloqueios focou principalmente no modelo de partilha de memória. Para linguagens convencionais, ferramentas e abordagens dedicadas ao paradigma de passagem de mensagens são escassas e incipientes. No entanto, existe um grande conjunto de trabalhos sobre modelos de concorrência que só recentemente começou a ser aplicado em linguagens como o Go. Visto que o run-time do Go deixa muitos bloqueios passar despercebidos e as soluções existentes detetam muitos bloqueios, mas só tentam resolver um conjunto muito pequeno de padrões. De modo a ganhar conhecimento sobre erros de bloqueio reais, nós começámos por construir e categorizar uma coleção de erros obtidos a partir de projetos Go open-source de alto perfil. De seguida, nós estendemos e implementámos um algoritmo que recebe uma abstração do comportamento da comunicação de um programa e quando todas as operações de comunicação nos canais necessários para o progresso estão presentes, mas um bloqueio é possível, apresenta o erro e oferece uma possível resolução do erro. A nossa extensão permite analisar um conjunto muito maior de programas reais. Concluímos com uma avaliação, comparando com duas outras soluções do estado da arte

Actris: session-type based reasoning in separation logic

Message passing is a useful abstraction to implement concurrent programs. For real-world systems, however, it is often combined with other programming and concurrency paradigms, such as higher-order functions, mutable state, shared-memory concurrency, and locks. We present Actris: a logic for proving functional correctness of programs that use a combination of the aforementioned features. Actris combines the power of modern concurrent separation logics with a first-class protocol mechanism - based on session types - for reasoning about message passing in the presence of other concurrency paradigms. We show that Actris provides a suitable level of abstraction by proving functional correctness of a variety of examples, including a distributed merge sort, a distributed load-balancing mapper, and a variant of the map-reduce model, using relatively simple specifications. Soundness of Actris is proved using a model of its protocol mechanism in the Iris framework. We mechanised the theory of Actris, together with tactics for symbolic execution of programs, as well as all examples in the paper, in the Coq proof assistant.Programming Language

Capturing High-level Nondeterminism in Concurrent Programs for Practical Concurrency Model Agnostic Record and Replay

With concurrency being integral to most software systems, developers combine high-level concurrency models in the same application to tackle each problem with appropriate abstractions. While languages and libraries offer a wide range of concurrency models, debugging support for applications that combine them has not yet gained much attention. Record & replay aids debugging by deterministically reproducing recorded bugs, but is typically designed for a single concurrency model only. This paper proposes a practical concurrency-model-agnostic record & replay approach for multi-paradigm concurrent programs, i.e. applications that combine concurrency models. Our approach traces high-level non- deterministic events by using a uniform model-agnostic trace format and infrastructure. This enables ordering- based record & replay support for a wide range of concurrency models, and thereby enables debugging of applications that combine them. In addition, it allows language implementors to add new concurrency mod- els and reuse the model-agnostic record & replay support. We argue that a concurrency-model-agnostic record & replay is practical and enables advanced debugging support for a wide range of concurrency models. The evaluation shows that our approach is expressive and flexible enough to support record & replay of applications using threads & locks, communicating event loops, communicating sequential processes, software transactional memory and combinations of those concurrency models. For the actor model, we reach recording performance competitive with an optimized special-purpose record & replay solution. The average recording overhead on the Savina actor benchmark suite is 10% (min. 0%, max. 23%). The performance for other concurrency models and combinations thereof is at a similar level. We believe our concurrency-model-agnostic approach helps developers of applications that mix and match concurrency models. We hope that this substrate inspires new tools and languages making building and maintaining of multi-paradigm concurrent applications simpler and safer