Automatic Verification of Erlang-Style Concurrency

This paper presents an approach to verify safety properties of Erlang-style, higher-order concurrent programs automatically. Inspired by Core Erlang, we introduce Lambda-Actor, a prototypical functional language with pattern-matching algebraic data types, augmented with process creation and asynchronous message-passing primitives. We formalise an abstract model of Lambda-Actor programs called Actor Communicating System (ACS) which has a natural interpretation as a vector addition system, for which some verification problems are decidable. We give a parametric abstract interpretation framework for Lambda-Actor and use it to build a polytime computable, flow-based, abstract semantics of Lambda-Actor programs, which we then use to bootstrap the ACS construction, thus deriving a more accurate abstract model of the input program. We have constructed Soter, a tool implementation of the verification method, thereby obtaining the first fully-automatic, infinite-state model checker for a core fragment of Erlang. We find that in practice our abstraction technique is accurate enough to verify an interesting range of safety properties. Though the ACS coverability problem is Expspace-complete, Soter can analyse these verification problems surprisingly efficiently.Comment: 12 pages plus appendix, 4 figures, 1 table. The tool is available at http://mjolnir.cs.ox.ac.uk/soter

Towards Erlang Verification by Term Rewriting

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-14125-1_7This paper presents a transformational approach to the verification of Erlang programs. We define a stepwise transformation from (first-order) Erlang programs to (non-deterministic) term rewrite systems that compute an overapproximation of the original Erlang program. In this way, existing techniques for term rewriting become available. Furthermore, one can use narrowing as a symbolic execution extension of rewriting in order to design a verification technique. We illustrate our approach with some examples, including a deadlock analysis of a simple Erlang program.Vidal Oriola, GF. (2013). Towards Erlang Verification by Term Rewriting. En Logic-Based Program Synthesis and Transformation. Springer. 109-126. doi:10.1007/978-3-319-14125-1_7S109126Albert, E., Arenas, P., Gómez-Zamalloa, M.: Symbolic Execution of Concurrent Objects in CLP. In: Russo, C., Zhou, N.-F. (eds.) PADL 2012. LNCS, vol. 7149, pp. 123–137. Springer, Heidelberg (2012)Albert, E., Vidal, G.: The narrowing-driven approach to functional logic program specialization. New Generation Computing 20(1), 3–26 (2002)Joe, A., Robert, V., Williams, M.: Concurrent programming in ERLANG. Prentice Hall (1993)Arts, T., Earle, C.B., Derrick, J.: Development of a verified Erlang program for resource locking. STTT 5(2–3), 205–220 (2004)Baader, F., Nipkow, T.: Term Rewriting and All That. Cambridge University Press (1998)Caballero, R., Martin-Martin, E., Riesco, A., Tamarit, S.: A Declarative Debugger for Sequential Erlang Programs. In: Veanes, M., Viganò, L. (eds.) TAP 2013. LNCS, vol. 7942, pp. 96–114. Springer, Heidelberg (2013)Claessen, K., Svensson, H.: A semantics for distributed Erlang. In: Sagonas, K.F., Armstrong, J. (eds.). In: Proc. of the 2005 ACM SIGPLAN Workshop on Erlang, pp. 78–87. ACM (2005)Earle, C.B.: Symbolic program execution using the Erlang verification tool. In: Alpuente, M. (eds.) Proc. of the 9th International Workshop on Functional and Logic Programming (WFLP 2000), pp. 42–55 (2000)Felleisen, M., Friedman, D.P., Kohlbecker, E.E., Duba, B.F.: A syntactic theory of sequential control. Theor. Comput. Sci. 52, 205–237 (1987)Fredlund, L.-A., Svensson, H.: McErlang: a model checker for a distributed functional programming language. In: Hinze, R., Ramsey, N. (eds). In: Proc. of ICFP 2007, pp. 125–136. ACM (2007)Giesl, J., Arts, T.: Verification of Erlang Processes by Dependency Pairs. Appl. Algebra Eng. Commun. Comput. 12(1/2), 39–72 (2001)Hanus, M. (ed.): Curry: An integrated functional logic language (vers. 0.8.3) (2012), http://www.curry-language.orgHuch, F.: Verification of Erlang Programs using Abstract Interpretation and Model Checking. In: Rémi, D., Lee, P. (eds.) Proc. of ICFP 1999, pp. 261–272. ACM (1999)J.-M., H.: Canonical forms and unification. In: Bibel, W., Kowalski, R. (eds.) 5th Conference on Automated Deduction Les Arcs. LNCS, pp. 318–334. Springer, Heidelberg (1980)Leucker, M., Noll, T.: Rewriting Logic as a Framework for Generic Verification Tools. Electr. Notes Theor. Comput. Sci. 36, 121–137 (2000)Meseguer, J.: Conditioned Rewriting Logic as a United Model of Concurrency. Theor. Comput. Sci. 96(1), 73–155 (1992)Neuhäußer, M.R., Noll, T.: Abstraction and Model Checking of Core Erlang Programs in Maude. Electr. Notes Theor. Comput. Sci. 176(4), 147–163 (2007)Nishida, N., Vidal, G.: A finite representation of the narrowing space. In: Proc. of the 23th International Symposium on Logic-Based Program Synthesis and Transformation (LOPSTR 2013). Technical Report TR-11-13, Universidad Complutense de Madrid, pp. 113–128 (To appear in Springer LNCS, 2013). http://users.dsic.upv.es/~gvidal/Noll, T.: A Rewriting Logic Implementation of Erlang. Electr. Notes Theor. Comput. Sci. 44(2), 206–224 (2001)Noll, T.: Equational Abstractions for Model Checking Erlang Programs. Electr. Notes Theor. Comput. Sci. 118, 145–162 (2005)Noll, T.G., Fredlund, L., Gurov, D.: The Erlang Verification Tool. In: Margaria, T., Yi, W. (eds.) TACAS 2001. LNCS, vol. 2031, pp. 582–586. Springer, Heidelberg (2001)Roy, C.K.: Thomas Noll, Banani Roy, and James R. Cordy. Towards automatic verification of Erlang programs by pi-calculus translation. In: Feeley,M., Trinder, P.W. (eds.) Proc. of the 2006 ACM SIGPLAN Workshop on Erlang, pp. 38–50. ACM (2006)Slagle, J.R.: Automated theorem-proving for theories with simplifiers, commutativity and associativity. Journal of the ACM 21(4), 622–642 (1974)Svensson, H., Fredlund, L.-A.: A more accurate semantics for distributed Erlang. In: Thompson, S.J., Fredlund. L.-A., (eds.) Proceedings of the 2007 ACM SIGPLAN Workshop on Erlang, pp. 43–54. ACM (2007)Vidal, G.: Closed symbolic execution for verifying program termination. In: Proc. of the 12th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2012), pp. 34–43. IEEE (2012)Visser, W., Havelund, K., Brat, G.P., Park, S., Lerda, F.: Model checking programs. Autom. Softw. Eng. 10(2), 203–232 (2003

Session Types in a Linearly Typed Multi-Threaded Lambda-Calculus

We present a formalization of session types in a multi-threaded lambda-calculus (MTLC) equipped with a linear type system, establishing for the MTLC both type preservation and global progress. The latter (global progress) implies that the evaluation of a well-typed program in the MTLC can never reach a deadlock. As this formulated MTLC can be readily embedded into ATS, a full-fledged language with a functional programming core that supports both dependent types (of DML-style) and linear types, we obtain a direct implementation of session types in ATS. In addition, we gain immediate support for a form of dependent session types based on this embedding into ATS. Compared to various existing formalizations of session types, we see the one given in this paper is unique in its closeness to concrete implementation. In particular, we report such an implementation ready for practical use that generates Erlang code from well-typed ATS source (making use of session types), thus taking great advantage of the infrastructural support for distributed computing in Erlang.Comment: This is the original version of the paper on supporting programming with dyadic session types in AT

Revisiting Actor Programming in C++

The actor model of computation has gained significant popularity over the last decade. Its high level of abstraction makes it appealing for concurrent applications in parallel and distributed systems. However, designing a real-world actor framework that subsumes full scalability, strong reliability, and high resource efficiency requires many conceptual and algorithmic additives to the original model. In this paper, we report on designing and building CAF, the "C++ Actor Framework". CAF targets at providing a concurrent and distributed native environment for scaling up to very large, high-performance applications, and equally well down to small constrained systems. We present the key specifications and design concepts---in particular a message-transparent architecture, type-safe message interfaces, and pattern matching facilities---that make native actors a viable approach for many robust, elastic, and highly distributed developments. We demonstrate the feasibility of CAF in three scenarios: first for elastic, upscaling environments, second for including heterogeneous hardware like GPGPUs, and third for distributed runtime systems. Extensive performance evaluations indicate ideal runtime behaviour for up to 64 cores at very low memory footprint, or in the presence of GPUs. In these tests, CAF continuously outperforms the competing actor environments Erlang, Charm++, SalsaLite, Scala, ActorFoundry, and even the OpenMPI.Comment: 33 page

High Performance Web Servers: A Study In Concurrent Programming Models

With the advent of commodity large-scale multi-core computers, the performance of software running on these computers has become a challenge to researchers and enterprise developers. While academic research and industrial products have moved in the direction of writing scalable and highly available services using distributed computing, single machine performance remains an active domain, one which is far from saturated. This thesis selects an archetypal software example and workload in this domain, and describes software characteristics affecting performance. The example is highly-parallel web-servers processing a static workload. Particularly, this work examines concurrent programming models in the context of high-performance web-servers across different architectures — threaded (Apache, Go and μKnot), event-driven (Nginx, μServer) and staged (WatPipe) — compared with two static workloads in two different domains. The two workloads are a Zipf distribution of file sizes representing a user session pulling an assortment of many small and a few large files, and a 50KB file representing chunked streaming of a large audio or video file. Significant effort is made to fairly compare eight web-servers by carefully tuning each via their adjustment parameters. Tuning plays a significant role in workload-specific performance. The two domains are no disk I/O (in-memory file set) and medium disk I/O. The domains are created by lowering the amount of RAM available to the web-server from 4GB to 2GB, forcing files to be evicted from the file-system cache. Both domains are also restricted to 4 CPUs. The primary goal of this thesis is to examine fundamental performance differences between threaded and event-driven concurrency models, with particular emphasis on user-level threading models. Additionally, a secondary goal of the work is to examine high-performance software under restricted hardware environments. Over-provisioned hardware environments can mask architectural and implementation shortcomings in software – the hypothesis in this work is that restricting resources stresses the application, bringing out important performance characteristics and properties. Experimental results for the given workload show that memory pressure is one of the most significant factors for the degradation of web-server performance, because it forces both the onset and amount of disk I/O. With an ever increasing need to support more content at faster rates, a web-server relies heavily on in-memory caching of files and related content. In fact, personal and small business web-servers are even run on minimal hardware, like the Raspberry Pi, with only 1GB of RAM and a small SD card for the file system. Therefore, understanding behaviour and performance in restricted contexts should be a normal aspect of testing a web server (and other software systems)

Implementing SOS with active objects: A case study of a multicore memory system

This paper describes the development of a parallel simulator of a multicore memory system from a model formalized as a structural operational semantics (SOS). Our implementation uses the Abstract Behavioral Specification (ABS) language, an executable, active object modelling language with a formal semantics, targeting distributed systems. We develop general design patterns in ABS for implementing SOS, and describe their application to the SOS model of multicore memory systems. We show how these patterns allow a formal correctness proof that the implementation simulates the formal operational model and discuss further parallelization and fairness of the simulator

Language run-time systems:An overview

The proliferation of high-level programming languages with advanced language features and the need for portability across increasingly heterogeneous and hierarchical architectures require a sophisticated run-time system to manage program execution and available resources. Additional benefits include isolated execution of untrusted code and the potential for dynamic optimisation, among others. This paper provides a high-level overview of language run-time systems with a focus on execution models, support for concurrency and parallelism, memory management, and communication, whilst briefly mentioning synchronisation, monitoring, and adaptive policy control. Two alternative approaches to run-time system design are presented and several challenges for future research are outlined. References to both seminal and recent work are provided

A semantics comparison workbench for a concurrent, asynchronous, distributed programming language

A number of high-level languages and libraries have been proposed that offer novel and simple to use abstractions for concurrent, asynchronous, and distributed programming. The execution models that realise them, however, often change over time---whether to improve performance, or to extend them to new language features---potentially affecting behavioural and safety properties of existing programs. This is exemplified by SCOOP, a message-passing approach to concurrent object-oriented programming that has seen multiple changes proposed and implemented, with demonstrable consequences for an idiomatic usage of its core abstraction. We propose a semantics comparison workbench for SCOOP with fully and semi-automatic tools for analysing and comparing the state spaces of programs with respect to different execution models or semantics. We demonstrate its use in checking the consistency of properties across semantics by applying it to a set of representative programs, and highlighting a deadlock-related discrepancy between the principal execution models of SCOOP. Furthermore, we demonstrate the extensibility of the workbench by generalising the formalisation of an execution model to support recently proposed extensions for distributed programming. Our workbench is based on a modular and parameterisable graph transformation semantics implemented in the GROOVE tool. We discuss how graph transformations are leveraged to atomically model intricate language abstractions, how the visual yet algebraic nature of the model can be used to ascertain soundness, and highlight how the approach could be applied to similar languages.Comment: Accepted by Formal Aspects of Computin

New Directions in Cloud Programming

Nearly twenty years after the launch of AWS, it remains difficult for most developers to harness the enormous potential of the cloud. In this paper we lay out an agenda for a new generation of cloud programming research aimed at bringing research ideas to programmers in an evolutionary fashion. Key to our approach is a separation of distributed programs into a PACT of four facets: Program semantics, Availablity, Consistency and Targets of optimization. We propose to migrate developers gradually to PACT programming by lifting familiar code into our more declarative level of abstraction. We then propose a multi-stage compiler that emits human-readable code at each stage that can be hand-tuned by developers seeking more control. Our agenda raises numerous research challenges across multiple areas including language design, query optimization, transactions, distributed consistency, compilers and program synthesis