Uniqueness Typing for Resource Management in Message-Passing Concurrency

We view channels as the main form of resources in a message-passing programming paradigm. These channels need to be carefully managed in settings where resources are scarce. To study this problem, we extend the pi-calculus with primitives for channel allocation and deallocation and allow channels to be reused to communicate values of different types. Inevitably, the added expressiveness increases the possibilities for runtime errors. We define a substructural type system which combines uniqueness typing and affine typing to reject these ill-behaved programs

Compositional Reasoning for Explicit Resource Management in Channel-Based Concurrency

We define a pi-calculus variant with a costed semantics where channels are treated as resources that must explicitly be allocated before they are used and can be deallocated when no longer required. We use a substructural type system tracking permission transfer to construct coinductive proof techniques for comparing behaviour and resource usage efficiency of concurrent processes. We establish full abstraction results between our coinductive definitions and a contextual behavioural preorder describing a notion of process efficiency w.r.t. its management of resources. We also justify these definitions and respective proof techniques through numerous examples and a case study comparing two concurrent implementations of an extensible buffer.Comment: 51 pages, 7 figure

Specifying and Verifying Concurrent Algorithms with Histories and Subjectivity

We present a lightweight approach to Hoare-style specifications for fine-grained concurrency, based on a notion of time-stamped histories that abstractly capture atomic changes in the program state. Our key observation is that histories form a partial commutative monoid, a structure fundamental for representation of concurrent resources. This insight provides us with a unifying mechanism that allows us to treat histories just like heaps in separation logic. For example, both are subject to the same assertion logic and inference rules (e.g., the frame rule). Moreover, the notion of ownership transfer, which usually applies to heaps, has an equivalent in histories. It can be used to formally represent helping---an important design pattern for concurrent algorithms whereby one thread can execute code on behalf of another. Specifications in terms of histories naturally abstract granularity, in the sense that sophisticated fine-grained algorithms can be given the same specifications as their simplified coarse-grained counterparts, making them equally convenient for client-side reasoning. We illustrate our approach on a number of examples and validate all of them in Coq.Comment: 17 page

Reasoning about explicit resource management

We investigate the behaviour and efficiency of concurrent processes with explicit resource management. Our study is based on a π-calculus variant called Rπ [4] where the only resources available are channels, which must be explicitly allocated before they can be used and can be deallocated when no longer required. A substructural type system guarantees the safe allocation and deallocation of channels, as well as safe channel reuse through strong updates. In this paper we use this type system to give compositional proof techniques for reasoning about the behaviour and efficiency of Rπ processes.peer-reviewe

A library for developing real-time and embedded applications in C

Next generation applications will demand more cost-effective programming abstractions to reduce increasing maintenance and development costs. In this context, the article explores the integration of an efficient programming language and high-level real-time programming abstractions. The resulting abstraction is called Embedded Cyber Physical C (ECP-C) and it is useful for designing real-time applications directly on C. The abstraction has its roots on the real-time Java: one of the most modern programming languages, which benefited from mature programming patterns previously developed for other languages. It also targets embedded processors running on limited hardware. ECP-C takes the programming abstractions described in real-time Java and reflects them into a C application system, providing extensions for multi-threading, resource sharing, memory management, external event, signaling, and memory access. It also reports on the performance results obtained in a set of infrastructures used to check ECP-C, providing clues on the overhead introduced by these mechanisms on limited infrastructures. (C) 2015 Elsevier B.V. All rights reserved.This work has been partially funded by Distributed Java Infrastructure for Real-Time Big-Data (CAS14/00118) and by eMadrid: Investigación y Desarrollo de tecnologías educativas en la Comunidad de Madrid (S2013/ICE-2715). This research was supported by the national project REM4VSS (TIN-2011-28339) and by European Union’s 7th Framework Programme Under Grant Agreement FP7-IC6-318763

A proof system for lock-free concurrency

Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaSoftware has become widespread, being used and relied upon on nearly every domain. Furthermore, as this globalization of software took place and multi-core architectures became the norm, several programs are now expected to run on a given device at the same time in a timely fashion. Attending this need, concurrent and distributed systems are a well known way of dealing with performance and scalability of computation. Although several such systems exist in the devices and services we depend on, it is frequent for those systems to be exploited or go wrong. Because most complex programs are built in modules and lack a formal specification of what they do, it is hard to prevent the emerging system from failing or being exploited. Therefore, one of the most sought after results by software industry is a way of reasoning about programs that prevents undesired behavior. Formal methods contribute to a rigorous specification, analysis, and verification of programs, having proven to be quite effective in this regard. Program logics,in particular, are able to verify validity of user-specified formulas and are the solution we propose to tackle this issue. Regarding concurrent programs, locks are a mechanism that make reasoning easier by serializing access to shared resources, taming concurrency. Since lock-free programs offer a better way of taking advantage of concurrency, we are especially interested in them. In this context, the LL/SC pair of primitives have proven to be more expressive than their widely used CAS counterpart. The goal of our work is then to develop a proof system capable of proving correctness of lock-free programs based on LL/SC primitives. In this dissertation we present a new program logic, based on those of concurrent separation logic and RGSep, which establishes a solid theoretical basis for reasoning about such programs

Architectural Principles for Database Systems on Storage-Class Memory

Database systems have long been optimized to hide the higher latency of storage media, yielding complex persistence mechanisms. With the advent of large DRAM capacities, it became possible to keep a full copy of the data in DRAM. Systems that leverage this possibility, such as main-memory databases, keep two copies of the data in two different formats: one in main memory and the other one in storage. The two copies are kept synchronized using snapshotting and logging. This main-memory-centric architecture yields nearly two orders of magnitude faster analytical processing than traditional, disk-centric ones. The rise of Big Data emphasized the importance of such systems with an ever-increasing need for more main memory. However, DRAM is hitting its scalability limits: It is intrinsically hard to further increase its density. Storage-Class Memory (SCM) is a group of novel memory technologies that promise to alleviate DRAM’s scalability limits. They combine the non-volatility, density, and economic characteristics of storage media with the byte-addressability and a latency close to that of DRAM. Therefore, SCM can serve as persistent main memory, thereby bridging the gap between main memory and storage. In this dissertation, we explore the impact of SCM as persistent main memory on database systems. Assuming a hybrid SCM-DRAM hardware architecture, we propose a novel software architecture for database systems that places primary data in SCM and directly operates on it, eliminating the need for explicit IO. This architecture yields many benefits: First, it obviates the need to reload data from storage to main memory during recovery, as data is discovered and accessed directly in SCM. Second, it allows replacing the traditional logging infrastructure by fine-grained, cheap micro-logging at data-structure level. Third, secondary data can be stored in DRAM and reconstructed during recovery. Fourth, system runtime information can be stored in SCM to improve recovery time. Finally, the system may retain and continue in-flight transactions in case of system failures. However, SCM is no panacea as it raises unprecedented programming challenges. Given its byte-addressability and low latency, processors can access, read, modify, and persist data in SCM using load/store instructions at a CPU cache line granularity. The path from CPU registers to SCM is long and mostly volatile, including store buffers and CPU caches, leaving the programmer with little control over when data is persisted. Therefore, there is a need to enforce the order and durability of SCM writes using persistence primitives, such as cache line flushing instructions. This in turn creates new failure scenarios, such as missing or misplaced persistence primitives. We devise several building blocks to overcome these challenges. First, we identify the programming challenges of SCM and present a sound programming model that solves them. Then, we tackle memory management, as the first required building block to build a database system, by designing a highly scalable SCM allocator, named PAllocator, that fulfills the versatile needs of database systems. Thereafter, we propose the FPTree, a highly scalable hybrid SCM-DRAM persistent B+-Tree that bridges the gap between the performance of transient and persistent B+-Trees. Using these building blocks, we realize our envisioned database architecture in SOFORT, a hybrid SCM-DRAM columnar transactional engine. We propose an SCM-optimized MVCC scheme that eliminates write-ahead logging from the critical path of transactions. Since SCM -resident data is near-instantly available upon recovery, the new recovery bottleneck is rebuilding DRAM-based data. To alleviate this bottleneck, we propose a novel recovery technique that achieves nearly instant responsiveness of the database by accepting queries right after recovering SCM -based data, while rebuilding DRAM -based data in the background. Additionally, SCM brings new failure scenarios that existing testing tools cannot detect. Hence, we propose an online testing framework that is able to automatically simulate power failures and detect missing or misplaced persistence primitives. Finally, our proposed building blocks can serve to build more complex systems, paving the way for future database systems on SCM