Unifying framework for development of message-passing applications

Import 19/05/2014This thesis focuses on the development of applications for distributed memory systems in the area of scientific and engineering computations. The work addresses the problems inherent to rapid development of parallel applications and the complexity of tools used during their implementation. Herein, an abstract computation model for describing MPI (Message Passing Interface) applications is proposed. In the described topic of interest, MPI is the de facto API. The proposed approach is based on the usage of visual programming for expressing parallel aspects and communication in a developed application. The intention is not to create a complete program visually; rather, a user writes sequential parts of the application in C++ and puts them into the visual program. This allows simple modification of the communication structure and provides immediate visual feedback of the program's behavior through a visual debugger. An abstract model is also designed in a way that provides a unified view of activities that are used during development, including debugging, performance analyses, and verifications. While tools exist in all of these areas, they are usually single-purpose tools. In the proposed approach, all these activities are roofed by a single conceptual frame; everything from configurations to providing results of analyses uses one set of terms. The thesis makes the following contributions: It introduces the syntax and semantics of the visual language for describing parallel aspects and communication in applications for distributed memory systems. The semantics of the language is formalized and the usage of the language is demonstrated for various problems. These examples show that the sizes of diagrams describing the visual programs remain reasonably small for many practical parallel applications. Measurements show that the performance of the resulting programs is close to manually created C++ applications. Also, the features of the proposed approach may help in various ways in common activities that occur during development. The ideas presented in this thesis are implemented as a tool named Kaira (\url{http://verif.cs.vsb.cz/kaira/}); it is an open source tool released under GPL (GNU General Public License).Tato práce se zabývá vývojem aplikací pro systémy s distribuovanou pamětí v oblasti vědecko-technických výpočtů. Práce se zaměřuje na rychlý vývoj para\-lelních aplikací a integraci nástrojů používaných při vývoji. Navržený přístup je založen na abstraktním výpočetním modelu a vizuálním programování. Tyto prostředky slouží k vyjádření paralelismu a komunikaci ve vyvíjené aplikaci. Sekvenční části aplikace jsou psány přímo v C++ a tento kód je kombinován s~vizuálním modelem. Navržený přístup dovoluje jednoduchou manipulaci s~programem a umožňuje poskytnout přímočarou vizualizaci chování aplikace. Abstraktní model je také navržen za účelem sjednocení aktivit, které se objevují při vývoji aplikací: debugování, analýza výkonu, predikce výkonu a verifikace. Pro tyto aktivity existuje mnoho různých etablovaných nástrojů, které jsou mnohem vyzrálejší a nabízejí více funkcí než zde předkládaná implementace. Z~pohledu této práce se ale většinou jedná o~programy zaměřené pouze na jednu z oblastí. Navrhovaný přístup spojuje všechny tyto aktivity do jednoho myšlenkového rámce s jednotnou terminologií, konfigurací a prezentací výsledků. Hlavní přínosy této práce spočívají ve vytvoření syntaxe a sémantiky vizuálního jazyka pro popis paralelních částí a komunikace v aplikacích pro systémy s distribuovanou pamětí. Sémantika jazyka je plně formalizována a využití jazyka je demonstrováno na různých příkladech. Tyto příklady ukazují, že diagramy reprezentující program zůstávají relativně malé a přehledné při řešení různých praktických problémů. V této práci je dále ukázáno, že navržený přístup může sjednotit a zjednodušit mnoho různých činností objevujících se v souvislosti s vývojem softwaru v oblasti zájmu této práce. Prezentované myšlenky jsou implementovány v podobě nástroje Kaira (\url{http://verif.cs.vsb.cz/kaira/}). Kaira je uvolněna jako open-source nástroj pod GPL (GNU General Public License).Prezenční460 - Katedra informatikyvyhově

Tree-AMP: Compositional Inference with Tree Approximate Message Passing

We introduce Tree-AMP, standing for Tree Approximate Message Passing, a python package for compositional inference in high-dimensional tree-structured models. The package provides a unifying framework to study several approximate message passing algorithms previously derived for a variety of machine learning tasks such as generalized linear models, inference in multi-layer networks, matrix factorization, and reconstruction using non-separable penalties. For some models, the asymptotic performance of the algorithm can be theoretically predicted by the state evolution, and the measurements entropy estimated by the free entropy formalism. The implementation is modular by design: each module, which implements a factor, can be composed at will with other modules to solve complex inference tasks. The user only needs to declare the factor graph of the model: the inference algorithm, state evolution and entropy estimation are fully automated.Comment: Source code available at https://github.com/sphinxteam/tramp and documentation at https://sphinxteam.github.io/tramp.doc

Termination in Concurrency, Revisited

Termination is a central property in sequential programming models: a term is terminating if all its reduction sequences are finite. Termination is also important in concurrency in general, and for message-passing programs in particular. A variety of type systems that enforce termination by typing have been developed. In this paper, we rigorously compare several type systems for $\pi$-calculus processes from the unifying perspective of termination. Adopting session types as reference framework, we consider two different type systems: one follows Deng and Sangiorgi's weight-based approach; the other is Caires and Pfenning's Curry-Howard correspondence between linear logic and session types. Our technical results precisely connect these very different type systems, and shed light on the classes of client/server interactions they admit as correct

Motion session types for robotic interactions

Robotics applications involve programming concurrent components synchronising through messages while simultaneously executing motion primitives that control the state of the physical world. Today, these applications are typically programmed in low-level imperative programming languages which provide little support for abstraction or reasoning. We present a unifying programming model for concurrent message-passing systems that additionally control the evolution of physical state variables, together with a compositional reasoning framework based on multiparty session types. Our programming model combines message-passing concurrent processes with motion primitives. Processes represent autonomous components in a robotic assembly, such as a cart or a robotic arm, and they synchronise via discrete messages as well as via motion primitives. Continuous evolution of trajectories under the action of controllers is also modelled by motion primitives, which operate in global, physical time. We use multiparty session types as specifications to orchestrate discrete message-passing concurrency and continuous flow of trajectories. A global session type specifies the communication protocol among the components with joint motion primitives. A projection from a global type ensures that jointly executed actions at end-points are communication safe and deadlock-free, i.e., session-typed components do not get stuck. Together, these checks provide a compositional verification methodology for assemblies of robotic components with respect to concurrency invariants such as a progress property of communications as well as dynamic invariants such as absence of collision. We have implemented our core language and, through initial experiments, have shown how multiparty session types can be used to specify and compositionally verify robotic systems implemented on top of off-the-shelf and custom hardware using standard robotics application libraries

Space Reclamation for Uncoordinated Checkpointing in Message-Passing Systems

Checkpointing and rollback recovery are techniques that can provide efficient recovery from transient process failures. In a message-passing system, the rollback of a message sender may cause the rollback of the corresponding receiver, and the system needs to roll back to a consistent set of checkpoints called recovery line. If the processes are allowed to take uncoordinated checkpoints, the above rollback propagation may result in the domino effect which prevents recovery line progression. Traditionally, only obsolete checkpoints before the global recovery line can be discarded, and the necessary and sufficient condition for identifying all garbage checkpoints has remained an open problem. A necessary and sufficient condition for achieving optimal garbage collection is derived and it is proved that the number of useful checkpoints is bounded by N(N+1)/2, where N is the number of processes. The approach is based on the maximum-sized antichain model of consistent global checkpoints and the technique of recovery line transformation and decomposition. It is also shown that, for systems requiring message logging to record in-transit messages, the same approach can be used to achieve optimal message log reclamation. As a final topic, a unifying framework is described by considering checkpoint coordination and exploiting piecewise determinism as mechanisms for bounding rollback propagation, and the applicability of the optimal garbage collection algorithm to domino-free recovery protocols is demonstrated

The role of concurrency in an evolutionary view of programming abstractions

In this paper we examine how concurrency has been embodied in mainstream programming languages. In particular, we rely on the evolutionary talking borrowed from biology to discuss major historical landmarks and crucial concepts that shaped the development of programming languages. We examine the general development process, occasionally deepening into some language, trying to uncover evolutionary lineages related to specific programming traits. We mainly focus on concurrency, discussing the different abstraction levels involved in present-day concurrent programming and emphasizing the fact that they correspond to different levels of explanation. We then comment on the role of theoretical research on the quest for suitable programming abstractions, recalling the importance of changing the working framework and the way of looking every so often. This paper is not meant to be a survey of modern mainstream programming languages: it would be very incomplete in that sense. It aims instead at pointing out a number of remarks and connect them under an evolutionary perspective, in order to grasp a unifying, but not simplistic, view of the programming languages development process