CPL: A Core Language for Cloud Computing -- Technical Report

Running distributed applications in the cloud involves deployment. That is, distribution and configuration of application services and middleware infrastructure. The considerable complexity of these tasks resulted in the emergence of declarative JSON-based domain-specific deployment languages to develop deployment programs. However, existing deployment programs unsafely compose artifacts written in different languages, leading to bugs that are hard to detect before run time. Furthermore, deployment languages do not provide extension points for custom implementations of existing cloud services such as application-specific load balancing policies. To address these shortcomings, we propose CPL (Cloud Platform Language), a statically-typed core language for programming both distributed applications as well as their deployment on a cloud platform. In CPL, application services and deployment programs interact through statically typed, extensible interfaces, and an application can trigger further deployment at run time. We provide a formal semantics of CPL and demonstrate that it enables type-safe, composable and extensible libraries of service combinators, such as load balancing and fault tolerance.Comment: Technical report accompanying the MODULARITY '16 submissio

Multitier Modules

Multitier programming languages address the complexity of developing distributed systems abstracting over low level implementation details such as data representation, serialization and network protocols. Since the functionalities of different peers can be defined in the same compilation unit, multitier languages do not force developers to modularize software along network boundaries. Unfortunately, combining the code for all tiers into the same compilation unit poses a scalability challenge or forces developers to resort to traditional modularization abstractions that are agnostic to the multitier nature of the language. In this paper, we address this issue with a module system for multitier languages. Our module system supports encapsulating each (cross-peer) functionality and defining it over abstract peer types. As a result, we disentangle modularization and distribution and we enable the definition of a distributed system as a composition of multitier modules, each representing a subsystem. Our case studies on distributed algorithms, distributed data structures, as well as on the Apache Flink task distribution system, show that multitier modules allow the definition of reusable (abstract) patterns of interaction in distributed software and enable separating the modularization and distribution concerns, properly separating functionalities in distributed systems

ContextErlang: A language for distributed context-aware self-adaptive applications

Self-adaptive software modifies its behavior at run time to satisfy changing requirements in a dynamic environment. Context-oriented programming (COP) has been recently proposed as a specialized programming paradigm for context-aware and adaptive systems. COP mostly focuses on run time adaptation of the application’s behavior by supporting modular descriptions of behavioral variations. However, self-adaptive applications must satisfy additional requirements, such as distribution and concurrency, support for unforeseen changes and enforcement of correct behavior in the presence of dynamic change. Addressing these issues at the language level requires a holistic design that covers all aspects and takes into account the possibly cumbersome interaction of those features, for example concurrency and dynamic change. We present ContextErlang, a COP programming language in which adaptive abstractions are seamlessly integrated with distribution and concurrency. We define ContextErlang’s formal semantics, validated through an executable prototype, and we show how it supports formal proofs that the language design ensures satisfaction of certain safety requirements. We provide empirical evidence that ContextErlang is an effective solution through case studies and a performance assessment. We also show how the same design principles that lead to the development of ContextErlang can be followed to systematically design contextual extensions of other languages. A concrete example is presented concerning ContextScala

Fault-tolerant Distributed Reactive Programming

In this paper, we present a holistic approach to provide fault tolerance for distributed reactive programming. Our solution automatically stores and recovers program state to handle crashes, automatically updates and shares distributed parts of the state to provide eventual consistency, and handles errors in a fine-grained manner to allow precise manual control when necessary. By making use of the reactive programming paradigm, we provide these mechanisms without changing the behavior of existing programs and with reasonable performance, as indicated by our experimental evaluation

TCEP:Adapting to Dynamic User Environments by Enabling Transitions between Operator Placement Mechanisms

Operator placement has a profound impact on the performance of a distributed complex event processing system (DCEP). Since the behavior of a placement mechanism strongly depends on its environment; a single placement mechanism is often not enough to fulfill stringent performance requirements under environmental changes. In this paper, we show how DCEP can benefit from the adaptive use of multiple placement mechanisms. We propose Tcep, a DCEP system to integrate multiple placement mechanisms. By enabling transitions, Tcep can seamlessly exchange distinct operator mechanisms at runtime. We make two main contributions that are highly important for a cost-efficient transition: i) a transition strategy for efficiently scheduling state migrations and ii) a lightweight learning algorithm to adaptively select an appropriate placement mechanism as a consequence of a transition. Our evaluations for important decentralized placement mechanisms in the context of an IoT scenario show that transitions can better fulfill QoS demands in a dynamic environment. Thereby efficient scheduling of state migrations can help to faster complete transitions by up to 94 %

Programming with implicit flows

Modern software differs significantly from traditional computer applications that mostly process reasonably small amounts of static input data-sets in batch mode. Modern software increasingly processes massive amounts of data, whereby it is also often the case that new input data is produced and/or existing data is modified on the fly. Consequently, programming models that facilitate the development of such software are emerging. What characterizes them is that data, respectively changes thereof, implicitly flow through computation modules. The software engineer declaratively defines computations as compositions of other computations without explicitly modeling how data should flow along dependency relations between data producer and data consumer modules, letting the runtime to automatically manage and optimize data flows. Keywords: Reactive programming, event, stream, big data, data-flow We have come a long way since the early computer systems which were painstakingly fed problem data-sets via punch cards. Computer systems have become much more convenient to interact with and are able to process much larger data-sets, which are kept in large-scale storage systems. However, computer systems are also much more commonly involved in processing of data that is produced or modified in an online fashion, as the program is executing, sometimes in a perpetual manner. This is particularly the case in applications which are specifically developed to react to real-world happenings such as temperature changes or other environmental cues captured through sensors. The last decade has thus seen the advent of abstractions and paradigms that support the development of reactive software. Central to such approaches is the concept of event which captures dynamic occurrences that trigger computations. Over the years, several steps have been made in this direction, including language-level support for events, continuous time-changing values (a.k.a. signals or behaviors), constraints, asynchronous execution and futures. The ever-increasing complexity of reactive applications has recently raised new interest around these abstractions. The new paradigm of reactive programming focuses on a more holistic view that demands for seamless integration of existing solutions, including constraints resolution to enforce functional dependencies, automatic updates of dependent values, and interoperability among different reactive abstractions such as signals and event streams. The goal is to raise the abstraction level: Rather than explicitly reifying events in the software, changes to values of variables are detected and propagated through programs by re-computing the values of all dependent variables implicitly, i.e., by the language runtime. Interestingly, a similar trend can be observed in recent big data analysis software. Not too long ago, such programs were typically perceived as resembling complex queries applied to very large yet static data-sets. A host of programming languages and models have been proposed for such programs. They mostly mix imperative and declarative traits to clearly expose the order of a non-cyclic computation network, and are centered on some form of data-structures conceptualizing the current state of computation. Despite improvements in running time of such analysis programs often due to parallel execution over powerful computation environments, their execution can still take sufficiently long to make repeated complete executions of the same program upon additions or changes to the underlying big data-sets prohibitively expensive. Consequently, recent improvements consist in enabling incremental computations, i.e., re-executing only those parts of queries that become invalid or incomplete by changes to analyzed data-sets. While reactive and big data analysis applications have little in common at first glance, we observe a shared trend in the respective programming models: they strive to capture what the computation ought to do, but not when (and how) it shall do so, as the data which is subject to the computation changes over time (thus we speak of "data-flows"). It is the execution engines and language runtimes that increasingly carry the burden of determining which parts of computations are affected by which fluctuations in the processed data. As it is unlikely that runtime systems can determine these things entirely on their own -at least in an efficient manner -or that such transparency would even serve the programmer, new abstractions are needed to capture such implicit flows in addition to underlying runtime support. In the following, we first overview the nature and origins of reactive programming and big data analysis and implicit flows 1 therein. Next, we briefly touch on the state of the art and open challenges towards a unified approach to programming with implicit flows. Unification makes sense not only because of the shared trend towards implicit flows. More importantly it helps coping with the complexity of software that increasingly combines features from both families of applications. Events and Reactive Programming Events are a common way for programmers to reason about significant conditions in the environment and in the execution of a program. Dedicated abstractions for events have been supported by some mainstream languages for a long time. For example, in C#, events can be defined as class attributes beside methods and fields and belong to a class' interface. Over the last few years, researchers have proposed increasingly sophisticated event models (cf. Box "Avanced programming with events"). The integration into the object-oriented (OO) programming model has been enhanced to extend OO concepts like inheritance to events and event handling. Early approaches like Java P S [2] implemented events as specific objects. In EScala [11] events are first-class entities. As in C#, they are object attributes just like methods and fields; their definition is subject to polymorphic access and late binding. Our investigations Events in isolation improve little over the observer design pattern. The difference becomes crucial when expressive operators for event combination are available to correlate events to define new (complex) events that capture high-level situations of interest. Advanced systems support operators to combine events with increasing levels of expressiveness. For example, the e 1 ||e 2 expression in EScala returns an event that fires when either e 1 or e 2 fires. Full-fledged embeddings of complex event processing like EventJava [3], or stream processing languages like SPL [5], support complex queries over event streams including time windows and joins. In parallel to the development of richer event models, other researchers focused on more inherent data-flow and changedriven solutions for reactive applications. These approaches have old roots. For example, the Garnet and Amulet graphical toolkits The fundamental concept in reactive languages is that programmers do not directly handle the control flow but the execution is driven by the implicit flow of data and the need to update values. Concretely, programmers specify constraints that express functional dependencies among values in the application, and the language runtime enforces these constraints without any further effort from the programmer. More recently, these approaches have inspired many embeddings of DSLs and functional constraints in existing (imperative) programming languages. The advantage of this solution is that programmers specify a functional dependency in an intuitive, declarative way. As a consequence, reactions are directly expressed, do not need to be inferred from the control flow, and can be easily composed. In practice, (continuous) time-changing values, a.k.a. signals, are not enough. The need for events (i.e., discrete timechanging values) is explained by two observations. (a) Events come from external phenomena that are inherently discrete, such as an interrupt or new data from a sensor. (b) Events are better suited for modeling certain behaviors: in principle a mouse click can be modeled as a boolean continuous time-changing value that switches to true when the mouse is clicked, but most programmers would rather think of a mouse click as an event. For this reason, existing reactive languages provide both signals and events. Reactive programming is an emerging trend and identifying the boundaries of this field is hard. However, the following principles seem valid in general. • Declarative style. Reactive behavior is defined in a direct, convenient, declarative style instead of encoding it in design patterns or through imperative updates of program state. Reactions are directly expressed and do not need to be encoded into the control flow of the program. • Composition. Abstractions allow for composition of more complex reactions. Traditional OO applications express reactions in callbacks that are executed when an observable changes. However, callbacks typically perform side effects to modify the state of the application but do not return a value. As a result, they are hard to combine. Instead, events can be combined through combinators, and signals can be combined directly into more complex reactive expressions. • Automation. In AOP, advices are triggered at points in the execution of the program (e.g., the end of a method call) that are referred to as join points. Join points can be seen as events that occur during the execution and treated uniformly with other events. For example, EScala before(method) and after(method) events are triggered before and after the execution of methods.Also, in event-based languages that integrate AOP features, programmers can refer to all events of a certain type, a feature that resembles AOP quantification. As an example of an expressive event system, we show a slice of a drawing application in EScala