CPL: A Core Language for Cloud Computing -- Technical Report

Running distributed applications in the cloud involves deployment. That is, distribution and configuration of application services and middleware infrastructure. The considerable complexity of these tasks resulted in the emergence of declarative JSON-based domain-specific deployment languages to develop deployment programs. However, existing deployment programs unsafely compose artifacts written in different languages, leading to bugs that are hard to detect before run time. Furthermore, deployment languages do not provide extension points for custom implementations of existing cloud services such as application-specific load balancing policies. To address these shortcomings, we propose CPL (Cloud Platform Language), a statically-typed core language for programming both distributed applications as well as their deployment on a cloud platform. In CPL, application services and deployment programs interact through statically typed, extensible interfaces, and an application can trigger further deployment at run time. We provide a formal semantics of CPL and demonstrate that it enables type-safe, composable and extensible libraries of service combinators, such as load balancing and fault tolerance.Comment: Technical report accompanying the MODULARITY '16 submissio

Versatile event correlation with algebraic effects

We present the first language design to uniformly express variants of n -way joins over asynchronous event streams from different domains, e.g., stream-relational algebra, event processing, reactive and concurrent programming. We model asynchronous reactive programs and joins in direct style, on top of algebraic effects and handlers. Effect handlers act as modular interpreters of event notifications, enabling fine-grained control abstractions and customizable event matching. Join variants can be considered as cartesian product computations with ”degenerate” control flow, such that unnecessary tuples are not materialized a priori. Based on this computational interpretation, we decompose joins into a generic, naive enumeration procedure of the cartesian product, plus variant-specific extensions, represented in terms of user-supplied effect handlers. Our microbenchmarks validate that this extensible design avoids needless materialization. Alongside a formal semantics for joining and prototypes in Koka and multicore OCaml, we contribute a systematic comparison of the covered domains and features. ERC, Advanced Grant No. 321217 ERC, Consolidator Grant No. 617805 DFG, SFB 1053 DFG, SA 2918/2-

Event Correlation with Algebraic Effects - Theory, Design and Implementation

Modern software systems are increasingly becoming distributed, reactive, and data intensive. In this context, events are an essential abstraction for representing and communicating interesting situations, enabling loosely-coupled and scalable systems. An integral part of these systems is event correlation, which is about computing relations between observed event notifications in space and time, e.g., to programmatically reason about the state of the system, for reacting to changes, for synchronization, for coordination or data processing. The importance of event correlation is witnessed by the considerable attention it has received from different research communities, under different names. This has led to the emergence of different "families" of event correlation approaches with a sheer staggering amount of specialized correlation semantics and software abstractions. We distinguish between four families: complex event processing, stream processing, reactive programming, and concurrent programming. We face a rather dire state of affairs: Existing event correlation approaches lack clarity, and most of them are not designed to be customizable or composable. Software systems evolve and their requirements change over time. Yet, no single approach is sufficiently adaptable to meet all requirements. We lack appropriate conceptual models and software abstractions that enable modular, extensible and cross-cutting compositions of features among the different event correlation families, fostering customizability and adaptability. It is time for a new generation of unifying, highly adaptable and customizable reactive software systems, by means of event correlation "à la carte". That is, we need language designs for event correlation that can express features from all families in a uniform, extensible and freely composable manner. This thesis makes event correlation "à la carte" a reality. We solve this issue by principled application of functional programming, in particular algebraic effects and effect handlers, which are a modern take on integrating side effects into functional languages in a modular and extensible way. The main statement of the thesis is that algebraic effects and effect handlers are a good programming abstraction for obtaining principled, versatile event correlation systems. To validate the thesis, we develop Cartesius, the first computational model that captures the essence of event correlation. It has an extensible and customiz- able semantics based on algebraic effects and effect handlers. We view instances of event correlation as cartesian products over streams of events, where user-defined effect handlers locally customize the control flow of the computation to obtain semantic variants that behave differently. Handlers compose freely and new kinds of correlation features can be added by introducing new user-defined side effects. Furthermore, we complement Cartesius with PolyJoin, which is an extensible programming language integration of declarative frontend syntax for event correlation systems. It can be deployed independently of Cartesius to provide a type-safe frontend for other event correlation systems in a programming language and subsumes mainstream language-integrated query techniques. To evaluate our approach, we conduct a survey, comparing the expressivity of Cartesius against representative systems from the above event correlation families. Our findings demonstrate that Cartesius captures the essence of other system's features and is fully customizable and adaptable, to a degree none of the surveyed event correlation approaches offer. Additionally, we evaluate the performance of our approach in terms of synthetic microbenchmarks on common event correlation variants. The benchmarks demonstrate that despite the modular decomposition into an expensive cartesian product and user-defined extensions, Cartesius' design is practical and achieves characteristic time and space complexity of the considered event correlation variants. Our "à la carte" approach based on algebraic effects and handlers exhibits a degree of unification, fine-grained customization and hybridization that no previous work on event correlation has attained

Event Correlation with Algebraic Effects - Theory, Design and Implementation

Modern software systems are increasingly becoming distributed, reactive, and data intensive. In this context, events are an essential abstraction for representing and communicating interesting situations, enabling loosely-coupled and scalable systems. An integral part of these systems is event correlation, which is about computing relations between observed event notifications in space and time, e.g., to programmatically reason about the state of the system, for reacting to changes, for synchronization, for coordination or data processing. The importance of event correlation is witnessed by the considerable attention it has received from different research communities, under different names. This has led to the emergence of different "families" of event correlation approaches with a sheer staggering amount of specialized correlation semantics and software abstractions. We distinguish between four families: complex event processing, stream processing, reactive programming, and concurrent programming. We face a rather dire state of affairs: Existing event correlation approaches lack clarity, and most of them are not designed to be customizable or composable. Software systems evolve and their requirements change over time. Yet, no single approach is sufficiently adaptable to meet all requirements. We lack appropriate conceptual models and software abstractions that enable modular, extensible and cross-cutting compositions of features among the different event correlation families, fostering customizability and adaptability. It is time for a new generation of unifying, highly adaptable and customizable reactive software systems, by means of event correlation "à la carte". That is, we need language designs for event correlation that can express features from all families in a uniform, extensible and freely composable manner. This thesis makes event correlation "à la carte" a reality. We solve this issue by principled application of functional programming, in particular algebraic effects and effect handlers, which are a modern take on integrating side effects into functional languages in a modular and extensible way. The main statement of the thesis is that algebraic effects and effect handlers are a good programming abstraction for obtaining principled, versatile event correlation systems. To validate the thesis, we develop Cartesius, the first computational model that captures the essence of event correlation. It has an extensible and customiz- able semantics based on algebraic effects and effect handlers. We view instances of event correlation as cartesian products over streams of events, where user-defined effect handlers locally customize the control flow of the computation to obtain semantic variants that behave differently. Handlers compose freely and new kinds of correlation features can be added by introducing new user-defined side effects. Furthermore, we complement Cartesius with PolyJoin, which is an extensible programming language integration of declarative frontend syntax for event correlation systems. It can be deployed independently of Cartesius to provide a type-safe frontend for other event correlation systems in a programming language and subsumes mainstream language-integrated query techniques. To evaluate our approach, we conduct a survey, comparing the expressivity of Cartesius against representative systems from the above event correlation families. Our findings demonstrate that Cartesius captures the essence of other system's features and is fully customizable and adaptable, to a degree none of the surveyed event correlation approaches offer. Additionally, we evaluate the performance of our approach in terms of synthetic microbenchmarks on common event correlation variants. The benchmarks demonstrate that despite the modular decomposition into an expensive cartesian product and user-defined extensions, Cartesius' design is practical and achieves characteristic time and space complexity of the considered event correlation variants. Our "à la carte" approach based on algebraic effects and handlers exhibits a degree of unification, fine-grained customization and hybridization that no previous work on event correlation has attained

CPL:a core language for cloud computing

Running distributed applications in the cloud involves deployment. That is, distribution and configuration of application services and middleware infrastructure. The considerable complexity of these tasks resulted in the emergence of declarative JSON-based domain-specific deployment languages to develop deployment programs. However, existing deployment programs unsafely compose artifacts written in different languages, leading to bugs that are hard to detect before run time. Furthermore, deployment languages do not provide extension points for custom implementations of existing cloud services such as application-specific load balancing policies. To address these shortcomings, we propose CPL (Cloud Platform Language), a statically-typed core language for programming both distributed applications as well as their deployment on a cloud platform. In CPL, application services and deployment programs interact through statically typed, extensible interfaces, and an application can trigger further deployment at run time. We provide a formal semantics of CPL and demonstrate that it enables type-safe, composable and extensible libraries of service combinators, such as load balancing and fault tolerance

Versatile event correlation with algebraic effects

We present the first language design to uniformly express variants of n -way joins over asynchronous event streams from different domains, e.g., stream-relational algebra, event processing, reactive and concurrent programming. We model asynchronous reactive programs and joins in direct style, on top of algebraic effects and handlers. Effect handlers act as modular interpreters of event notifications, enabling fine-grained control abstractions and customizable event matching. Join variants can be considered as cartesian product computations with ”degenerate” control flow, such that unnecessary tuples are not materialized a priori. Based on this computational interpretation, we decompose joins into a generic, naive enumeration procedure of the cartesian product, plus variant-specific extensions, represented in terms of user-supplied effect handlers. Our microbenchmarks validate that this extensible design avoids needless materialization. Alongside a formal semantics for joining and prototypes in Koka and multicore OCaml, we contribute a systematic comparison of the covered domains and features. ERC, Advanced Grant No. 321217 ERC, Consolidator Grant No. 617805 DFG, SFB 1053 DFG, SA 2918/2-