Improving interoperability in distributed multi-tier software stacks

Distributed multi-tier software stacks organise and deploy software components as a hierarchy of interacting tiers. The components are typically heterogeneous, i.e. each component may be written in a different language and may interoperate using a variety of protocols. Tiered software is modular but leads to a range of interoperability challenges including the following. (1) Interoperating components in multiple languages and paradigms increases developer cognitive load since they must simultaneously reason in multiple languages and paradigms. (2) There must be correct interoperation of components, e.g. adherence to the API or communication protocols between components. (3) Interoperation between different components can lead to diverse modes of failure as each component can fail in unique ways. Many of these challenges are the result of contributing factors like tight coupling or polyglot programmming. This thesis investigates techniques to improve heterogeneous interoperability in distributed multi-tier software stacks. Some common approaches include microservices and tierless languages. Microservices are perceived to offer better reliability than components in multi-tier software stacks through the loose coupling of services. The reliability of microservices is investigated by combining the established properties of dependence and state with reliability. This defines a new three-dimensional space: the Microservices Dependency State Reliability (MDSR) classification with six classes. The feasibility of statically identifying MDSR classes is demonstrated with a prototype analyser that identifies all six classes in Flask microservices web applications. The reliability implications of the different MDSR classes are evaluated by running three case study applications (Hipster-Shop, JPyL & WordPress) against a fault injector. Key results are as follows. (1) All applications fail catastrophically if a critical microservice fails. (2) Applications survive the failure of individual minor microservice(s). (3) The failure of any chain of microservices in JPyL & Hipster is catastrophic. (4) Individual microservices do not necessarily have minor reliability implications. In a tierless language, the compiler generates the code for each component and ensures their correct interoperation. They are mainly used to implement web stacks. However, their use in implementing IoT stacks is less common. This investigation compares interoperation in tiered and tierless IoT stacks through the systematic evaluation of four implementations of the prototype UoG smart campus IoT system: two tierless and two Python-based tiered. Key results of the study are as follows. (1) Tierless languages have the potential to significantly reduce the development effort for IoT systems, requiring 70% less code than the tiered implementations. (2) Tierless languages have the potential to significantly improve the reliability of IoT systems. (3) The first comparison of a tierless codebase for resource-rich sensor nodes and one for resourceconstrained sensor nodes shows that they have very similar functional structure and code sizes - within 7%. Tier elimination is a technique that removes a tier/component by integrating two tiers. Specifically, this thesis investigates the implications of eliminating the Apache web server in a 4-tier web stack: Jupyter Notebook, Apache, Python, Linux (JAPyL) and replacing it with PHP libraries in the frontend webpage to get the 3-tier (JPL). The study reveals the following. (1) The JPL 3-tier web stack requires that the developer uses fewer programming languages and paradigms than JAPyL, i.e two compared with four languages and two compared with three paradigms. (2) JPL requires 42% less code than JAPyL. (3) In JPL, some of the functionalities can be automated due to the decreased abstraction levels at the upper layers of the stack. (4) However, the latency in JPL is two to three times greater than that of JAPyL. So while tier elimination reduces developer effort and semantic friction the tradeoffs are high performance overhead & resource consumption and increasing code complexity

Eliom: A core ML language for Tierless Web programming

International audienceEliom is a dialect of OCaml for Web programming in which server and client pieces of code can be mixed in the same file using syntactic annotations. This allows to build a whole application as a single distributed program, in which it is possible to define in a composable way reusable widgets with both server and client behaviors. Our language also enables simple and type-safe communication. Eliom matches the specificities of the Web by allowing the programmer to interleave client and server code while maintaining efficient one-way server-to-client communication. The Eliom language is both sufficiently small to be implemented on top of an existing language and sufficiently powerful to allow expressing many idioms of Web programming. In this paper, we present a formalization of the core language of Eliom. We provide a type system, the execution model and a compilation scheme

Could tierless languages reduce IoT development grief?

Internet of Things (IoT) software is notoriously complex, conventionally comprising multiple tiers. Traditionally an IoT developer must use multiple programming languages and ensure that the components interoperate correctly. A novel alternative is to use a single tierless language with a compiler that generates the code for each component and ensures their correct interoperation. We report a systematic comparative evaluation of two tierless language technologies for IoT stacks: one for resource-rich sensor nodes (Clean with iTask), and one for resource-constrained sensor nodes (Clean with iTask and mTask). The evaluation is based on four implementations of a typical smart campus application: two tierless and two Python-based tiered. (1) We show that tierless languages have the potential to significantly reduce the development effort for IoT systems, requiring 70% less code than the tiered implementations. Careful analysis attributes this code reduction to reduced interoperation (e.g. two embedded domain-specific languages (DSLs) and one paradigm versus seven languages and two paradigms), automatically generated distributed communication, and powerful IoT programming abstractions. (2) We show that tierless languages have the potential to significantly improve the reliability of IoT systems, describing how Clean iTask/mTask maintains type safety, provides higher order failure management, and simplifies maintainability. (3) We report the first comparison of a tierless IoT codebase for resource-rich sensor nodes with one for resource-constrained sensor nodes. The comparison shows that they have similar code size (within 7%), and functional structure. (4) We present the first comparison of two tierless IoT languages, one for resource-rich sensor nodes, and the other for resource-constrained sensor nodes

Functional EDSLs for Web Applications

This thesis aims to make the development of complex web applications easier,faster and safer through the application of strongly typedfunctional programming techniques.Traditional web applications are commonly written in the de facto standardlanguage of the web, JavaScript, which, being untyped, provides no guaranteesregarding the data processed by programs, increasing the burden of testing anddefensive programming.Modern web applications are often highly complex, with multipleinterdependent parts interacting over the Internet.Such applications are traditionally implemented with each component as aseparate program, exposing its functionality to other components throughdifferent API:s over some communication protocol such as HTTP.This process is mostly manual, and thus error-prone and labour intensive,with accidental API incompatibility between components being particularlyproblematic.Even in a conventional typed language, the absence of such incompatibilitiesis not guaranteed.While the different components may well be type-safe in isolation, there is noguarantee that the whole is type-safe as the communication between componentsis not type-checked.We present a web application development framework, based on the Haskellprogramming language, to increase programmer productivity and software qualityby addressing these issues.In our framework, applications with an arbitrary number of components arewritten, compiled and type-checked as a single program, guaranteeing that theapplication as a whole, including network communication, is type-safe.Communication between components is automatically generated by our framework,eliminating the risk of API incompatibilities completely.Additionally, our framework includes astate-of-the-art compiler from Haskell to JavaScript,a novel, lightweight foreign function interface to allow programs to leverageexisting JavaScript code, an embedded language for integrating low-level,high-performance kernels into otherwise high-level web applications, as well asa highly expressive relational database language

Multitier Modules

Multitier programming languages address the complexity of developing distributed systems abstracting over low level implementation details such as data representation, serialization and network protocols. Since the functionalities of different peers can be defined in the same compilation unit, multitier languages do not force developers to modularize software along network boundaries. Unfortunately, combining the code for all tiers into the same compilation unit poses a scalability challenge or forces developers to resort to traditional modularization abstractions that are agnostic to the multitier nature of the language. In this paper, we address this issue with a module system for multitier languages. Our module system supports encapsulating each (cross-peer) functionality and defining it over abstract peer types. As a result, we disentangle modularization and distribution and we enable the definition of a distributed system as a composition of multitier modules, each representing a subsystem. Our case studies on distributed algorithms, distributed data structures, as well as on the Apache Flink task distribution system, show that multitier modules allow the definition of reusable (abstract) patterns of interaction in distributed software and enable separating the modularization and distribution concerns, properly separating functionalities in distributed systems

A Glimpse of Hopjs

International audienceHop.js is a multitier programming environment for JavaScript. Itallows a single JavaScript program to describe the client-side and theserver-side components of a web application. Its runtime environmentensures consistent executions of the application on the server and onthe client.This paper overviews the Hop.js design. It shows the JavaScriptextensions that makes it possible to conceive web applicationsglobally. It presents how Hop.js interacts with the outside world. Italso briefly presents the Hop.js implementation. It presents theHop.js web server implementation, the handling of server-sideparallelism, and the JavaScript and HTML compilers

Typed concurrent functional programming with channels, actors and sessions

The age of writing single-threaded applications is over. To develop scalable applications, developers must make use of concurrency and parallelism. Nonetheless, introducing concurrency and parallelism is difficult: naïvely implemented, concurrent code is prone to issues such as race conditions and deadlocks. Moving to the distributed setting introduces yet more issues, in particular the possibility of failure. To cope with many of the problems of concurrent programming, language designers have proposed a class of programming languages known as communication-centric programming languages, which provide lightweight processes which do not share memory, but instead communicate using explicit message passing. The focus of this thesis is on typed communication-centric functional programming languages, using type systems to provide static guarantees about the runtime behaviour of concurrent programs. We investigate two strands of work: the relationship between typed channel- and actor-based languages, and the integration of asynchrony, exception handling, and session types in a functional programming language. In the first strand, we investigate two particular subclasses of communication-centric languages: channel-based languages such as Go, and languages based on the actor model, such as Erlang. We distil the essence of the languages into two concurrent λ-calculi: λch for simply-typed channels, and lact for simply-typed actors, and provide type- and semantics-preserving translations between them. In doing so, we clear up confusion between the two models, give theoretical foundations for recent implementations of type-parameterised actors, and also provide a theoretical grounding for frameworks which emulate actors in channel-based languages. Furthermore, by extending the core calculi, we note that actor synchronisation drastically simplifies the translation from channels into actors, and show that Erlang’s selective receive mechanism can be implemented without specialised constructs. In the second strand, we integrate session types, asynchrony, and exception handling in a functional programming language. Session types are a behavioural type system for communication channel endpoints, allowing conformance to protocols to be checked statically. We provide the first integration of exception handling and asynchronous session types in a core functional language, Exceptional GV, and prove that it satisfies preservation, global progress, and that it is confluent and terminating. We demonstrate the practical applicability of the approach by extending the Links tierless web programming language with exception handling, in turn providing the first implementation of exception handling in the presence of session types in a functional language. As a result, we show the first application of session types to web programming, providing examples including a two-factor authentication workflow and a chat application