Tools for monitoring and controlling distributed applications

The Meta system is a UNIX-based toolkit that assists in the construction of reliable reactive systems, such as distributed monitoring and debugging systems, tool integration systems and reliable distributed applications. Meta provides mechanisms for instrumenting a distributed application and the environment in which it executes, and Meta supplies a service that can be used to monitor and control such an instrumented application. The Meta toolkit is built on top of the ISIS toolkit; they can be used together in order to build fault-tolerant and adaptive, distributed applications

A Usability Study of End-User Construction of Direct Manipulation User Interfaces

This paper describes an empirical study of end-users that tested the usability of The Programmers\u27 Playground graphical environment. The Programmers\u27 Playground is a software library and run-time system for constructing distributed multimedia applications. Playground\u27s graphical environment enables end-users to create direct manipulation graphical user interfaces (GUIs) and to dynamically configure communication among distributed application components. In this study, 28 end-users with no prior experience in distributed computing or user interface construction were timed and evaluated on several tasks using our graphical environment. Tasks included the use of direct and indirect constraint relationships, visual configuration of distributed applications, and graphical user interface construction. The results show that a wide variety of end-users (i.e., not just programmers) can learn and apply these concepts, utilizing our graphical environment to constructe distributed multimedia applications

Building Interactive Distributed Applications in C++ with The Programmers\u27 Playground

The objective of The Programmers\u27 Playground, described in this manual, is to provide a development environment and underlying support for end-user construction of distributed multimedia applications from reusable self-describing software components. Playground provides a set of software tools and a methodology for simplifying the design and construction of applications that interact with each other and with people in a distributed computer system. This manual explains how to write interactive distributed applications using Playground. The only background necessary to get started is an understanding of basic data structures and control constructs in C++. If you already know C++, then with the tools provided by Playground, you will be able to write distributed applications without learning a new programming language and without needing to learn about how communication works in a distributed system

EOS: A project to investigate the design and construction of real-time distributed embedded operating systems

The EOS project is investigating the design and construction of a family of real-time distributed embedded operating systems for reliable, distributed aerospace applications. Using the real-time programming techniques developed in co-operation with NASA in earlier research, the project staff is building a kernel for a multiple processor networked system. The first six months of the grant included a study of scheduling in an object-oriented system, the design philosophy of the kernel, and the architectural overview of the operating system. In this report, the operating system and kernel concepts are described. An environment for the experiments has been built and several of the key concepts of the system have been prototyped. The kernel and operating system is intended to support future experimental studies in multiprocessing, load-balancing, routing, software fault-tolerance, distributed data base design, and real-time processing

Reliable scientific service compositions

Abstract. Distributed service oriented architectures (SOAs) are increas-ingly used by users, who are insufficiently skilled in the art of distributed system programming. A good example are computational scientists who build large-scale distributed systems using service-oriented Grid comput-ing infrastructures. Computational scientists use these infrastructure to build scientific applications, which are composed from basic Web ser-vices into larger orchestrations using workflow languages, such as the Business Process Execution Language. For these users reliability of the infrastructure is of significant importance and that has to be provided in the presence of hardware or operational failures. The primitives avail-able to achieve such reliability currently leave much to be desired by users who do not necessarily have a strong education in distributed sys-tem construction. We characterise scientific service compositions and the environment they operate in by introducing the notion of global scien-tific BPEL workflows. We outline the threats to the reliability of such workflows and discuss the limited support that available specifications and mechanisms provide to achieve reliability. Furthermore, we propose a line of research to address the identified issues by investigating auto-nomic mechanisms that assist computational scientists in building, exe-cuting and maintaining reliable workflows.

Optimizing CMS build infrastructure via Apache Mesos

The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux. Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. We present how we migrated our continuos integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos.Comment: Submitted to proceedings of the 21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015), Okinawa, Japa

Kulla, a container-centric construction model for building infrastructure-agnostic distributed and parallel applications

This paper presents the design, development, and implementation of Kulla, a virtual container-centric construction model that mixes loosely coupled structures with a parallel programming model for building infrastructure-agnostic distributed and parallel applications. In Kulla, applications, dependencies and environment settings, are mapped with construction units called Kulla-Blocks. A parallel programming model enables developers to couple those interoperable structures for creating constructive structures named Kulla-Bricks. In these structures, continuous dataflow and parallel patterns can be created without modifying the code of applications. Methods such as Divide&Containerize (data parallelism), Pipe&Blocks (streaming), and Manager/Block (task parallelism) were developed to create Kulla-Bricks. Recursive combinations of Kulla instances can be grouped in deployment structures called Kulla-Boxes, which are encapsulated into VCs to create infrastructure-agnostic parallel and/or distributed applications. Deployment strategies were created for Kulla-Boxes to improve the IT resource profitability. To show the feasibility and flexibility of this model, solutions combining real-world applications were implemented by using Kulla instances to compose parallel and/or distributed system deployed on different IT infrastructures. An experimental evaluation based on use cases solving satellite and medical image processing problems revealed the efficiency of Kulla model in comparison with some traditional state-of-the-art solutions.This work has been partially supported by the EU project "ASPIDE: Exascale Programing Models for Extreme Data Processing" under grant 801091 and the project "CABAHLA-CM: Convergencia Big data-Hpc: de los sensores a las Aplicaciones" S2018/TCS-4423 from Madrid Regional Government

Supporting Live Development of SOAP and CORBA Clients

We present middleware for a Client Development Environment that facilitates live development of client applications for SOAP or CORBA servers. We use JPie, a tightly integrated programming environment for live software construction in Java, as the target platform for our design. JPie provides dynamic classes whose signature and implementation can be modified at run time, with changes taking effect immediately upon existing instances of the class. We extend this model to automate addition, mutation, and deletion of dynamic server methods within dynamic clients. Our implementation simplifies distributed application development by masking technical differences between local and remote method invocations. Moreover, the live development model allows server-side changes to be dynamically integrated into a running client to support simultaneous live development of both the client and server

Development of a Photovoltaic Array Emulator in a Real Time Control Environment Using xPC Target

Part 12: Integration of Power Electronics Systems with ICT - IIInternational audienceThis paper is devoted to the design and construction of a photovoltaic array emulator for high power applications in order to test all kind of photovoltaic inverters. To develop such device, a rapid prototyping tool based on xPC Target of Matlab/Simulink has been used, providing a real-time testing environment. PV array emulator can be used to evaluate the performance of photovoltaic inverters as any test conditions can be programmed. The proposed emulator operates as a distributed control system taking advantage of the TCP/IP protocol features

Formal Specification of a Dynamically Configurable Distributed System

The Programmers\u27 Playground is a programming environment that supports end-user construction of distributed multimedia applications. The system implements a new programming model that is based, in part, upon ideas from the formal I/O automaton model of Lynch and Tuttle. Important features of The Programmers\u27 Playground are a separation of communication and computation and graphical support for dynamic reconfiguration. This paper provides a formal specification of the Playground programming model and runtime system in terms of the I/O automaton model on which it is based. Exploiting the compositionality properties of the I/O automaton model, the formal specification is describd as a composition of several modules. A behavioral specification of each module is presented, followed by an I/O automaton that implements each specification. We present the specification in two stages, a centralized specification that captures the allowable behaviors, and then a detailed distributed implementation