Functional and dynamic programming in the design of parallel prefix networks

A parallel prefix network of width n takes n inputs, a1, a2, . . ., an, and computes each yi = a1 ○ a2 ○ ⋅ ⋅ ⋅ ○ ai for 1 ≤ i ≤ n, for an associative operator ○. This is one of the fundamental problems in computer science, because it gives insight into how parallel computation can be used to solve an apparently sequential problem. As parallel programming becomes the dominant programming paradigm, parallel prefix or scan is proving to be a very important building block of parallel algorithms and applications. There are many different parallel prefix networks, with different properties such as number of operators, depth and allowed fanout from the operators. In this paper, ideas from functional programming are combined with search to enable a deep exploration of parallel prefix network design. Networks that improve on the best known previous results are generated. It is argued that precise modelling in a functional programming language, together with simple visualization of the networks, gives a new, more experimental, approach to parallel prefix network design, improving on the manual techniques typically employed in the literature. The programming idiom that marries search with higher order functions may well have wider application than the network generation described here

Poster: Towards safe refactoring for intelligent parallelization of Java 8 streams

The Java 8 Stream API sets forth a promising new programming model that incorporates functional-like, MapReduce-style features into a mainstream programming language. However, using streams correctly and efficiently may involve subtle considerations. In this poster, we present our ongoing work and preliminary results towards an automated refactoring approach that assists developers in writing optimal stream code. The approach, based on ordering and typestate analysis, determines when it is safe and advantageous to convert streams to parallel and optimize a parallel streams

Challenging the abstraction penalty in parallel patterns libraries

In the last years, pattern-based programming has been recognized as a good practice for efficiently exploiting parallel hardware resources. Following this approach, multiple libraries have been designed for providing such high-level abstractions to ease the parallel programming. However, those libraries do not share a common interface. To pave the way, GrPPI has been designed for providing an intermediate abstraction layer between application developers and existing parallel programming frameworks like OpenMP, Intel TBB or ISO C++ threads. On the other hand, FastFlow has been adopted as an efficient object-based programming framework that may benefit from being supported as an additional GrPPI backend. However, the object-based approach presents some major challenges to be incorporated under the GrPPI type safe functional programming style. In this paper, we present the integration of FastFlow as a new GrPPI backend to demonstrate that structured parallel programming frameworks perfectly fit the GrPPI design. Additionally, we also demonstrate that GrPPI does not incur in additional overheads for providing its abstraction layer, and we study the programmability in terms of lines of code and cyclomatic complexity. In general, the presented work acts as reciprocal validation of both FastFlow (as an efficient, native structured parallel programming framework) and GrPPI (as an efficient abstraction layer on top of existing parallel programming frameworks).This work has been partially supported by the European Commission EU H2020-ICT-2014-1 Project RePhrase (No. 644235) and by the Spanish Ministry of Economy and Competitiveness through TIN2016-79637-P “Towards Unification of HPC and Big Data Paradigms”

A distributed object-oriented graphical programming system

technical reportThis report presents the design of a distributed parallel object system (DPOS) and its implementation using a graphical editing interface. DPOS brings together concepts of object-oriented programming and graphical programming with aspects of modern functional languages. Programs are defined as networks of active processes called "Process Objects" and interconnecting communications lines. These active objects are independent single threaded programs that employ much of the modularity, encapsulation of function, and encapsulation of data found in sequential object-oriented programming. The system defines a clear and simple approach to generating and managing parallelism and interprocess communication in a distributed parallel environment. DPOS contributes several new solutions to the problems of distributed parallel programming that are improvements over existing systems. The key improvements of this system include: a more complete and versatile means of dynamic process creation; the specification of complex network topologies in an intuitively clear and understandable way; seperation of the management of parallelism from the definition of computation; automatic resolution of low level critical section issues; the ability to design and develop separate processes as traditional single threaded programs; the encapsulation and incremental development of programs subnetworks; application of graphical programming concepts to high level programming

Poster: Towards safe refactoring for intelligent parallelization of Java 8 streams

The Java 8 Stream API sets forth a promising new programming model that incorporates functional-like, MapReduce-style features into a mainstream programming language. However, using streams correctly and efficiently may involve subtle considerations. In this poster, we present our ongoing work and preliminary results to- wards an automated refactoring approach that assists developers in writing optimal stream code. The approach, based on ordering and typestate analysis, determines when it is safe and advantageous to convert streams to parallel and optimize a parallel streams

Parallel Programming with the System Applications to Numerical Code Coupling

Writing parallel programs is not easy, and debugging them is usually a nightmare. To cope with these difficulties, a structured approach to parallel programming using skeletons and templates based compilation techniques has been developed over the past years by several researchers, including the P3L group in Pisa. The system marries the functional programming language with the skeletons, yielding a powerful parallel programming system and methodology: allows the programmer to write and debug a sequential version of his program (which, if not easy, could be considered as routine), and then the parallel version is automatically deduced by recompilation of the source program. The invaluable advantage of this approach is stagging: the programmer has just to concentrate on the easy part, the sequential programming, relieving on the system to obtain the hard part, the parallel version. As an additional benefit, the semantics adequacy of the sequential and parallel versions of the program is no more the programmer's concern: it is now the entire responsability of the compiler. In this paper, we report on the successful application of in the field of scientific computing, where the system has been used to solve a problem of numerical code coupling, obtaining parallelization for free. The interaction has been quite successful, as, in the process of solving the coupling problem, a wealth of new ideas have emerged on the design of the system, which are now incorporated in the current version of : coloring of virtual and physical computing network nodes to specify their relative mapping, and the new notion of parfuns or parallel computing sub-networks reified as functions at the programmer's level. Those two notions both increase efficiency and ease the writing of programs, being a step to smoother integration of parallel computing into the functional programming paradigm

Adding imperative programming to the pattern calculus

University of Technology, Sydney. Dept. of Software Engineering.By focusing on data and flow control, imperative languages provide a finely grained and efficient mechanism for directly manipulating state and memory. By focusing on functions, polymorphism increases the modularity and reusability of programs. The pattern calculus gives a new account of polymorphism over arbitrary datatypes which has been used as the foundation for building the functional language FISh2. The power of the new polymorphism is not limited to a functional setting and it can be extended into an imperative setting. The main contribution of this thesis is to expand the pattern calculus with imperative features and implement this within a version of FISh2. Two approaches are developed in expanding the calculus to imperative programming based on two setting: functional and imperative. Based on a functional setting, updatable locations are given separate location types; while based on an imperative setting, locations and their values share the same types. In both approaches, structured locations can be defined in the same way the calculus defines structured data. Hence, generic functions on locations can be defined by pattern-matching on (location) constructors. In that way, the power of the combination exceeds that of the boundary of functional or imperative alone. In particular, with the generic assignment function, we have a new approach on memory management which performs inplace update whenever it is reasonable to do so. Similar ideas could be used to extend the power of parametric polymorphism to parallel programming. To illustrate the approach, a key problem is addressed in detail, namely, distributing a data structure over a network of processors

Safe Parallelism: Compiler Analysis Techniques for Ada and OpenMP

There is a growing need to support parallel computation in Ada to cope with the performance requirements of the most advanced functionalities of safety-critical systems. In that regard, the use of parallel programming models is paramount to exploit the benefits of parallelism. Recent works motivate the use of OpenMP for being a de facto standard in high-performance computing for programming shared memory architectures. These works address two important aspects towards the introduction of OpenMP in Ada: the compatibility of the OpenMP syntax with the Ada language, and the interoperability of the OpenMP and the Ada runtimes, demonstrating that OpenMP complements and supports the structured parallelism approach of the tasklet model. This paper addresses a third fundamental aspect: functional safety from a compiler perspective. Particularly, it focuses on race conditions and considers the fine-grain and unstructured capabilities of OpenMP. Hereof, this paper presents a new compiler analysis technique that: (1) identifies potential race conditions in parallel Ada programs based on OpenMP or Ada tasks or both, and (2) provides solutions for the detected races.This work was supported by the Spanish Ministry of Science and Innovation under contract TIN2015-65316-P, and by the FCT (Portuguese Foundation for Science and Technology) within the CISTER Research Unit (CEC/04234).Peer ReviewedPostprint (author's final draft

A versatile programming model for dynamic task scheduling on cluster computers

This dissertation studies the development of application programs for parallel and distributed computer systems, especially PC clusters. A methodology is proposed to increase the efficiency of code development, the productivity of programmers and enhance performance of executing the developed programs on PC clusters while facilitating improvement of scalability and code portability of these programs. A new programming model, named the Super-Programming Model (SPM), is created. Programs are developed assuming an instruction set architecture comprised of SuperInstructions (SIs). SPM models the target system as a large Virtual Machine (VM); VM contains functional units which are underlain with sub-computer systems and SIs are implemented with codes. When these functional units execute SIs, their codes will run on member computers to perform the corresponding operations. This approach resembles the process of designing instruction sets for microprocessors but the VM employs much coarser instructions and data structures. SIs use Super-Data Blocks (SDBs) as their operands. Each SI is assigned to a single member computer and is indivisible (i.e., its implementation is not interrupted for I/O). SIs have predictable execution times because SDB sizes are limited by predefined thresholds. These qualities of SIs help dynamic load balancing. Employing software to implement instructions makes this approach more flexible. The developed programs fit to architectures of cluster systems better. SPM provides mechanisms, such as dynamic load balancing, to assure the efficient execution of programs. The vast majority of current programming models lack such mechanisms for distributed environments that suffer from long communication latencies. Since SPM employs coarse-grain tasks, the overall management overhead is small. SDB access can often overlap the execution of other SIs; a cache system further decreases average memory latencies. Since all SDBs are virtual entities, with the runtime system support, they can be accessed in parallel and efficiently minimizes additional constraints to parallelism from underlying computer systems. In this research, a reference implementation of VM has been developed. A performance estimation model is developed that takes these features into account. Finally, the definition of scalability for parallel/distributed processing is refined to represent a multi-dimensional entity. Sample cases are analyzed

Investigating the use of pure functional programming for agent-based simulation

This thesis investigates the use of pure functional programming in Agent-Based Simulation (ABS) with the language Haskell. The central theme of this thesis is to do with purity, which identifies the lack of unrestricted side effects and referential transparency. Thematically, the research presented in this thesis is split into two parts. The first part deals with the approach to a pure functional ABS implementation and the second part with exploring benefits enabled through pure functional programming. First, the thesis explores how to implement ABS in a pure functional way, discussing both a time and event-driven approach. In each case Arrowized Functional Reactive Programming is used to derive fundamental abstractions and concepts. As use cases the well known agent-based SIR and the Sugarscape models are used. Additionally, the thesis focuses on why it is beneficial to implement ABS in a pure functional way. For this research topic, we explore both robust parallel and concurrent programming, where the main focus is on how to speed up the simulation while keeping it as pure as possible. In the parallel part, we rely on built-in language features and are able to speed the simulation up while retaining purity. In the concurrency part, Software Transactional Memory is used, sacrificing purity but still retaining certain guarantees about reproducibility. Finally, the thesis explores automated code testing of ABS implementations using property-based testing to show how to encode agent specifications and model invariants and perform model verification and hypothesis testing. The contribution of this thesis is threefold: 1. Development of pure functional implementation techniques for ABS through the use of Arrowized Functional Reactive Programming. 2. Development of techniques using Software Transactional Memory to implement robust concurrent ABS. 3. Development of a new testing approach to ABS using randomised propertybased testing for declarative and stochastic code testing. The results of the respective contributions support the view that pure functional programming indeed has its place in ABS. First, a pure functional approach leads to implementations which are more likely to be valid due to the focus on purity by avoiding computations with unrestricted side effects. Secondly, pure parallel computation and Software Transactional Memory (lockfree) based concurrency make it possible to gain substantial speedup, with the latter one dramatically outperforming traditional lock-based approaches. While pure parallel computation fully retains static guarantees, Software Transactional Memory is not pure, but is still able to retain certain guarantees regarding reproducibility. Finally, property-based testing is shown to be extremely useful, as it naturally maps to the stochastic nature of ABS and is therefore suitable to be integrated into the development process as an additional tool for testing specifications and hypotheses