Process-Oriented Parallel Programming with an Application to Data-Intensive Computing

We introduce process-oriented programming as a natural extension of object-oriented programming for parallel computing. It is based on the observation that every class of an object-oriented language can be instantiated as a process, accessible via a remote pointer. The introduction of process pointers requires no syntax extension, identifies processes with programming objects, and enables processes to exchange information simply by executing remote methods. Process-oriented programming is a high-level language alternative to multithreading, MPI and many other languages, environments and tools currently used for parallel computations. It implements natural object-based parallelism using only minimal syntax extension of existing languages, such as C++ and Python, and has therefore the potential to lead to widespread adoption of parallel programming. We implemented a prototype system for running processes using C++ with MPI and used it to compute a large three-dimensional Fourier transform on a computer cluster built of commodity hardware components. Three-dimensional Fourier transform is a prototype of a data-intensive application with a complex data-access pattern. The process-oriented code is only a few hundred lines long, and attains very high data throughput by achieving massive parallelism and maximizing hardware utilization.Comment: 20 pages, 1 figur

Incrementally developing parallel applications with AspectJ

This paper presents a methodology to develop more modular parallel applications, based on aspect oriented programming. Traditional object oriented mechanisms implement application core functionality and parallelisation concerns are plugged by aspect oriented mechanisms. Parallelisation concerns are separated into four categories: functional or/and data partition, concurrency, distribution and optimisation. Modularising these categories into separate modules using aspect oriented programming enables (un)pluggability of parallelisation concerns. This approach leads to more incremental application development, easier debugging and increased reuse of core functionality and parallel code, when compared with traditional object oriented approaches. A detailed analysis of a simple parallel application - a prime number sieve - illustrates the methodology and shows how to accomplish these gains.Fundo Europeu de Desenvolvimento Regional (FEDER) - PPC-VM project POSI/CHS/47158/2002.Fundação para a Ciência e a Tecnologia (FCT) - PPC-VM project POSI/CHS/47158/2002

Parallel Computation in Econometrics: A Simplified Approach

Parallel computation has a long history in econometric computing, but is not at all wide spread. We believe that a major impediment is the labour cost of coding for parallel architectures. Moreover, programs for specific hardware often become obsolete quite quickly. Our approach is to take a popular matrix programming language (Ox), and implement a message-passing interface using MPI. Next, object-oriented programming allows us to hide the specific parallelization code, so that a program does not need to be rewritten when it is ported from the desktop to a distributed network of computers. Our focus is on so-called embarrassingly parallel computations, and we address the issue of parallel random number generation.Code optimization; Econometrics; High-performance computing; Matrix-programming language; Monte Carlo; MPI; Ox; Parallel computing; Random number generation.

Architecture for object-oriented programming model

Current mainstream architectures have ISAs that are not able to maintain all the information provided by the application programmer using a high level programming language. Typically, the information that is lost in compiling to a low-level ISA is related to parallelism and speculation [14]. For example some loops are typically expressed as parallel loops by the programmer but later the processor is not able to determine this level of parallelism; conditional execution might apply control independent execution that at execution time is basically impossible to detect; function and object-level parallelism is lost when code is transformed into a low-level ISA that is oblivious to programmer intentions and high-level programming structures. Object Oriented Programming Languages are arguably the most successful programming medium because they help the programmer to use well-known practices about data distribution through operations related with the associated data. Therefore object oriented models express data/execution locality more naturally and in an efficient manner. Other OO software mechanisms such as derivation and polymorphism further help the programmer to exploit locality better. Once object oriented programs have been compiled then all information about data/execution locality is completely lost in current assembly code (ISA code). Maintaining this information until runtime is crucial to improve locality and security. Finally, Object Oriented Programming Models maintain the idea of memory (data memory) far from the programmer. These are all desirable qualities that is mostly lost in the compilation to a low-level ISA that is oblivious to the Object-Oriented Programming model. This report considers implementing the Object Oriented (OO) Programming Model directly in the hardware to serve as a base to exploit object/level parallelism, speculation and heterogeneous computing. Towards this goal, we present new computer architecture that implements the OO Programming Models. All its hardware structures are objects and its Instruction Set directly utilizes objects hiding totally the notion of memory and other complex hardware structures. It also maintains all high-level programming language information until execution time. This enables efficient extraction of available parallelism in OO serial or parallel code at execution time with minimal compiler support. We will demonstrate the potential of this novel computer architecture through several examples.Postprint (published version

A Verified Integration of Imperative Parallel Programming Paradigms in an Object-Oriented Language

CC++ is a parallel object-oriented programming language that uses parallel composition, atomic functions, and single- assignment variables to express concurrency. We show that this programming paradigm is equivalent to several traditional imperative communication and synchronization models, namely: semaphores, monitors, and asynchronous channels. A collection of libraries which integrates these traditional models with CC++ is specified, implemented, and formally verified