The paradigm compiler: Mapping a functional language for the connection machine

The Paradigm Compiler implements a new approach to compiling programs written in high level languages for execution on highly parallel computers. The general approach is to identify the principal data structures constructed by the program and to map these structures onto the processing elements of the target machine. The mapping is chosen to maximize performance as determined through compile time global analysis of the source program. The source language is Sisal, a functional language designed for scientific computations, and the target language is Paris, the published low level interface to the Connection Machine. The data structures considered are multidimensional arrays whose dimensions are known at compile time. Computations that build such arrays usually offer opportunities for highly parallel execution; they are data parallel. The Connection Machine is an attractive target for these computations, and the parallel for construct of the Sisal language is a convenient high level notation for data parallel algorithms. The principles and organization of the Paradigm Compiler are discussed

Execution time supports for adaptive scientific algorithms on distributed memory machines

Optimizations are considered that are required for efficient execution of code segments that consists of loops over distributed data structures. The PARTI (Parallel Automated Runtime Toolkit at ICASE) execution time primitives are designed to carry out these optimizations and can be used to implement a wide range of scientific algorithms on distributed memory machines. These primitives allow the user to control array mappings in a way that gives an appearance of shared memory. Computations can be based on a global index set. Primitives are used to carry out gather and scatter operations on distributed arrays. Communications patterns are derived at runtime, and the appropriate send and receive messages are automatically generated

SNAP: Stateful Network-Wide Abstractions for Packet Processing

Early programming languages for software-defined networking (SDN) were built on top of the simple match-action paradigm offered by OpenFlow 1.0. However, emerging hardware and software switches offer much more sophisticated support for persistent state in the data plane, without involving a central controller. Nevertheless, managing stateful, distributed systems efficiently and correctly is known to be one of the most challenging programming problems. To simplify this new SDN problem, we introduce SNAP. SNAP offers a simpler "centralized" stateful programming model, by allowing programmers to develop programs on top of one big switch rather than many. These programs may contain reads and writes to global, persistent arrays, and as a result, programmers can implement a broad range of applications, from stateful firewalls to fine-grained traffic monitoring. The SNAP compiler relieves programmers of having to worry about how to distribute, place, and optimize access to these stateful arrays by doing it all for them. More specifically, the compiler discovers read/write dependencies between arrays and translates one-big-switch programs into an efficient internal representation based on a novel variant of binary decision diagrams. This internal representation is used to construct a mixed-integer linear program, which jointly optimizes the placement of state and the routing of traffic across the underlying physical topology. We have implemented a prototype compiler and applied it to about 20 SNAP programs over various topologies to demonstrate our techniques' scalability

Deductive Optimization of Relational Data Storage

Optimizing the physical data storage and retrieval of data are two key database management problems. In this paper, we propose a language that can express a wide range of physical database layouts, going well beyond the row- and column-based methods that are widely used in database management systems. We use deductive synthesis to turn a high-level relational representation of a database query into a highly optimized low-level implementation which operates on a specialized layout of the dataset. We build a compiler for this language and conduct experiments using a popular database benchmark, which shows that the performance of these specialized queries is competitive with a state-of-the-art in memory compiled database system

A class-based approach to parallelization of legacy codes

Computation-intensive legacy codes for numerical models stand to benefit from application of parallel computing. However, parallelization of legacy codes poses special challenges. These codes are very large and complex. Manual parallelization has proven to be extremely time-consuming and error-prone. Furthermore, while a large number of parallelization tools exist, they cannot handle these complex legacy codes. Development of automatic parallelization tools for legacy codes remains a research area of considerable interest;This thesis describes a new approach to automatic parallelization of legacy codes. Our approach focuses on special classes of codes as opposed to parallelization of arbitrary codes. The advantage is that we are able to use high-level knowledge of the special class to manage the complexity of the parallelization problem. This approach provides a pragmatic solution for parallelization: it requires the user to specify the high-level knowledge, but automates tasks which are time-consuming, tedious, and error-prone for the user;Using this new approach, we have developed parAgent--a parallelizing tool which facilitates quick development of efficient parallel codes for legacy Fortran-77 codes based on the explicit time-marching finite difference model. parAgent has been used on several well-known and widely-used Mesoscale Meteorological codes. It took only a few weeks to parallelize each of these legacy codes. Qualitatively, the performance of parallelization have been found to be on par with manual parallelization;This new approach can be applied to a variety of problem domains. The key benefits are: substantial reuse of existing software and considerable saving of time and effort for developing efficient parallel code. Although the new approach and parAgent have been developed with parallelization as the main objective, the information provided by the tool can be used for various purposes. For example, the information about the underlying numerical method and the exchange of data is valuable to the application scientist