2 research outputs found
Simplifying Parallelization of Scientific Codes by a Function-Centric Approach in Python
The purpose of this paper is to show how existing scientific software can be
parallelized using a separate thin layer of Python code where all parallel
communication is implemented. We provide specific examples on such layers of
code, and these examples may act as templates for parallelizing a wide set of
serial scientific codes. The use of Python for parallelization is motivated by
the fact that the language is well suited for reusing existing serial codes
programmed in other languages. The extreme flexibility of Python with regard to
handling functions makes it very easy to wrap up decomposed computational tasks
of a serial scientific application as Python functions. Many
parallelization-specific components can be implemented as generic Python
functions, which may take as input those functions that perform concrete
computational tasks. The overall programming effort needed by this
parallelization approach is rather limited, and the resulting parallel Python
scripts have a compact and clean structure. The usefulness of the
parallelization approach is exemplified by three different classes of
applications in natural and social sciences.Comment: 29 pages, submitted to Computational Science and Discover
Python-based Distributed Programming with Trickle
Abstract Trickle is a an extension to the Python programming language that provides explicit but simple mechanisms to write distributed scripts and programs. Trickle links together remote Python interpreters running on heterogeneous machines so that work can be deployed and results collected. A Trickle program interacts with remote interpreters by injecting functions or classes. Remote objects can be instantiated and invoked synchronously or asynchronously. Also, the injected code need not reside on the remote interpreters; code is dynamically transferred as needed. Trickle leverages off of Python’s list comprehensions and generators to simplify parallel invocation, result gathering, and dynamic scheduling. The Trickle run-time system uses a broadcast mechanism to find eligible Trickle virtual machines. Python programmers can immediately use Trickle to dispatch work to idle machines with minimal setup and easy to learn mechanisms. This paper describes the Trickle extension interface, its implementation, and presents an example application and its performance