General Purpose Parallel Computation on a DNA Substrate

In this paper I describe and extend a new DNA computing paradigm introduced in Blumberg for building massively parallel machines in the DNA-computing models described by Adelman, Cai et. al., and Liu et. al. Employing only DNA operations which have been reported as successfully performed, I present an implementation of a Connection Machine, a SIMD (single-instruction multiple-data) parallel computer as an illustration of how to apply this approach to building computers in this domain (and as an implicit demonstration of PRAM equivalence). This is followed with a description of how to implement a MIMD (multiple-instruction multiple-data) parallel machine. The implementations described herein differ most from existing models in that they employ explicit communication between processing elements (and hence strands of DNA)

Optimizing simulation on shared-memory platforms: The smart cities case

Modern advancements in computing architectures have been accompanied by new emergent paradigms to run Parallel Discrete Event Simulation models efficiently. Indeed, many new paradigms to effectively use the available underlying hardware have been proposed in the literature. Among these, the Share-Everything paradigm tackles massively-parallel shared-memory machines, in order to support speculative simulation by taking into account the limits and benefits related to this family of architectures. Previous results have shown how this paradigm outperforms traditional speculative strategies (such as data-separated Time Warp systems) whenever the granularity of executed events is small. In this paper, we show performance implications of this simulation-engine organization when the simulation models have a variable granularity. To this end, we have selected a traffic model, tailored for smart cities-oriented simulation. Our assessment illustrates the effects of the various tuning parameters related to the approach, opening to a higher understanding of this innovative paradigm

Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations

Gaussian processes (GP) are Bayesian non- parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank covariance matrix approximations for distributing the computational load among parallel machines to achieve time efficiency and scalability. We theoretically guarantee the predictive performance of our proposed parallel GPs to be equivalent to that of some centralized approximate GP regression methods: The computation of their centralized counterparts can be distributed among parallel machines, hence achieving greater time efficiency and scalability. We analytically compare the properties of our parallel GPs such as time, space, and communication complexity. Empirical evaluation on two real-world datasets in a cluster of 20 computing nodes shows that our parallel GPs are significantly more time-efficient and scalable than their centralized counterparts and exact/full GP while achieving predictive performances comparable to full GP

Parallel Algorithms for the Solution of the Schrodinger Equation

Many of the traditional numerical algorithms do not map easily onto the architecture of parallel computers that have emerged recently. For the economic use of these expensive machines and to reduce the total computing time, it is necessary to develop efficient parallel algorithms. The purpose of the thesis is to develop several parallel algorithms for the numerical solution of the Schrodinger equation which arises in many branches of atomic and molecular physics. Common models of systems which are of interest may represent stable configurations of two particles, the bound state or eigenvalue problem. Alternately one may consider either singlechannel or multi-channel scattering. All three mathematical models will be investigated in this work. Emphasis is placed on parallel algorithms for MIMD machines. All the algorithms have been implemented and tested on a transputer network which is a MIMD machine without shared memory. Existing numerical methods such as those ascribed to Numerov and De Vogelaere have been investigated and parallel versions of them have been developed. Two exponentially fitted versions of the De Vogelacre algorithm have been developed and they are found to be more efficient than the normal De Vogelaere algorithm

Models and heuristics for robust resource allocation in parallel and distributed computing systems

Includes bibliographical references.This is an overview of the robust resource allocation research efforts that have been and continue to be conducted by the CSU Robustness in Computer Systems Group. Parallel and distributed computing systems, consisting of a (usually heterogeneous) set of machines and networks, frequently operate in environments where delivered performance degrades due to unpredictable circumstances. Such unpredictability can be the result of sudden machine failures, increases in system load, or errors caused by inaccurate initial estimation. The research into developing models and heuristics for parallel and distributed computing systems that create robust resource allocations is presented.This research was supported by NSF under grant No. CNS-0615170 and by the Colorado State University George T. Abell Endowment

Dynamic Control Flow in Large-Scale Machine Learning

Many recent machine learning models rely on fine-grained dynamic control flow for training and inference. In particular, models based on recurrent neural networks and on reinforcement learning depend on recurrence relations, data-dependent conditional execution, and other features that call for dynamic control flow. These applications benefit from the ability to make rapid control-flow decisions across a set of computing devices in a distributed system. For performance, scalability, and expressiveness, a machine learning system must support dynamic control flow in distributed and heterogeneous environments. This paper presents a programming model for distributed machine learning that supports dynamic control flow. We describe the design of the programming model, and its implementation in TensorFlow, a distributed machine learning system. Our approach extends the use of dataflow graphs to represent machine learning models, offering several distinctive features. First, the branches of conditionals and bodies of loops can be partitioned across many machines to run on a set of heterogeneous devices, including CPUs, GPUs, and custom ASICs. Second, programs written in our model support automatic differentiation and distributed gradient computations, which are necessary for training machine learning models that use control flow. Third, our choice of non-strict semantics enables multiple loop iterations to execute in parallel across machines, and to overlap compute and I/O operations. We have done our work in the context of TensorFlow, and it has been used extensively in research and production. We evaluate it using several real-world applications, and demonstrate its performance and scalability.Comment: Appeared in EuroSys 2018. 14 pages, 16 figure

Developing Tools for Networks of Processors

A great deal of research eort is currently being made in the realm of so called natural computing. Natural computing mainly focuses on the denition, formal description, analysis, simulation and programming of new models of computation (usually with the same expressive power as Turing Machines) inspired by Nature, which makes them particularly suitable for the simulation of complex systems.Some of the best known natural computers are Lindenmayer systems (Lsystems, a kind of grammar with parallel derivation), cellular automata, DNA computing, genetic and evolutionary algorithms, multi agent systems, arti- cial neural networks, P-systems (computation inspired by membranes) and NEPs (or networks of evolutionary processors). This chapter is devoted to this last model