Highly Scalable Algorithms for Robust String Barcoding

String barcoding is a recently introduced technique for genomic-based identification of microorganisms. In this paper we describe the engineering of highly scalable algorithms for robust string barcoding. Our methods enable distinguisher selection based on whole genomic sequences of hundreds of microorganisms of up to bacterial size on a well-equipped workstation, and can be easily parallelized to further extend the applicability range to thousands of bacterial size genomes. Experimental results on both randomly generated and NCBI genomic data show that whole-genome based selection results in a number of distinguishers nearly matching the information theoretic lower bounds for the problem

Performing work with asynchronous processors: Message-delay-sensitive bounds

AbstractThis paper considers the problem of performing tasks in asynchronous distributed settings. This problem, called Do-All, has been substantially studied in synchronous models, but there is a dearth of efficient algorithms for asynchronous message-passing processors. Do-All can be trivially solved without any communication by an algorithm where each processor performs all tasks. Assuming p processors and t tasks, this requires work Θ(p·t). Thus, it is important to develop subquadratic solutions (when p and t are comparable) by trading computation for communication. Following the observation that it is not possible to obtain subquadratic work when the message delay d is substantial, e.g., d=Θ(t), this work pursues a message-delay-sensitive approach. Here, the upper bounds on work and communication are given as functions of p, t, and d, the upper bound on message delays, however, algorithms have no knowledge of d and they cannot rely on the existence of an upper bound on d. This paper presents two families of asynchronous algorithms achieving, for the first time, subquadratic work as long as d=o(t). The first family uses as its basis a shared-memory algorithm without having to emulate atomic registers assumed by that algorithm. These deterministic algorithms have work O(tpε+pd⌈t/d⌉ε) for any ε>0. The second family uses specific permutations of tasks, with certain combinatorial properties, to sequence the work of the processors. These randomized (deterministic) algorithms have expected (worst-case) work O(tlogp+pdlog(2+t/d)). Another important contribution in this work is the first delay-sensitive lower bound for this problem that helps explain the behavior of our algorithms: any randomized (deterministic) algorithm has expected (worst-case) work of Ω(t+pdlogd+1t)

RAMBO Implementation HOW-TO

In this document we describe the implementation and use of RAMBO service. We assume that the reader has some background in IOA [6] and the existing RAMBO publications [4, 5, 3, 7, 2]. 1 RAMBO: Theory and Implementation This paper presents an implementation of a practical algorithm that implements long-lived, survivable, atomic read/write objects for dynamic networks, where participants may join, leave, or fail during the course of computation. Such settings are becoming increasingly common in modern distributed applications that rely on multitudes of communicating, computing devices. This is the case with highly dynamic environments, such as those encountered in major civilian rescue efforts and extensive military deployment operations. The only way to ensure survivability of data is through redundancy: the data is replicated and maintained at several network locations. Replication introduces the challenges of maintaining consistency among the replicas, and managing dynamic participation as the collections of network locations storing the replicas change due to arrivals, departures, and failures of nodes. A different approach to implementing atomic read/write objects for dynamic networks was developed by Lynch and Shvartsman [4] and extended by Gilbert et al. [3]. This memory service, called RAMBO (Reconfigurable Atomic Memory for Basic Objects) maintains atomic (linearizable) readable/writable data in highly dynamic environments. In order to achiev

Efficient Parallelism Vs Reliable Distribution: A Trade-Off for Concurrent Computations

. Concurrent computations should combine efficiency with reliability, where efficiency is usually associated with parallel and reliability with distributed computing. Such a desirable combination is not always possible, because of an intuitive trade-off: efficiency requires removing redundancy from computations whereas reliability requires some redundancy. We survey a spectrum of algorithmic models (from fail-stop, synchronous to asynchronous and from approximate to exact computations) in which reliability is guaranteed with small trade-offs in efficiency. We illustrate a number of cases where optimal trade-offs are achievable. A basic property of all these models, which is of some interest in the study of concurrency, is that "true" read/write concurrency is necessary for fault tolerance. In particular, we outline (from [14]) how algorithms can be designed so that, in each execution, the total "true" concurrency used can be closely related to the faults that can be tolerated. 1 Introd..

The Complexity of Distributed Cooperation in the Presence of Failures

Abstract. We consider the problem of performing N tasks in a distributed system of P processors that are subject to failures. An optimal solution would have the system perform Θ(N) tasks, however extant results show that this is impossible (when N = P) in the presence of an omniscient failure-inducing adversary. In this work we present complexity results for this problem for the case when the processors are assisted by a perfect load-balancing oracle in an attempt to foil the adversary. We generalize several existing results for this setting and we present new bounds for this problem that shed light on the behavior of deterministic distributed cooperation algorithms for the cases when the adversary imposes comparatively few failures. When a solution for the cooperation problem is implemented using a global load-balancing strategy, our results can be used to explain the work complexity of the solution. In particular, if the complexity of implementing the load-balancing oracle is known for a given model of computation, then our results can be directly translated into complexity results for that model

The complexity of synchronous iterative Do-All with crashes

Abstract. Do-All is the problem of performing N tasks in a distributed system of P failure-prone processors [8]. Many distributed and parallel algorithms have been developed for this problem and several algorithm simulations have been developed by iterating Do-All algorithms. The efficiency of the solutions for Do-All is measured in terms of work complexity where all processing steps taken by the processors are counted. We present the first non-trivial lower bounds for Do-All that capture the dependence of work on N, P and f, the number of processor crashes. For the model of computation where processors are able to make perfect load-balancing decisions locally, we also present matching upper bounds. We define the r-iterative Do-All problem that abstracts the repeated use of Do-All such as found in algorithm simulations. Our f-sensitive analysis enables us to derive a tight bound for r-iterative Do-All work (that is stronger than the r-fold work complexity of a single Do-All). Our approach that models perfect load-balancing allows for the analysis of specific algorithms to be divided into two parts: (i) the analysis of the cost of tolerating failures while performing work, and (ii) the analysis of the cost of implementing load-balancing. We demonstrate the utility and generality of this approach by improving the analysis of two known efficient algorithms. Finally we present a new upper bound on simulations of synchronous shared-memory algorithms on crash-prone processors.

Optimal Scheduling for Disconnected Cooperation

We consider a distributed environment consisting of n processors that need to perform t tasks. We assume that communication is initially unavailable and that processors begin work in isolation. At some unknown point of time an unknown collection of processors may establish communication. Before processors begin communication they execute tasks in the order given by their schedules. Our goal is to schedule work of isolated processors so that when communication is established for the rst time, the number of redundantly executed tasks is controlled. We quantify worst case redundancy as a function of processor advancements through their schedules. In this work we rene and simplify an extant deterministic construction for schedules with n t, and we develop a new analysis of its waste. The new analysis shows that for any pair of schedules, the number of redundant tasks can be controlled for the entire range of t tasks. Our new result is asymptotically optimal: the tails of these schedules are within a 1 +O(n 1 4 ) factor of the lower bound. We also present two new deterministic constructions one for t n, and the other for t n 3=2 , which substantially improve pairwise waste for all prexes of length t= p n, and oer near optimal waste for the tails of the schedules. Finally, we present bounds for waste of any collection of k 2 processors for both deterministic and randomized constructions.