A Bulk-Parallel Priority Queue in External Memory with STXXL

We propose the design and an implementation of a bulk-parallel external memory priority queue to take advantage of both shared-memory parallelism and high external memory transfer speeds to parallel disks. To achieve higher performance by decoupling item insertions and extractions, we offer two parallelization interfaces: one using "bulk" sequences, the other by defining "limit" items. In the design, we discuss how to parallelize insertions using multiple heaps, and how to calculate a dynamic prediction sequence to prefetch blocks and apply parallel multiway merge for extraction. Our experimental results show that in the selected benchmarks the priority queue reaches 75% of the full parallel I/O bandwidth of rotational disks and and 65% of SSDs, or the speed of sorting in external memory when bounded by computation.Comment: extended version of SEA'15 conference pape

CloudTree: A Library to Extend Cloud Services for Trees

In this work, we propose a library that enables on a cloud the creation and management of tree data structures from a cloud client. As a proof of concept, we implement a new cloud service CloudTree. With CloudTree, users are able to organize big data into tree data structures of their choice that are physically stored in a cloud. We use caching, prefetching, and aggregation techniques in the design and implementation of CloudTree to enhance performance. We have implemented the services of Binary Search Trees (BST) and Prefix Trees as current members in CloudTree and have benchmarked their performance using the Amazon Cloud. The idea and techniques in the design and implementation of a BST and prefix tree is generic and thus can also be used for other types of trees such as B-tree, and other link-based data structures such as linked lists and graphs. Preliminary experimental results show that CloudTree is useful and efficient for various big data applications

GPU LSM: A Dynamic Dictionary Data Structure for the GPU

We develop a dynamic dictionary data structure for the GPU, supporting fast insertions and deletions, based on the Log Structured Merge tree (LSM). Our implementation on an NVIDIA K40c GPU has an average update (insertion or deletion) rate of 225 M elements/s, 13.5x faster than merging items into a sorted array. The GPU LSM supports the retrieval operations of lookup, count, and range query operations with an average rate of 75 M, 32 M and 23 M queries/s respectively. The trade-off for the dynamic updates is that the sorted array is almost twice as fast on retrievals. We believe that our GPU LSM is the first dynamic general-purpose dictionary data structure for the GPU.Comment: 11 pages, accepted to appear on the Proceedings of IEEE International Parallel and Distributed Processing Symposium (IPDPS'18

Sequence-Based Mapping and Genome Editing Reveal Mutations in Stickleback Hps5 Cause Oculocutaneous Albinism and the casper Phenotype.

Here, we present and characterize the spontaneous X-linked recessive mutation casper, which causes oculocutaneous albinism in threespine sticklebacks (Gasterosteus aculeatus). In humans, Hermansky-Pudlak syndrome results in pigmentation defects due to disrupted formation of the melanin-containing lysosomal-related organelle (LRO), the melanosome. casper mutants display not only reduced pigmentation of melanosomes in melanophores, but also reductions in the iridescent silver color from iridophores, while the yellow pigmentation from xanthophores appears unaffected. We mapped casper using high-throughput sequencing of genomic DNA from bulked casper mutants to a region of the stickleback X chromosome (chromosome 19) near the stickleback ortholog of Hermansky-Pudlak syndrome 5 (Hps5). casper mutants have an insertion of a single nucleotide in the sixth exon of Hps5, predicted to generate an early frameshift. Genome editing using CRISPR/Cas9 induced lesions in Hps5 and phenocopied the casper mutation. Injecting single or paired Hps5 guide RNAs revealed higher incidences of genomic deletions from paired guide RNAs compared to single gRNAs. Stickleback Hps5 provides a genetic system where a hemizygous locus in XY males and a diploid locus in XX females can be used to generate an easily scored visible phenotype, facilitating quantitative studies of different genome editing approaches. Lastly, we show the ability to better visualize patterns of fluorescent transgenic reporters in Hps5 mutant fish. Thus, Hps5 mutations present an opportunity to study pigmented LROs in the emerging stickleback model system, as well as a tool to aid in assaying genome editing and visualizing enhancer activity in transgenic fish

Efficient Management of Short-Lived Data

Motivated by the increasing prominence of loosely-coupled systems, such as mobile and sensor networks, which are characterised by intermittent connectivity and volatile data, we study the tagging of data with so-called expiration times. More specifically, when data are inserted into a database, they may be tagged with time values indicating when they expire, i.e., when they are regarded as stale or invalid and thus are no longer considered part of the database. In a number of applications, expiration times are known and can be assigned at insertion time. We present data structures and algorithms for online management of data tagged with expiration times. The algorithms are based on fully functional, persistent treaps, which are a combination of binary search trees with respect to a primary attribute and heaps with respect to a secondary attribute. The primary attribute implements primary keys, and the secondary attribute stores expiration times in a minimum heap, thus keeping a priority queue of tuples to expire. A detailed and comprehensive experimental study demonstrates the well-behavedness and scalability of the approach as well as its efficiency with respect to a number of competitors.Comment: switched to TimeCenter latex styl

Using Hashing to Solve the Dictionary Problem (In External Memory)

We consider the dictionary problem in external memory and improve the update time of the well-known buffer tree by roughly a logarithmic factor. For any \lambda >= max {lg lg n, log_{M/B} (n/B)}, we can support updates in time O(\lambda / B) and queries in sublogarithmic time, O(log_\lambda n). We also present a lower bound in the cell-probe model showing that our data structure is optimal. In the RAM, hash tables have been used to solve the dictionary problem faster than binary search for more than half a century. By contrast, our data structure is the first to beat the comparison barrier in external memory. Ours is also the first data structure to depart convincingly from the indivisibility paradigm

Markov chain sampling of the O(n)O(n) loop models on the infinite plane

It was recently proposed in https://journals.aps.org/pre/abstract/10.1103/PhysRevE.94.043322 [Herdeiro & Doyon Phys.,Rev.,E (2016)] a numerical method showing a precise sampling of the infinite plane 2d critical Ising model for finite lattice subsections. The present note extends the method to a larger class of models, namely the $O(n)$ loop gas models for $n \in (1,2]$. We argue that even though the Gibbs measure is non local, it is factorizable on finite subsections when sufficient information on the loops touching the boundaries is stored. Our results attempt to show that provided an efficient Markov chain mixing algorithm and an improved discrete lattice dilation procedure the planar limit of the $O(n)$ models can be numerically studied with efficiency similar to the Ising case. This confirms that scale invariance is the only requirement for the present numerical method to work.Comment: v2: added conclusion section, changes in introduction and appendice