Parallelizing Windowed Stream Joins in a Shared-Nothing Cluster

The availability of large number of processing nodes in a parallel and distributed computing environment enables sophisticated real time processing over high speed data streams, as required by many emerging applications. Sliding window stream joins are among the most important operators in a stream processing system. In this paper, we consider the issue of parallelizing a sliding window stream join operator over a shared nothing cluster. We propose a framework, based on fixed or predefined communication pattern, to distribute the join processing loads over the shared-nothing cluster. We consider various overheads while scaling over a large number of nodes, and propose solution methodologies to cope with the issues. We implement the algorithm over a cluster using a message passing system, and present the experimental results showing the effectiveness of the join processing algorithm.Comment: 11 page

The genetic architecture underlying the evolution of a rare piscivorous life history form in brown trout after secondary contact and strong introgression

Identifying the genetic basis underlying phenotypic divergence and reproductive isolation is a longstanding problem in evolutionary biology. Genetic signals of adaptation and reproductive isolation are often confounded by a wide range of factors, such as variation in demographic history or genomic features. Brown trout ( ) in the Loch Maree catchment, Scotland, exhibit reproductively isolated divergent life history morphs, including a rare piscivorous (ferox) life history form displaying larger body size, greater longevity and delayed maturation compared to sympatric benthivorous brown trout. Using a dataset of 16,066 SNPs, we analyzed the evolutionary history and genetic architecture underlying this divergence. We found that ferox trout and benthivorous brown trout most likely evolved after recent secondary contact of two distinct glacial lineages, and identified 33 genomic outlier windows across the genome, of which several have most likely formed through selection. We further identified twelve candidate genes and biological pathways related to growth, development and immune response potentially underpinning the observed phenotypic differences. The identification of clear genomic signals divergent between life history phenotypes and potentially linked to reproductive isolation, through size assortative mating, as well as the identification of the underlying demographic history, highlights the power of genomic studies of young species pairs for understanding the factors shaping genetic differentiation