26,413 research outputs found
Parallelizing Windowed Stream Joins in a Shared-Nothing Cluster
The availability of large number of processing nodes in a parallel and
distributed computing environment enables sophisticated real time processing
over high speed data streams, as required by many emerging applications.
Sliding window stream joins are among the most important operators in a stream
processing system. In this paper, we consider the issue of parallelizing a
sliding window stream join operator over a shared nothing cluster. We propose a
framework, based on fixed or predefined communication pattern, to distribute
the join processing loads over the shared-nothing cluster. We consider various
overheads while scaling over a large number of nodes, and propose solution
methodologies to cope with the issues. We implement the algorithm over a
cluster using a message passing system, and present the experimental results
showing the effectiveness of the join processing algorithm.Comment: 11 page
Exploiting citation networks for large-scale author name disambiguation
We present a novel algorithm and validation method for disambiguating author
names in very large bibliographic data sets and apply it to the full Web of
Science (WoS) citation index. Our algorithm relies only upon the author and
citation graphs available for the whole period covered by the WoS. A pair-wise
publication similarity metric, which is based on common co-authors,
self-citations, shared references and citations, is established to perform a
two-step agglomerative clustering that first connects individual papers and
then merges similar clusters. This parameterized model is optimized using an
h-index based recall measure, favoring the correct assignment of well-cited
publications, and a name-initials-based precision using WoS metadata and
cross-referenced Google Scholar profiles. Despite the use of limited metadata,
we reach a recall of 87% and a precision of 88% with a preference for
researchers with high h-index values. 47 million articles of WoS can be
disambiguated on a single machine in less than a day. We develop an h-index
distribution model, confirming that the prediction is in excellent agreement
with the empirical data, and yielding insight into the utility of the h-index
in real academic ranking scenarios.Comment: 14 pages, 5 figure
Analyzing Learned Molecular Representations for Property Prediction
Advancements in neural machinery have led to a wide range of algorithmic
solutions for molecular property prediction. Two classes of models in
particular have yielded promising results: neural networks applied to computed
molecular fingerprints or expert-crafted descriptors, and graph convolutional
neural networks that construct a learned molecular representation by operating
on the graph structure of the molecule. However, recent literature has yet to
clearly determine which of these two methods is superior when generalizing to
new chemical space. Furthermore, prior research has rarely examined these new
models in industry research settings in comparison to existing employed models.
In this paper, we benchmark models extensively on 19 public and 16 proprietary
industrial datasets spanning a wide variety of chemical endpoints. In addition,
we introduce a graph convolutional model that consistently matches or
outperforms models using fixed molecular descriptors as well as previous graph
neural architectures on both public and proprietary datasets. Our empirical
findings indicate that while approaches based on these representations have yet
to reach the level of experimental reproducibility, our proposed model
nevertheless offers significant improvements over models currently used in
industrial workflows
- …