Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture

We present the architecture behind Twitter's real-time related query suggestion and spelling correction service. Although these tasks have received much attention in the web search literature, the Twitter context introduces a real-time "twist": after significant breaking news events, we aim to provide relevant results within minutes. This paper provides a case study illustrating the challenges of real-time data processing in the era of "big data". We tell the story of how our system was built twice: our first implementation was built on a typical Hadoop-based analytics stack, but was later replaced because it did not meet the latency requirements necessary to generate meaningful real-time results. The second implementation, which is the system deployed in production, is a custom in-memory processing engine specifically designed for the task. This experience taught us that the current typical usage of Hadoop as a "big data" platform, while great for experimentation, is not well suited to low-latency processing, and points the way to future work on data analytics platforms that can handle "big" as well as "fast" data

The X-ray emission of the gamma Cassiopeiae stars

Long considered as the "odd man out" among X-ray emitting Be stars, \gamma Cas (B0.5e IV) is now recognized as the prototype of a class of stars that emit hard thermal X-rays. Our classification differs from the historical use of the term "gamma Cas stars" defined from optical properties alone. The luminosity output of this class contributes significantly to the hard X-ray production in massive stars in the Galaxy. The gamma Cas stars have light curves showing variability on a few broadly-defined timescales and spectra indicative of an optically thin plasma consisting of one or more hot thermal components. By now 9--13 Galactic \approx B0-1.5e main sequence stars are judged to be members or candidate members of the \gamma Cas class. Conservative criteria for this designation are for a \approxB0-1.5e III-V star to have an X-ray luminosity of 10^{32}--10^{33} ergs s^{-1}, a hot thermal spectrum containing the short wavelength Ly \alpha FeXXV and FeXXVI lines and the fluorescence FeK feature all in emission. If thermality cannot be demonstrated, for example from either the presence of these Ly \alpha lines or curvature of the hard continuum; these are the gamma Cas candidates. We discuss the history of the discovery of the complicated characteristics of the variability in the optical, UV, and X-ray domains, leading to suggestions for the physical cause of the production of hard X-rays. These include scenarios in which matter from the Be star accretes onto a degenerate secondary star and interactions between magnetic fields on the Be star and its decretion disk. The greatest aid to the choice of the causal mechanism is the temporal correlations of X-ray light curves and spectra with diagnostics in the optical and UV wavebands. We show why the magnetic star-disk interaction scenario is the most tenable explanation for the creation of hard X-rays on these stars.Comment: Review paper for "X-ray Emissions from Hot Stars and their Winds" compendium to be published by Advances in Space Research in mid-2016. Paper is comprised of 66 pages, 15 figure

Proceedings of 2007 Kentucky Water Resources Annual Symposium

This conference was planned and conducted as part of the state water resources research annual program with the support and collaboration of the Department of the Interior, U.S. Geological Survey and the University of Kentucky Research Foundation, under Grant Agreement Number 06HQGR0087. The views and conclusions contained in this document and presented at the symposium are those of the abstract authors and presenters and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Government or other symposium organizers and sponsors

Finding event correlations in federated wireless sensor networks

Due to copyright restrictions, the access to the full text of this article is only available via subscription.Event correlation engines help us find events of interest inside raw sensor data streams and help reduce the data volume, simultaneously. This paper discusses some of the challenges faced in finding event correlations over federated wireless sensor networks (WSNs) including high data volumes, uncertain or missing data, application-specific dependencies and widely varying data ranges and sampling frequencies. Analysisover real geo-tracking data of moving objects confirms some of these challenges. Federation at the data layer above the WSNs is presented as a feasible alternative.TÜBİTAK ; IBM Shared University Research program ; European Commissio