SURGE: Continuous Detection of Bursty Regions Over a Stream of Spatial Objects

With the proliferation of mobile devices and location-based services, continuous generation of massive volume of streaming spatial objects (i.e., geo-tagged data) opens up new opportunities to address real-world problems by analyzing them. In this paper, we present a novel continuous bursty region detection problem that aims to continuously detect a bursty region of a given size in a specified geographical area from a stream of spatial objects. Specifically, a bursty region shows maximum spike in the number of spatial objects in a given time window. The problem is useful in addressing several real-world challenges such as surge pricing problem in online transportation and disease outbreak detection. To solve the problem, we propose an exact solution and two approximate solutions, and the approximation ratio is $\frac{1-\alpha}{4}$ in terms of the burst score, where $\alpha$ is a parameter to control the burst score. We further extend these solutions to support detection of top-$k$ bursty regions. Extensive experiments with real-world data are conducted to demonstrate the efficiency and effectiveness of our solutions

Finding event correlations in federated wireless sensor networks

Due to copyright restrictions, the access to the full text of this article is only available via subscription.Event correlation engines help us find events of interest inside raw sensor data streams and help reduce the data volume, simultaneously. This paper discusses some of the challenges faced in finding event correlations over federated wireless sensor networks (WSNs) including high data volumes, uncertain or missing data, application-specific dependencies and widely varying data ranges and sampling frequencies. Analysisover real geo-tracking data of moving objects confirms some of these challenges. Federation at the data layer above the WSNs is presented as a feasible alternative.TÜBİTAK ; IBM Shared University Research program ; European Commissio

Design and Implementation of a Middleware for Uniform, Federated and Dynamic Event Processing

In recent years, real-time processing of massive event streams has become an important topic in the area of data analytics. It will become even more important in the future due to cheap sensors, a growing amount of devices and their ubiquitous inter-connection also known as the Internet of Things (IoT). Academia, industry and the open source community have developed several event processing (EP) systems that allow users to define, manage and execute continuous queries over event streams. They achieve a significantly better performance than the traditional store-then-process'' approach in which events are first stored and indexed in a database. Because EP systems have different roots and because of the lack of standardization, the system landscape became highly heterogenous. Today's EP systems differ in APIs, execution behaviors and query languages. This thesis presents the design and implementation of a novel middleware that abstracts from different EP systems and provides a uniform API, execution behavior and query language to users and developers. As a consequence, the presented middleware overcomes the problem of vendor lock-in and different EP systems are enabled to cooperate with each other. In practice, event streams differ dramatically in volume and velocity. We show therefore how the middleware can connect to not only different EP systems, but also database systems and a native implementation. Emerging applications such as the IoT raise novel challenges and require EP to be more dynamic. We present extensions to the middleware that enable self-adaptivity which is needed in context-sensitive applications and those that deal with constantly varying sets of event producers and consumers. Lastly, we extend the middleware to fully support the processing of events containing spatial data and to be able to run distributed in the form of a federation of heterogenous EP systems

A Survey on IT-Techniques for a Dynamic Emergency Management in Large Infrastructures

This deliverable is a survey on the IT techniques that are relevant to the three use cases of the project EMILI. It describes the state-of-the-art in four complementary IT areas: Data cleansing, supervisory control and data acquisition, wireless sensor networks and complex event processing. Even though the deliverable’s authors have tried to avoid a too technical language and have tried to explain every concept referred to, the deliverable might seem rather technical to readers so far little familiar with the techniques it describes

Top-k spatial-keyword publish/subscribe over sliding window

© 2017, Springer-Verlag Berlin Heidelberg. With the prevalence of social media and GPS-enabled devices, a massive amount of geo-textual data have been generated in a stream fashion, leading to a variety of applications such as location-based recommendation and information dissemination. In this paper, we investigate a novel real-time top-k monitoring problem over sliding window of streaming data; that is, we continuously maintain the top-k most relevant geo-textual messages (e.g., geo-tagged tweets) for a large number of spatial-keyword subscriptions (e.g., registered users interested in local events) simultaneously. To provide the most recent information under controllable memory cost, sliding window model is employed on the streaming geo-textual data. To the best of our knowledge, this is the first work to study top-k spatial-keyword publish/subscribe over sliding window. A novel centralized system, called Skype (Top-kSpatial-keyword Publish/Subscribe), is proposed in this paper. In Skype, to continuously maintain top-k results for massive subscriptions, we devise a novel indexing structure upon subscriptions such that each incoming message can be immediately delivered on its arrival. To reduce the expensive top-k re-evaluation cost triggered by message expiration, we develop a novel cost-basedk-skyband technique to reduce the number of re-evaluations in a cost-effective way. Extensive experiments verify the great efficiency and effectiveness of our proposed techniques. Furthermore, to support better scalability and higher throughput, we propose a distributed version of Skype, namely DSkype, on top of Storm, which is a popular distributed stream processing system. With the help of fine-tuned subscription/message distribution mechanisms, DSkype can achieve orders of magnitude speed-up than its centralized version

Aggregate Analytic Window Query over Spatial Data

Analytic window query is a commonly used query in the relational databases. It answers the aggregations of data over a sliding window. For example, to get the average prices of a stock for each day. However, it is not supported in the spatial databases. Because the spatial data are not in a one-dimension space, there is no straightforward way to extend the original analytic window query to spatial databases. But these queries are useful and meaningful. For example, to find the average number of visits for all the POIs in the circle with a fixed radius for each POI as the centre. In this paper, we define the aggregate analytic window query over spatial data and propose algorithms for grid index and tree-index. We also analyze the complexity of the algorithms to prove they are efficient and practical

When Things Matter: A Data-Centric View of the Internet of Things

With the recent advances in radio-frequency identification (RFID), low-cost wireless sensor devices, and Web technologies, the Internet of Things (IoT) approach has gained momentum in connecting everyday objects to the Internet and facilitating machine-to-human and machine-to-machine communication with the physical world. While IoT offers the capability to connect and integrate both digital and physical entities, enabling a whole new class of applications and services, several significant challenges need to be addressed before these applications and services can be fully realized. A fundamental challenge centers around managing IoT data, typically produced in dynamic and volatile environments, which is not only extremely large in scale and volume, but also noisy, and continuous. This article surveys the main techniques and state-of-the-art research efforts in IoT from data-centric perspectives, including data stream processing, data storage models, complex event processing, and searching in IoT. Open research issues for IoT data management are also discussed

The design and development of multi-agent based RFID middleware system for data and devices management

Thesis (D. Tech. (Electrical Engineering)) - Central University of technology, Free State, 2012Radio frequency identification technology (RFID) has emerged as a key technology for automatic identification and promises to revolutionize business processes. While RFID technology adoption is improving rapidly, reliable and widespread deployment of this technology still faces many significant challenges. The key deployment challenges include how to use the simple, unreliable raw data generated by RFID deployments to make business decisions; and how to manage a large number of deployed RFID devices. In this thesis, a multi-agent based RFID middleware which addresses some of the RFID data and device management challenges was developed. The middleware developed abstracts the auto-identification applications from physical RFID device specific details and provides necessary services such as device management, data cleaning, event generation, query capabilities and event persistence. The use of software agent technology offers a more scalable and distributed system architecture for the proposed middleware. As part of a multi-agent system, application-independent domain ontology for RFID devices was developed. This ontology can be used or extended in any application interested with RFID domain ontology. In order to address the event processing tasks within the proposed middleware system, a temporal-based RFID data model which considers both applications’ temporal and spatial granules in the data model itself for efficient event processing was developed. The developed data model extends the conventional Entity-Relationship constructs by adding a time attribute to the model. By maintaining the history of events and state changes, the data model captures the fundamental RFID application logic within the data model. Hence, this new data model supports efficient generation of application level events, updating, querying and analysis of both recent and historical events. As part of the RFID middleware, an adaptive sliding-window based data cleaning scheme for reducing missed readings from RFID data streams (called WSTD) was also developed. The WSTD scheme models the unreliability of the RFID readings by viewing RFID streams as a statistical sample of tags in the physical world, and exploits techniques grounded in sampling theory to drive its cleaning processes. The WSTD scheme is capable of efficiently coping with both environmental variations and tag dynamics by automatically and continuously adapting its cleaning window size, based on observed readings